From 506905fd90babb3cac2d250941d04a108141a5d9 Mon Sep 17 00:00:00 2001 From: DJ Enriquez Date: Thu, 17 Dec 2015 22:30:10 -0800 Subject: [PATCH 1/5] CloudFront-s3-middleware Adding a more detailed document regarding how to use CloudFront as middleware for an s3 backed registry. Signed-off-by: DJ Enriquez --- docs/cloudfronts3.md | 47 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+) create mode 100644 docs/cloudfronts3.md diff --git a/docs/cloudfronts3.md b/docs/cloudfronts3.md new file mode 100644 index 00000000..a1b00faf --- /dev/null +++ b/docs/cloudfronts3.md @@ -0,0 +1,47 @@ +# CloudFront as Middleware /w S3 backend + +## Use Case +Adding CloudFront as a middleware for your registry can dramatically improve pull times. Your registry will have the ability to retrieve your images from edge servers, rather than the geographically limited location of your s3 bucket. The farther your registry is from your bucket, the more improvements you will see. See [Amazon CloudFront](https://aws.amazon.com/cloudfront/details/). + +## Configuring CloudFront for Distribution +If you are unfamiliar with creating a CloudFront distribution, see [Getting Started with Cloudfront](http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/GettingStarted.html). + +Defaults can be kept in most areas except: + +#### Origin: + +The CloudFront distribution must be created such that the `Origin Path` is set to the directory level of the root "docker" key in S3. If your registry exists on the root of the bucket, this path should be left blank. + +#### Behaviors: + - Viewer Protocol Policy: HTTPS Only + - Allowed HTTP Methods: GET, HEAD, OPTIONS, PUT, POST, PATCH, DELETE + - Cached HTTP Methods: OPTIONS (checked) + - Restrict Viewer Access (Use Signed URLs or Signed Cookies): Yes + - Trusted Signers: Self (Can add other accounts as long as you have access to CloudFront Key Pairs for those additional accounts) + +## Registry configuration +Here the `middleware` option is used. It is still important to keep the `storage` option as CloudFront will only handle `pull` actions; `push` actions are still directly written to S3. + +The following example shows what you will need at minimum: +``` +. +. +. +storage: + s3: + region: us-east-1 + bucket: docker.myregistry.com +middleware: + storage: + - name: cloudfront + options: + baseurl: https://abcdefghijklmn.cloudfront.net/ + privatekey: /etc/docker/cloudfront/pk-ABCEDFGHIJKLMNOPQRST.pem + keypairid: ABCEDFGHIJKLMNOPQRST +. +. +. +``` + +## CloudFront Key-Pair +A CloudFront key-pair is required for all AWS accounts needing access to your CloudFront distribution. For information, please see [Creating CloudFront Key Pairs](http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/private-content-trusted-signers.html#private-content-creating-cloudfront-key-pairs). \ No newline at end of file From 4a9f6c7975dd00fa85de16a084fed0734cc83374 Mon Sep 17 00:00:00 2001 From: DJ Enriquez Date: Fri, 18 Dec 2015 11:55:19 -0800 Subject: [PATCH 2/5] Applying recommended changes - Corrected headings - Added blank line after headings - Changed example snippet to use `...` Signed-off-by: DJ Enriquez --- docs/cloudfronts3.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/docs/cloudfronts3.md b/docs/cloudfronts3.md index a1b00faf..2536f6e1 100644 --- a/docs/cloudfronts3.md +++ b/docs/cloudfronts3.md @@ -1,18 +1,20 @@ # CloudFront as Middleware /w S3 backend ## Use Case + Adding CloudFront as a middleware for your registry can dramatically improve pull times. Your registry will have the ability to retrieve your images from edge servers, rather than the geographically limited location of your s3 bucket. The farther your registry is from your bucket, the more improvements you will see. See [Amazon CloudFront](https://aws.amazon.com/cloudfront/details/). ## Configuring CloudFront for Distribution + If you are unfamiliar with creating a CloudFront distribution, see [Getting Started with Cloudfront](http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/GettingStarted.html). Defaults can be kept in most areas except: -#### Origin: +### Origin: The CloudFront distribution must be created such that the `Origin Path` is set to the directory level of the root "docker" key in S3. If your registry exists on the root of the bucket, this path should be left blank. -#### Behaviors: +### Behaviors: - Viewer Protocol Policy: HTTPS Only - Allowed HTTP Methods: GET, HEAD, OPTIONS, PUT, POST, PATCH, DELETE - Cached HTTP Methods: OPTIONS (checked) @@ -20,13 +22,12 @@ The CloudFront distribution must be created such that the `Origin Path` is set t - Trusted Signers: Self (Can add other accounts as long as you have access to CloudFront Key Pairs for those additional accounts) ## Registry configuration + Here the `middleware` option is used. It is still important to keep the `storage` option as CloudFront will only handle `pull` actions; `push` actions are still directly written to S3. The following example shows what you will need at minimum: ``` -. -. -. +... storage: s3: region: us-east-1 @@ -38,10 +39,9 @@ middleware: baseurl: https://abcdefghijklmn.cloudfront.net/ privatekey: /etc/docker/cloudfront/pk-ABCEDFGHIJKLMNOPQRST.pem keypairid: ABCEDFGHIJKLMNOPQRST -. -. -. +... ``` ## CloudFront Key-Pair -A CloudFront key-pair is required for all AWS accounts needing access to your CloudFront distribution. For information, please see [Creating CloudFront Key Pairs](http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/private-content-trusted-signers.html#private-content-creating-cloudfront-key-pairs). \ No newline at end of file + +A CloudFront key-pair is required for all AWS accounts needing access to your CloudFront distribution. For information, please see [Creating CloudFront Key Pairs](http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/private-content-trusted-signers.html#private-content-creating-cloudfront-key-pairs). From 8108539037d4324537da4afdb90fcc605bab64d1 Mon Sep 17 00:00:00 2001 From: DJ Enriquez Date: Fri, 18 Dec 2015 12:48:13 -0800 Subject: [PATCH 3/5] Relocating documentation to s3 storage-driver doc Since the CloudFront middleware does not work without an S3 backend, it became obvious that this documentation should exist within the S3 storage-driver documentation. Signed-off-by: DJ Enriquez --- docs/cloudfronts3.md | 47 --------------------------------- docs/storage-drivers/s3.md | 53 ++++++++++++++++++++++++++++++++++++-- 2 files changed, 51 insertions(+), 49 deletions(-) delete mode 100644 docs/cloudfronts3.md diff --git a/docs/cloudfronts3.md b/docs/cloudfronts3.md deleted file mode 100644 index 2536f6e1..00000000 --- a/docs/cloudfronts3.md +++ /dev/null @@ -1,47 +0,0 @@ -# CloudFront as Middleware /w S3 backend - -## Use Case - -Adding CloudFront as a middleware for your registry can dramatically improve pull times. Your registry will have the ability to retrieve your images from edge servers, rather than the geographically limited location of your s3 bucket. The farther your registry is from your bucket, the more improvements you will see. See [Amazon CloudFront](https://aws.amazon.com/cloudfront/details/). - -## Configuring CloudFront for Distribution - -If you are unfamiliar with creating a CloudFront distribution, see [Getting Started with Cloudfront](http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/GettingStarted.html). - -Defaults can be kept in most areas except: - -### Origin: - -The CloudFront distribution must be created such that the `Origin Path` is set to the directory level of the root "docker" key in S3. If your registry exists on the root of the bucket, this path should be left blank. - -### Behaviors: - - Viewer Protocol Policy: HTTPS Only - - Allowed HTTP Methods: GET, HEAD, OPTIONS, PUT, POST, PATCH, DELETE - - Cached HTTP Methods: OPTIONS (checked) - - Restrict Viewer Access (Use Signed URLs or Signed Cookies): Yes - - Trusted Signers: Self (Can add other accounts as long as you have access to CloudFront Key Pairs for those additional accounts) - -## Registry configuration - -Here the `middleware` option is used. It is still important to keep the `storage` option as CloudFront will only handle `pull` actions; `push` actions are still directly written to S3. - -The following example shows what you will need at minimum: -``` -... -storage: - s3: - region: us-east-1 - bucket: docker.myregistry.com -middleware: - storage: - - name: cloudfront - options: - baseurl: https://abcdefghijklmn.cloudfront.net/ - privatekey: /etc/docker/cloudfront/pk-ABCEDFGHIJKLMNOPQRST.pem - keypairid: ABCEDFGHIJKLMNOPQRST -... -``` - -## CloudFront Key-Pair - -A CloudFront key-pair is required for all AWS accounts needing access to your CloudFront distribution. For information, please see [Creating CloudFront Key Pairs](http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/private-content-trusted-signers.html#private-content-creating-cloudfront-key-pairs). diff --git a/docs/storage-drivers/s3.md b/docs/storage-drivers/s3.md index 3ce5df23..b1d74d47 100644 --- a/docs/storage-drivers/s3.md +++ b/docs/storage-drivers/s3.md @@ -21,7 +21,7 @@ An implementation of the `storagedriver.StorageDriver` interface which uses Amaz `region`: The name of the aws region in which you would like to store objects (for example `us-east-1`). For a list of regions, you can look at http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html -`bucket`: The name of your s3 bucket where you wish to store objects (needs to already be created prior to driver initialization). +`bucket`: The name of your S3 bucket where you wish to store objects (needs to already be created prior to driver initialization). `encrypt`: (optional) Whether you would like your data encrypted on the server side (defaults to false if not specified). @@ -29,6 +29,55 @@ An implementation of the `storagedriver.StorageDriver` interface which uses Amaz `v4auth`: (optional) Whether you would like to use aws signature version 4 with your requests. This defaults to false if not specified (note that the eu-central-1 region does not work with version 2 signatures, so the driver will error out if initialized with this region and v4auth set to false) -`chunksize`: (optional) The default part size for multipart uploads (performed by WriteStream) to s3. The default is 10 MB. Keep in mind that the minimum part size for s3 is 5MB. You might experience better performance for larger chunk sizes depending on the speed of your connection to s3. +`chunksize`: (optional) The default part size for multipart uploads (performed by WriteStream) to S3. The default is 10 MB. Keep in mind that the minimum part size for S3 is 5MB. You might experience better performance for larger chunk sizes depending on the speed of your connection to S3. `rootdirectory`: (optional) The root directory tree in which all registry files will be stored. Defaults to the empty string (bucket root). + +# CloudFront as Middleware /w S3 backend + +## Use Case + +Adding CloudFront as a middleware for your S3 backed registry can dramatically improve pull times. Your registry will have the ability to retrieve your images from edge servers, rather than the geographically limited location of your S3 bucket. The farther your registry is from your bucket, the more improvements you will see. See [Amazon CloudFront](https://aws.amazon.com/cloudfront/details/). + +## Configuring CloudFront for Distribution + +If you are unfamiliar with creating a CloudFront distribution, see [Getting Started with Cloudfront](http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/GettingStarted.html). + +Defaults can be kept in most areas except: + +### Origin: + +The CloudFront distribution must be created such that the `Origin Path` is set to the directory level of the root "docker" key in S3. If your registry exists on the root of the bucket, this path should be left blank. + +### Behaviors: + + - Viewer Protocol Policy: HTTPS Only + - Allowed HTTP Methods: GET, HEAD, OPTIONS, PUT, POST, PATCH, DELETE + - Cached HTTP Methods: OPTIONS (checked) + - Restrict Viewer Access (Use Signed URLs or Signed Cookies): Yes + - Trusted Signers: Self (Can add other accounts as long as you have access to CloudFront Key Pairs for those additional accounts) + +## Registry configuration + +Here the `middleware` option is used. It is still important to keep the `storage` option as CloudFront will only handle `pull` actions; `push` actions are still directly written to S3. + +The following example shows what you will need at minimum: +``` +... +storage: + s3: + region: us-east-1 + bucket: docker.myregistry.com +middleware: + storage: + - name: cloudfront + options: + baseurl: https://abcdefghijklmn.cloudfront.net/ + privatekey: /etc/docker/cloudfront/pk-ABCEDFGHIJKLMNOPQRST.pem + keypairid: ABCEDFGHIJKLMNOPQRST +... +``` + +## CloudFront Key-Pair + +A CloudFront key-pair is required for all AWS accounts needing access to your CloudFront distribution. For information, please see [Creating CloudFront Key Pairs](http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/private-content-trusted-signers.html#private-content-creating-cloudfront-key-pairs). \ No newline at end of file From 5dfc78997dc9d817fd23eb4b4670db25c6bf8f51 Mon Sep 17 00:00:00 2001 From: DJ Enriquez Date: Tue, 22 Dec 2015 11:33:28 -0900 Subject: [PATCH 4/5] Adding suggested changes Adding changed suggested by moxiegirl. Signed-off-by: DJ Enriquez --- docs/storage-drivers/s3.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/storage-drivers/s3.md b/docs/storage-drivers/s3.md index b1d74d47..fd68125e 100644 --- a/docs/storage-drivers/s3.md +++ b/docs/storage-drivers/s3.md @@ -21,7 +21,7 @@ An implementation of the `storagedriver.StorageDriver` interface which uses Amaz `region`: The name of the aws region in which you would like to store objects (for example `us-east-1`). For a list of regions, you can look at http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html -`bucket`: The name of your S3 bucket where you wish to store objects (needs to already be created prior to driver initialization). +`bucket`: The name of your S3 bucket where you wish to store objects. The bucket must exist prior to the driver initialization. `encrypt`: (optional) Whether you would like your data encrypted on the server side (defaults to false if not specified). @@ -29,7 +29,7 @@ An implementation of the `storagedriver.StorageDriver` interface which uses Amaz `v4auth`: (optional) Whether you would like to use aws signature version 4 with your requests. This defaults to false if not specified (note that the eu-central-1 region does not work with version 2 signatures, so the driver will error out if initialized with this region and v4auth set to false) -`chunksize`: (optional) The default part size for multipart uploads (performed by WriteStream) to S3. The default is 10 MB. Keep in mind that the minimum part size for S3 is 5MB. You might experience better performance for larger chunk sizes depending on the speed of your connection to S3. +`chunksize`: (optional) The default part size for multipart uploads (performed by WriteStream) to S3. The default is 10 MB. Keep in mind that the minimum part size for S3 is 5MB. Depending on the speed of your connection to S3, a larger chunk size may result in better performance. `rootdirectory`: (optional) The root directory tree in which all registry files will be stored. Defaults to the empty string (bucket root). From b49c4fff95e2f6ee88e26148316c24e6e9adaa29 Mon Sep 17 00:00:00 2001 From: DJ Enriquez Date: Tue, 5 Jan 2016 15:22:35 -0800 Subject: [PATCH 5/5] Applying recommended changes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Adding clarification to the “chunksize” definition - Changing short-hand “/w” to “with” Signed-off-by: DJ Enriquez --- docs/storage-drivers/s3.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/storage-drivers/s3.md b/docs/storage-drivers/s3.md index fd68125e..0f4326fa 100644 --- a/docs/storage-drivers/s3.md +++ b/docs/storage-drivers/s3.md @@ -29,11 +29,11 @@ An implementation of the `storagedriver.StorageDriver` interface which uses Amaz `v4auth`: (optional) Whether you would like to use aws signature version 4 with your requests. This defaults to false if not specified (note that the eu-central-1 region does not work with version 2 signatures, so the driver will error out if initialized with this region and v4auth set to false) -`chunksize`: (optional) The default part size for multipart uploads (performed by WriteStream) to S3. The default is 10 MB. Keep in mind that the minimum part size for S3 is 5MB. Depending on the speed of your connection to S3, a larger chunk size may result in better performance. +`chunksize`: (optional) The default part size for multipart uploads (performed by WriteStream) to S3. The default is 10 MB. Keep in mind that the minimum part size for S3 is 5MB. Depending on the speed of your connection to S3, a larger chunk size may result in better performance; faster connections will benefit from larger chunk sizes. `rootdirectory`: (optional) The root directory tree in which all registry files will be stored. Defaults to the empty string (bucket root). -# CloudFront as Middleware /w S3 backend +# CloudFront as Middleware with S3 backend ## Use Case