Commit graph

214 commits

Author SHA1 Message Date
Richard Scothern
94935f39bc Add pull through cache functionality to the Registry which can be configured
with a new `proxy` section in the configuration file.

Create a new registry type which delegates storage to a proxyBlobStore
and proxyManifestStore.  These stores will pull through data if not present
locally.  proxyBlobStore takes care not to write duplicate data to disk.

Add a scheduler to cleanup expired content. The scheduler runs as a background
goroutine.  When a blob or manifest is pulled through from the remote registry,
an entry is added to the scheduler with a TTL.  When the TTL expires the
scheduler calls a pre-specified function to remove the fetched resource.

Add token authentication to the registry middleware.  Get a token at startup
and preload the credential store with the username and password supplied in the
config file.

Allow resumable digest functionality to be disabled at runtime and disable
it when the registry is a pull through cache.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-08-04 16:09:55 -07:00
Stephen Day
ef1c72b978 Merge pull request #805 from RichardScothern/811
Spelling corrections
2015-08-04 14:50:32 -07:00
Aaron Lehmann
3a414deddb Merge pull request #514 from denverdino/master
Storage Driver for Aliyun OSS
2015-08-04 12:01:02 -07:00
Li Yi
d7d34a6c5b Add the OSS link
Change-Id: I940fb50f467ce6fc5fbdd9ceb0f9d848e422ced7
Signed-off-by: Li Yi <denverdino@gmail.com>
2015-08-04 09:18:18 +08:00
Richard Scothern
fb9662ad5a Add blob delete entry to api description and regenerate api.md
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-08-03 11:59:19 -07:00
Richard Scothern
bffa20d6bb Spelling corrections
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-07-31 16:43:11 -07:00
Alex Chan
51bd34eaed Fix a few typos in the docs
Signed-off-by: Alex Chan <alex.chan@metaswitch.com>
2015-07-31 13:36:43 +01:00
tgic
3eaab7da95 add link to Aliyun OSS doc
Signed-off-by: tgic <farmer1992@gmail.com>
2015-07-31 13:22:56 +08:00
tgic
76174ad57e update doc add endpoint 2015-07-31 13:22:55 +08:00
Li Yi
c3b42db014 Add the secure access with HTTPS
Signed-off-by: Li Yi <denverdino@gmail.com>
2015-07-31 13:22:54 +08:00
Li Yi
9e4975d8ff Support OSS driver
Signed-off-by: Li Yi <denverdino@gmail.com>
2015-07-31 13:22:54 +08:00
Aaron Lehmann
6c2ef78aa7 Rename dev-config.yml and example-config.yml to config-dev.yml and config-example.yml
Better for sort order.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-07-30 15:51:50 -07:00
Stephen Day
ad1b7d8f35 Merge pull request #740 from stevvooe/disable-storage-redirects
Allow disabling of storage driver redirects
2015-07-30 14:28:31 -07:00
Richard Scothern
7eabb7800e Merge pull request #790 from aaronlehmann/config-files
Sample config file changes
2015-07-30 13:20:43 -07:00
Aaron Lehmann
f0c5284d2d Sample config file changes
Rename config.yml to dev-config.yml

Add example-config.yml, a simple configuration file for the official
This was originally made for the the distribution-library-image repo,
but is being moved here to make sure it stays in sync.

Update Dockerfile and docs for the rename.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-07-30 12:14:39 -07:00
Dave Tucker
9348141616 Fix dead links in configuration.md
Fixes #672

Signed-off-by: Dave Tucker <dt@docker.com>
2015-07-30 17:56:04 +01:00
Aaron Lehmann
70d9a9d543 Automatically generate a HTTP secret if none is provided
Log a warning if the registry generates its own secret.

Update configuration doc, and remove the default secret from the
development config file.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-07-29 14:28:56 -07:00
Derek McGowan
0355c3026c Merge pull request #744 from aaronlehmann/manifest-put-response-code
Manifest PUT should return 201 Created
2015-07-28 10:42:54 -07:00
Olivier Gambier
549a3921a4 Merge pull request #752 from stevvooe/add-missing-ampersands
Correctly generate URL examples specifications
2015-07-24 22:05:22 -07:00
Stephen J Day
dcbf2967fd Correctly generate URL examples specifications
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-07-24 19:00:25 -07:00
Stephen J Day
9f9a7f230b Allow disabling of starage driver redirects
Storage drivers can implement a method called URLFor which can return a direct
url for a given path. The functionality allows the registry to direct clients
to download content directly from the backend storage. This is commonly used
with s3 and cloudfront. Under certain conditions, such as when the registry is
not local to the backend, these redirects can hurt performance and waste
incoming bandwidth on pulls. This feature addition allows one to disable this
feature, if required.

Signed-off-by: Stephen J Day <stephen.day@docker.com>

Conflicts:
	configuration/configuration.go
	registry/handlers/app.go
	registry/storage/catalog_test.go
	registry/storage/manifeststore_test.go
	registry/storage/registry.go
2015-07-24 16:59:35 -07:00
Stephen Day
d1dfac4b98 Revert "Fix typo" 2015-07-24 13:28:10 -07:00
Stephen Day
9e9e9a2859 Merge pull request #677 from RichardScothern/soft-delete-remove-links
Manifest and layer soft deletion
2015-07-24 13:02:36 -07:00
Aaron Lehmann
cf32056218 Manifest PUT should return 201 Created
Change handler, update descriptors table, regenerate API spec, and
update test.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-07-24 12:58:16 -07:00
Olivier Gambier
d7fb111442 Merge pull request #733 from dv/patch-1
Update authentication.md
2015-07-24 10:59:02 -07:00
Richard
9c1dd69439 Manifest and layer soft deletion.
Implement the delete API by implementing soft delete for layers
and blobs by removing link files and updating the blob descriptor
cache.  Deletion is configurable - if it is disabled API calls
will return an unsupported error.

We invalidate the blob descriptor cache by changing the linkedBlobStore's
blobStatter to a blobDescriptorService and naming it blobAccessController.

Delete() is added throughout the relevant API to support this functionality.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-07-24 09:57:20 -07:00
David Verhasselt
d3b59fc0ba Add paragraph on manual set-up
Signed-off-by: David Verhasselt <david@crowdway.com>

Add clarifying requirement of nginx 1.7.5 to authentication.md file.

Signed-off-by: David Verhasselt <david@crowdway.com>

Revert "Update authentication.md"

This reverts commit a79c28f4376fad333edd94e9b25211830e63abbd.

Add paragraph on manual set-up
2015-07-24 13:28:26 +03:00
Jianqing Wang
8c98f97a0d Fix typo
Signed-off-by: tsing <tsing@nicescale.com>
2015-07-24 11:43:28 +08:00
Stephen Day
984037f7fc Merge pull request #729 from stevvooe/pagination-spec-cleanup
Clean up pagination specification
2015-07-23 15:46:32 -07:00
Olivier Gambier
2445340f37 Merge pull request #730 from stevvooe/lettered-spec-revisions
Identify V2 API changes by letter
2015-07-23 10:42:25 -07:00
Stephen Day
34e5b1849c Merge pull request #493 from nevermosby/storage-support-openstack-swift
Storage support openstack swift
2015-07-22 23:53:36 -07:00
Stephen J Day
c5792bf87e Identify V2 API changes by letter
To avoid confusion with the registry version number or other project versions,
the specification updates are now lettered.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-07-22 20:49:27 -07:00
Stephen J Day
88831b0523 Clean up pagination specification
Some missing descriptions and error code for tags pagination was cleaned up to
ensure clarity. Specifically, we ensure the request variations are named and
the proper error codes are included.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-07-22 20:00:28 -07:00
Stephen J Day
006214d902 Paginate catalog and tag results with Link header
Move the specification to use a Link header, rather than a "next" entry in the
json results. This prevents requiring clients from parsing the request body to
issue the next request. It also ensures that the returned response body does
not change in between requests.

The ordering of the specification has been slightly tweaked, as well. Listing
image tags has been moved after the catalog specification. Tag pagination now
heavily references catalog pagination.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-07-21 21:45:14 -07:00
Stephen J Day
aebe850f73 Catalog V2 API specification proposal
This contains a proposal for a catalog API, provided access to the internal
contents of a registry instance. The API endpoint is prefixed with an
underscore, which is illegal in images names, to prevent collisions with
repositories names. To avoid issues with large result sets, a paginated version
of the API is proposed. We make an addition to the tags API to support
pagination to ensure the specification is conistent.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-07-21 21:45:14 -07:00
davidli
b5c7001bc9 Update configuration doc for authurl with adding the Openstack Identity v1.0 example
Signed-off-by: Li Wenquan <wenquan.li@hp.com>
2015-07-21 23:55:10 +02:00
Olivier Jacques
470f84078e Swift documentation updates
Signed-off-by: Olivier Jacques <olivier.jacques@hp.com>
2015-07-21 23:55:10 +02:00
Sylvain Baubeau
3699edcc93 Add metadata and use table for Swift driver documentation
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
davidli
af99dbd6bf Add support for Openstack Identity v3 API
Signed-off-by: Li Wenquan <wenquan.li@hp.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
5f6a07756f Fix path to swift storage driver documentation
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
davidli
f555defdb7 Add the reference for openstack swift in the documentation.
Signed-off-by: Li Wenquan <wenquan.li@hp.com>
2015-07-21 23:55:08 +02:00
Sylvain Baubeau
ea7c53df08 Add Openstack Swift storage driver
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:08 +02:00
Stephen Day
7333361596 Merge pull request #717 from kyzh/patch-2
Change confusing enpoint name
2015-07-21 13:58:53 -07:00
Florentin Raud
3f33d20b2a Change confusing enpoint name
Since the actual port is 5003, it would make sense to name it local-5003 instead of local-8082

Signed-off-by: Florentin Raud <florentin.raud@gmail.com>
2015-07-21 09:05:27 +01:00
Stephen Day
856638e299 Merge pull request #715 from moxiegirl/update-docs-dockerfile
Updating to use base image from master
2015-07-20 15:44:03 -07:00
Florentin Raud
96a26a7425 Fix a broken link
The link was pointing to deployement.md, while the file is deploying.md

Note: 
Currently the problem is not visible on https://docs.docker.com/registry/authentication/
The issue is visible on https://github.com/docker/distribution/blob/master/docs/authentication.md
2015-07-20 14:59:54 +01:00
Mary Anthony
8b4e4bb3a6 Updating to use base image from master
Signed-off-by: Mary Anthony <mary@docker.com>
2015-07-19 16:11:05 -07:00
Stephen J Day
26b7fe4a91 Use "Size" field to describe blobs over "Length"
After consideration, we've changed the main descriptor field name to for number
of bytes to "size" to match convention. While this may be a subjective
argument, commonly we refer to files by their "size" rather than their
"length". This will match other conventions, like `(FileInfo).Size()` and
methods on `io.SizeReaderAt`. Under more broad analysis, this argument doesn't
necessarily hold up. If anything, "size" is shorter than "length".

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-07-17 17:07:11 -07:00
Ayose Cazorla
dbf3ce3cde Fix link to token specification
Link was broken in cf9b4ab5e9

Signed-off-by: Ayose <ayosec@gmail.com>
2015-07-16 04:51:34 +01:00
Stephen Day
2451dd1c32 Merge pull request #691 from ojacques/master
docs: fix yaml sections formatting at docs.docker.com
2015-07-13 14:16:21 -07:00
Stephen Day
171ed44bf9 Merge pull request #686 from rxbynerd/include-intermediate-tls-config-in-docs
Include configuration explanation for intermediate TLS certificates
2015-07-13 14:08:14 -07:00
moxiegirl
79a4ca2abe Merge pull request #682 from moxiegirl/fix-issue-s681
Fixing old ref to DHE to DTR fixes #681
2015-07-11 19:28:07 -07:00
Olivier Jacques
f03e25221f Fix indentation for several subs
Signed-off-by: Olivier Jacques <olivier.jacques@hp.com>
2015-07-11 10:38:39 +02:00
Olivier Jacques
b3683863dd docs: fix yaml sections format at docs.docker.com
yaml sections in the documentation does not display well on
docs.docker.com. This is due to the syntax highlighting
which uses highlight.js and does not support yaml
currently.
The fix is to remove triple back ticks and indent instead.
We loose yaml syntax highlighting on github, but it displays
an acceptable version on both github and docs.docker.com.

Signed-off-by: Olivier Jacques <olivier.jacques@hp.com>
2015-07-10 23:22:06 +02:00
Luke Carpenter
03db92abb3 update copy with content from @moxiegirl
Signed-off-by: Luke Carpenter <x@rubynerd.net>
2015-07-10 22:10:25 +01:00
Stephen J Day
b54cc5ed47 Allow single character repository names
The main goal of this changeset is to allow repository name components to
consist of a single character. The number of components allowed and the slash
separation requirements have also been clarified.

To go along with this simplification, errant constants and unneeded error types
have been removed.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-07-10 12:10:42 -06:00
Luke Carpenter
3389ca8fa6 Remove mention of a third-party service
Signed-off-by: Luke Carpenter <x@rubynerd.net>
2015-07-09 23:40:32 +01:00
Luke Carpenter
58e5c619ac Include configuration explanation for intermediate TLS certificates
Intermediate certificates are issued by TLS providers who themselves are
an intermediate of a certificate in the trust store. Therefore, to prove
the chain of trust is valid, you need to include their certificate as
well as yours when you send your certificate to the client.

Contrary to what I said in issue #683, distribution can handle these
certificate bundles like nginx. As discussed in #docker-distribution,
I have updated the deployment documentation (which recommends the use of
a TLS certificate from a provider) to include instructions on how to
handle the intermediate certificate when a user is configuring
distribution.

Signed-off-by: Luke Carpenter <x@rubynerd.net>
2015-07-09 23:33:08 +01:00
Mary Anthony
fa32197c81 Fixing old ref to DHE to DTR fixes #681
Signed-off-by: Mary Anthony <mary@docker.com>
2015-07-08 14:05:49 -07:00
Stephen Day
c56e28826e Merge pull request #670 from stevvooe/remove-ipc
Remove half-baked Storage Driver IPC support
2015-06-30 19:21:54 -07:00
Stephen Day
0122a6f1ff Merge pull request #667 from stevvooe/fix-broken-docs-link
Fix broken specification link in docs
2015-06-29 18:56:04 -07:00
Stephen J Day
d3d4423ff7 Remove half-baked Storage Driver IPC support
This removes documentation and code related to IPC based storage driver
plugins. The existence of this functionality was an original feature goal but
is now not maintained and actively confusing incoming contributions. We will
likely explore some driver plugin mechanism in the future but we don't need
this laying around in the meantime.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-06-29 16:52:12 -07:00
Stephen J Day
17562b3f44 Fix broken specification link in docs
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-06-26 17:21:21 -07:00
Stephen Day
eec0e998af Revert "Updates to authentication"
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-06-26 14:21:57 -07:00
Stephen Day
5ed143bc06 Merge pull request #659 from vdemeester/specs-api-table-fix
Fix table render in specs/api.md
2015-06-26 10:17:35 -07:00
Mary Anthony
9510ef3637 Updates to authentication
Fixing heading not rendering
Fixing bad links:
deployed.md > deploying.md
spec/auth/token.md > /spec/auth/token.md

Signed-off-by: Mary Anthony <mary@docker.com>
2015-06-26 03:08:26 -07:00
Vincent Demeester
536e2fccf9 Fix table render in specs/api.md
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2015-06-26 10:34:48 +02:00
Mary Anthony
eec3ca02ba Adding metadata files
Signed-off-by: Mary Anthony <mary@docker.com>
2015-06-25 17:32:57 -07:00
Olivier Gambier
4e95367e55 Fixed doc generation
+ rollback the (now) unecessary changes

Signed-off-by: Olivier Gambier <olivier@docker.com>
2015-06-17 18:58:53 -07:00
Mary Anthony
1aa8b00bdf Adding in the better sed
Renaming to index.md;rereading of Hugo showed me my mistake; removing commented out/Markdown has no comment feature
Updating with Olivier. Yay! It looks great

Signed-off-by: Mary Anthony <mary@docker.com>
2015-06-17 17:31:13 -07:00
Stephen Day
3ea67df373 Merge pull request #623 from ahmetalpbalkan/azure-vendor
storage/driver/azure: Update vendored Azure SDK
2015-06-16 17:41:38 -07:00
Ahmet Alp Balkan
daa22cacba storage/driver/azure: Update vendored Azure SDK
This change refreshes the updated version of Azure SDK
for Go that has the latest changes.

I manually vendored the new SDK (github.com/Azure/azure-sdk-for-go)
and I removed `management/` `core/` packages manually simply because
they're not used here and they have a fork of `net/http` and `crypto/tls`
for a particular reason. It was introducing a 44k SLOC change otherwise...

This also undoes the `include_azure` flag (actually Steven removed the
driver from imports but forgot to add the build flag apparently, so the
flag wasn't really including azure. 😄 ). This also must be obsolete
now.

Fixes #620, #175.

Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
2015-06-16 17:13:44 -07:00
moxiegirl
020bd0b45f Merge pull request #631 from moxiegirl/test-tooling
Updating for new docs build and tooling infrastructure 1.7 release; Thank you Olivier. I promise to look at all of this after Dockercon.  We'll make it perfect.
2015-06-15 17:57:25 -07:00
Olivier Gambier
d678e3dc57 Merge pull request #608 from dmcgowan/http-basic-auth
Implementation of a basic authentication scheme using standard .htpasswd
2015-06-15 17:40:58 -07:00
Mary Anthony
832cb9d52c Updating for Hugo
Updating for tooling tests
Updating with the new sed scripts to protect links
updating with new image
Updating with comments

Signed-off-by: Mary Anthony <mary@docker.com>
2015-06-15 14:39:33 -07:00
Stephen Day
e57e731821 Merge pull request #548 from duglin/MoveErrors
Move ErrorCode logic to new errcode package
2015-06-15 14:33:28 -07:00
Doug Davis
441f7cac87 Round 4
Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-06-11 21:33:35 -07:00
Stephen J Day
d9d55bcbab Minor formatting fixes related to htpasswd auth
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-06-11 17:06:35 -07:00
Olivier Gambier
9b75807a0b Merge pull request #622 from amouat/patch-2
Fix service command
2015-06-11 15:27:43 -07:00
Stephen J Day
01f730ad71 Document usage of htpasswd access controller
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-06-11 15:23:40 -07:00
Adrian Mouat
a0baf93725 Fix service command
Pretty sure that service and docker were the wrong way around

Signed-off-by: Adrian Mouat <adrian.mouat@gmail.com>
2015-06-11 22:36:31 +01:00
Adrian Mouat
f670f00058 Fix typo
I think negociate is very occasionally used, but it certainly looks wrong to me.

Signed-off-by: Adrian Mouat <adrian.mouat@gmail.com>
2015-06-11 21:45:26 +01:00
Olivier Gambier
ad23a43bc4 Enhance building doc to reflect the new data default location
Signed-off-by: Olivier Gambier <olivier@docker.com>
2015-06-11 11:08:16 -07:00
Olivier Gambier
8ed0c66745 Enhancing doc to persist data
Signed-off-by: Olivier Gambier <olivier@docker.com>
2015-06-11 01:41:15 -07:00
Olivier Gambier
6e0cfc17dc Saner default data location
Signed-off-by: Olivier Gambier <olivier@docker.com>
2015-06-10 20:54:24 -07:00
Olivier Gambier
b230183b0f Merge pull request #553 from dmp42/docs-rework
Docs rework
2015-06-09 14:48:39 -07:00
Stephen Day
f63313de1f Merge pull request #599 from stevvooe/clarify-deletion-by-digest-constraint
Clarify digest in API specification
2015-06-08 19:04:53 -07:00
Stephen J Day
7e6b4e8c52 Add description of digests to API specification
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-06-08 19:04:13 -07:00
Olivier Gambier
25bd3fc777 Link to compose
Signed-off-by: Olivier Gambier <olivier@docker.com>
2015-06-08 15:57:06 -07:00
Olivier Gambier
b8d785c79e Use bcrypt
Signed-off-by: Olivier Gambier <olivier@docker.com>
2015-06-08 15:57:06 -07:00
Olivier Gambier
81e8657d7b Fix here doc conf generation
Signed-off-by: Olivier Gambier <olivier@docker.com>
2015-06-08 15:57:05 -07:00
Olivier Gambier
8c1784c838 Keeping in synch
- commenting out both the "JSON" and "token" specs, unless someone thinks they should be here
- added help, glossary, introduction and authentication documents

Signed-off-by: Olivier Gambier <olivier@docker.com>
2015-06-08 15:57:05 -07:00
Olivier Gambier
96d79eb30e Adding authentication
Signed-off-by: Olivier Gambier <olivier@docker.com>
2015-06-08 15:57:05 -07:00
Olivier Gambier
c1c638d01f Minor keyword add
Signed-off-by: Olivier Gambier <olivier@docker.com>
2015-06-08 15:55:52 -07:00
Olivier Gambier
c405f3717a Minor fixes
Signed-off-by: Olivier Gambier <olivier@docker.com>
2015-06-08 15:55:52 -07:00
Olivier Gambier
56ff32c683 Revising glossary
Signed-off-by: Olivier Gambier <olivier@docker.com>
2015-06-08 15:55:52 -07:00
Olivier Gambier
10ba376a99 Minor cleanup in the index
- adding glossary
- removing empty "advanced"
- commenting out building and architecture for now
- minor text enhancements

Signed-off-by: Olivier Gambier <olivier@docker.com>
2015-06-08 15:55:51 -07:00
Olivier Gambier
c3b3802503 Small fixes
Signed-off-by: Olivier Gambier <olivier@docker.com>
2015-06-08 15:55:51 -07:00
Olivier Gambier
b7b8e64f1d Moved instructions up
Make it more obvious that environment variables is the way to go.

Signed-off-by: Olivier Gambier <olivier@docker.com>
2015-06-08 15:55:51 -07:00
Olivier Gambier
26d87ed1a5 No reason to detail here
Signed-off-by: Olivier Gambier <olivier@docker.com>
2015-06-08 15:55:51 -07:00