Since the CloudFront middleware does not work without an S3 backend, it
became obvious that this documentation should exist within the S3
storage-driver documentation.
Signed-off-by: DJ Enriquez <dj.enriquez@infospace.com>
This is a follow-on to PR #62, and it borrows much of the format
from #993, but uses specific formats for the image manifest and manifest
list (fat manifest) instead of a combined generic format.
The intent of this proposed manifest format is to allow multi-arch, and
allow for full content-addressability of images in the Docker engine.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Adding a more detailed document regarding how to use CloudFront as
middleware for an s3 backed registry.
Signed-off-by: DJ Enriquez <dj.enriquez@infospace.com>
Add a generic Manifest interface to represent manifests in the registry and
remove references to schema specific manifests.
Add a ManifestBuilder to construct Manifest objects. Concrete manifest builders
will exist for each manifest type and implementations will contain manifest
specific data used to build a manifest.
Remove Signatures() from Repository interface.
Signatures are relevant only to schema1 manifests. Move access to the signature
store inside the schema1 manifestStore. Add some API tests to verify
signature roundtripping.
schema1
-------
Change the way data is stored in schema1.Manifest to enable Payload() to be used
to return complete Manifest JSON from the HTTP handler without knowledge of the
schema1 protocol.
tags
----
Move tag functionality to a seperate TagService and update ManifestService
to use the new interfaces. Implement a driver based tagService to be backward
compatible with the current tag service.
Add a proxyTagService to enable the registry to get a digest for remote manifests
from a tag.
manifest store
--------------
Remove revision store and move all signing functionality into the signed manifeststore.
manifest registration
---------------------
Add a mechanism to register manifest media types and to allow different manifest
types to be Unmarshalled correctly.
client
------
Add ManifestServiceOptions to client functions to allow tags to be passed into Put and
Get for building correct registry URLs. Change functional arguments to be an interface type
to allow passing data without mutating shared state.
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
Signed-off-by: Richard Scothern <richard.scothern@docker.com>
tarsum is not actually used by the registry. Remove support for it.
Convert numerous uses in unit tests to SHA256.
Update docs to remove mentions of tarsums (which were often inaccurate).
Remove tarsum dependency.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Fixes a huge problem where a nginx variable was not escaped and nginx was adding the literal string "always" to all requests.
The most obvious result was that basic auth was entirely broken.
Signed-off-by: Carson Anderson <ca@carson-anderson.com>
If this example was copied and pasted, the shell would try to interpolate `$upstream_http_docker_distribution_api_version` and `$docker_distribution_api_version`.
Signed-off-by: Andrew Meredith <andymeredith@gmail.com>
This change clarifies the way the catalog endpoint returns results
when pagination was not explicitly requested.
Signed-off-by: Patrick Devine <patrick.devine@docker.com>
This extends the specification for the Bearer token response to include
information pertaining to when an issued Bearer token will expire.
This also allows the client to accept `access_token` as an alias for `token`.
Signed-off-by: Matt Moore <mattmoor@google.com>
The docs don't render emoji, so replaced the `⚠️` with
a `**Warning**:` to keep the formatting consisten with
`**Note**:` used in other parts of the docs.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This adds a variable to help nginx add the
Docker-Distribution-Api-Version when using basic auth, and not add the
header when it sees it from the upstream.
Also fix some minor spelling/grammar issues.
Signed-off-by: Sharif Nassar <sharif@mrwacky.com>
It seems that enabling proxy stops my instance from accepting local pushes, but I can't find mention of that in the docs.
Signed-off-by: Jason Freidman <jason@periscope.io>
This allows the administrator to specify an externally-reachable URL for
the registry. It takes precedence over the X-Forwarded-Proto and
X-Forwarded-Host headers, and the hostname in the request.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Associate HTTP 401s with Authentication errors rather than Authorization
errors. Changes the meaning of the UNAUTHORIZED error to be authentication
specific.
Defines DENIED error code to be associated with authorization
errors which result in HTTP 403 responses.
Add 'No Such Repository' errors to more endpoints.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
This PR add some description to manifest spec.
It clarifies the relationship between `fsLayers` and `history` fields.
Signed-off-by: xiekeyang <xiekeyang@huawei.com>
Add "readonly" under the storage/maintenance section. When this is set
to true, uploads and deletions will return 503 Service Unavailable
errors.
Document the parameter and add some unit testing.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Split the discussion of v2 authentication into two parts:
1) A specification of the handshake between the client, registry and
authentication service.
2) A description of how `docker/distribution` implements this using JWT.
This should make it clearer that `#2` is an implementation detail, and
that clients should regard tokens as opaque entities that only the
registry and authentication service should understand.
Signed-off-by: Matt Moore <mattmoor@google.com>
Overriding configuration parameters with environment variables used to
work by walking the configuration structure and checking for a
corresponding environment variable for each item. This was very limiting
because only variables corresponding to items that already existed in
the configuration structure would be checked. For example, an
environment variable corresponding to nested maps would only be noticed
if the outer map's key already existed.
This commit changes environment variable overriding to iterate over the
environment instead. For environment variables beginning with the
REGISTRY_ prefix, it splits the rest of their names on "_", and
interprets that as a path to the variable to unmarshal into. Map keys
are created as necessary. If we encounter an empty interface partway
through following the path, it becomes an implicit
map[string]interface{}.
With the new unit tests added here, parser.go now has 89.2% test
coverage.
TestParseWithExtraneousEnvStorageParams was removed, because the limit
of one storage driver is no longer enforced while parsing environment
variables. Now, Storage.Type will panic if multiple drivers are
specified.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Normalized mirror.md to match other recipes.
Slightly tweaked content, emphasizing the privacy concerns about using login/password on the backend.
Signed-off-by: Olivier Gambier <olivier@docker.com>
Making metadata:
- more consistent
- more specific (fixed copy pasting)
- refine coverage
Insecure information cleanup
Removing no longer used files:
- mkdocs is gone
- the registry diagram is not used, and is a bit silly :)
Minor fixes
Fixing links
Recipes:
- harmonized code sections style to the rest of the docs
- harmonized recipe "style"
- listing new recipes
Enhance deploying
Signed-off-by: Olivier Gambier <olivier@docker.com>
Also, add timeout and status code parameters to the HTTP checker, and
remove the threshold parameter for the file checker.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Add default storagedriver health check to example configuration files
with parameters matching the previous hardcoded configuration.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Add a section to the config file called "health". Within this section,
"filecheckers" and "httpcheckers" list checks to run. Each check
specifies a file or URI, a time interval for the check, and a threshold
specifying how many times the check must fail to reach an unhealthy
state.
Document the new options in docs/configuration.md.
Add unit testing for both types of checkers. Add an UnregisterAll
function in the health package to support the unit tests, and an
Unregister function for consistency with Register.
Fix a string conversion problem in the health package's HTTP checker.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
The storage cache option layerinfo is deprecated,
so use blobdescriptor instead in example config files.
Signed-off-by: xg.song <xg.song@venusource.com>
The example configuration files add X-Content-Type-Options: nosniff.
Add coverage in existing registry/handlers unit tests.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
distribution errors. Fill in missing checks for mutations on a registry pull-through
cache. Add unit tests and update documentation.
Also, give v2.ErrorCodeUnsupported an HTTP status code, previously it was
defaulting to 500, now its 405 Method Not Allowed.
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
Several error codes are generally useful but tied to the v2 specification
definitions. This change moves these error code definitions into the common
package for use by the health package, which is not tied to the v2 API.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
We are headed in a different direction. The dist tool analog will not be a part
of this repository.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
This page was missing styling once exported to HTML. Adding a
<!--[metadata]> block similar to the ones the other *.md files have
appears to solve the problem.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
This fixes several flaws in the link rewriter:
- broken links on the docs side
- multiple links on one line being mangled
- byzantine logic
Also generalize the logic of the Dockerfile so it is no longer specific to "registry" (env variable), which is a first step in making it possible to upstream all this into the base image and docs project.
Added a number of tests (test.md) to validate the link replacer behavior (against test.compare.md), and embedded the test so that the doc build will fail if the link replacer does not behave.
This is still sed, unfortunately.
Signed-off-by: Olivier Gambier <olivier@docker.com>
with a new `proxy` section in the configuration file.
Create a new registry type which delegates storage to a proxyBlobStore
and proxyManifestStore. These stores will pull through data if not present
locally. proxyBlobStore takes care not to write duplicate data to disk.
Add a scheduler to cleanup expired content. The scheduler runs as a background
goroutine. When a blob or manifest is pulled through from the remote registry,
an entry is added to the scheduler with a TTL. When the TTL expires the
scheduler calls a pre-specified function to remove the fetched resource.
Add token authentication to the registry middleware. Get a token at startup
and preload the credential store with the username and password supplied in the
config file.
Allow resumable digest functionality to be disabled at runtime and disable
it when the registry is a pull through cache.
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
Rename config.yml to dev-config.yml
Add example-config.yml, a simple configuration file for the official
This was originally made for the the distribution-library-image repo,
but is being moved here to make sure it stays in sync.
Update Dockerfile and docs for the rename.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Log a warning if the registry generates its own secret.
Update configuration doc, and remove the default secret from the
development config file.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Storage drivers can implement a method called URLFor which can return a direct
url for a given path. The functionality allows the registry to direct clients
to download content directly from the backend storage. This is commonly used
with s3 and cloudfront. Under certain conditions, such as when the registry is
not local to the backend, these redirects can hurt performance and waste
incoming bandwidth on pulls. This feature addition allows one to disable this
feature, if required.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Conflicts:
configuration/configuration.go
registry/handlers/app.go
registry/storage/catalog_test.go
registry/storage/manifeststore_test.go
registry/storage/registry.go
Implement the delete API by implementing soft delete for layers
and blobs by removing link files and updating the blob descriptor
cache. Deletion is configurable - if it is disabled API calls
will return an unsupported error.
We invalidate the blob descriptor cache by changing the linkedBlobStore's
blobStatter to a blobDescriptorService and naming it blobAccessController.
Delete() is added throughout the relevant API to support this functionality.
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
Signed-off-by: David Verhasselt <david@crowdway.com>
Add clarifying requirement of nginx 1.7.5 to authentication.md file.
Signed-off-by: David Verhasselt <david@crowdway.com>
Revert "Update authentication.md"
This reverts commit a79c28f4376fad333edd94e9b25211830e63abbd.
Add paragraph on manual set-up
To avoid confusion with the registry version number or other project versions,
the specification updates are now lettered.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Some missing descriptions and error code for tags pagination was cleaned up to
ensure clarity. Specifically, we ensure the request variations are named and
the proper error codes are included.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Move the specification to use a Link header, rather than a "next" entry in the
json results. This prevents requiring clients from parsing the request body to
issue the next request. It also ensures that the returned response body does
not change in between requests.
The ordering of the specification has been slightly tweaked, as well. Listing
image tags has been moved after the catalog specification. Tag pagination now
heavily references catalog pagination.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
This contains a proposal for a catalog API, provided access to the internal
contents of a registry instance. The API endpoint is prefixed with an
underscore, which is illegal in images names, to prevent collisions with
repositories names. To avoid issues with large result sets, a paginated version
of the API is proposed. We make an addition to the tags API to support
pagination to ensure the specification is conistent.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Since the actual port is 5003, it would make sense to name it local-5003 instead of local-8082
Signed-off-by: Florentin Raud <florentin.raud@gmail.com>
After consideration, we've changed the main descriptor field name to for number
of bytes to "size" to match convention. While this may be a subjective
argument, commonly we refer to files by their "size" rather than their
"length". This will match other conventions, like `(FileInfo).Size()` and
methods on `io.SizeReaderAt`. Under more broad analysis, this argument doesn't
necessarily hold up. If anything, "size" is shorter than "length".
Signed-off-by: Stephen J Day <stephen.day@docker.com>
yaml sections in the documentation does not display well on
docs.docker.com. This is due to the syntax highlighting
which uses highlight.js and does not support yaml
currently.
The fix is to remove triple back ticks and indent instead.
We loose yaml syntax highlighting on github, but it displays
an acceptable version on both github and docs.docker.com.
Signed-off-by: Olivier Jacques <olivier.jacques@hp.com>
The main goal of this changeset is to allow repository name components to
consist of a single character. The number of components allowed and the slash
separation requirements have also been clarified.
To go along with this simplification, errant constants and unneeded error types
have been removed.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Intermediate certificates are issued by TLS providers who themselves are
an intermediate of a certificate in the trust store. Therefore, to prove
the chain of trust is valid, you need to include their certificate as
well as yours when you send your certificate to the client.
Contrary to what I said in issue #683, distribution can handle these
certificate bundles like nginx. As discussed in #docker-distribution,
I have updated the deployment documentation (which recommends the use of
a TLS certificate from a provider) to include instructions on how to
handle the intermediate certificate when a user is configuring
distribution.
Signed-off-by: Luke Carpenter <x@rubynerd.net>
This removes documentation and code related to IPC based storage driver
plugins. The existence of this functionality was an original feature goal but
is now not maintained and actively confusing incoming contributions. We will
likely explore some driver plugin mechanism in the future but we don't need
this laying around in the meantime.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Fixing heading not rendering
Fixing bad links:
deployed.md > deploying.md
spec/auth/token.md > /spec/auth/token.md
Signed-off-by: Mary Anthony <mary@docker.com>
Renaming to index.md;rereading of Hugo showed me my mistake; removing commented out/Markdown has no comment feature
Updating with Olivier. Yay! It looks great
Signed-off-by: Mary Anthony <mary@docker.com>
This change refreshes the updated version of Azure SDK
for Go that has the latest changes.
I manually vendored the new SDK (github.com/Azure/azure-sdk-for-go)
and I removed `management/` `core/` packages manually simply because
they're not used here and they have a fork of `net/http` and `crypto/tls`
for a particular reason. It was introducing a 44k SLOC change otherwise...
This also undoes the `include_azure` flag (actually Steven removed the
driver from imports but forgot to add the build flag apparently, so the
flag wasn't really including azure. 😄 ). This also must be obsolete
now.
Fixes#620, #175.
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
Updating for new docs build and tooling infrastructure 1.7 release; Thank you Olivier. I promise to look at all of this after Dockercon. We'll make it perfect.
Updating for tooling tests
Updating with the new sed scripts to protect links
updating with new image
Updating with comments
Signed-off-by: Mary Anthony <mary@docker.com>
- commenting out both the "JSON" and "token" specs, unless someone thinks they should be here
- added help, glossary, introduction and authentication documents
Signed-off-by: Olivier Gambier <olivier@docker.com>
