with a new `proxy` section in the configuration file.
Create a new registry type which delegates storage to a proxyBlobStore
and proxyManifestStore. These stores will pull through data if not present
locally. proxyBlobStore takes care not to write duplicate data to disk.
Add a scheduler to cleanup expired content. The scheduler runs as a background
goroutine. When a blob or manifest is pulled through from the remote registry,
an entry is added to the scheduler with a TTL. When the TTL expires the
scheduler calls a pre-specified function to remove the fetched resource.
Add token authentication to the registry middleware. Get a token at startup
and preload the credential store with the username and password supplied in the
config file.
Allow resumable digest functionality to be disabled at runtime and disable
it when the registry is a pull through cache.
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
registry/storage/blob_test.go:149: arg d for printf verb %s of wrong type: github.com/docker/distribution.Descriptor
Signed-off-by: Doug Davis <dug@us.ibm.com>
The response code isn't actually sent to the client, because the
connection has already closed by this point. But it causes the status
code to appear as 499 in the logs instead of 0.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
When a client disconnects without completing a HTTP request, we were
attempting to process the partial request, which usually leads to a 400
error. These errors can pollute the logs and make it more difficult to
track down real bugs.
This change uses CloseNotifier to detect disconnects. In combination
with checking Content-Length, we can detect a disconnect before sending
the full payload, and avoid logging a 400 error.
This logic is only applied to PUT, POST, and PATCH endpoints, as these
are the places where disconnects during a request are most likely to
happen.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Some structures use int for sizes and UNIX timestamps. On some
platforms, int is 32 bits, so this can lead to the year 2038 issues and
overflows when dealing with large containers or layers.
Consistently use int64 to store sizes and UNIX timestamps in
api/types/types.go. Update related to code accordingly (i.e.
strconv.FormatInt instead of strconv.Itoa).
Use int64 in progressreader package to avoid integer overflow when
dealing with large quantities. Update related code accordingly.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
This patch fixes a bug where a user specifies a v1 mirror for
--registry-mirror and pull an image from the Hub.
It used to not fallback because of an unexpected error returned when
trying to JSON marshal nginx output.
We now ensure that any unexpected error falls back to the next endpoint
in the list.
Signed-off-by: Tibor Vass <tibor@docker.com>
includes the http response.
When debugging non-successful registry requests this will place
the error details and http status fields in the same log line
giving easier visibility to what error occured in the request.
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
Log a warning if the registry generates its own secret.
Update configuration doc, and remove the default secret from the
development config file.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
This passed in the #744 before merge, but apparently the test changed
since the PR was created in ways that led to a new failures.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Currently login and search do not load per registry certificates.
This is a regression caused by the last refactor since this was recently fixed.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Storage drivers can implement a method called URLFor which can return a direct
url for a given path. The functionality allows the registry to direct clients
to download content directly from the backend storage. This is commonly used
with s3 and cloudfront. Under certain conditions, such as when the registry is
not local to the backend, these redirects can hurt performance and waste
incoming bandwidth on pulls. This feature addition allows one to disable this
feature, if required.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Conflicts:
configuration/configuration.go
registry/handlers/app.go
registry/storage/catalog_test.go
registry/storage/manifeststore_test.go
registry/storage/registry.go
Add a trusted flag to force the cli to resolve a tag into a digest via the notary trust library and pull by digest.
On push the flag the trust flag will indicate the digest and size of a manifest should be signed and push to a notary server.
If a tag is given, the cli will resolve the tag into a digest and pull by digest.
After pulling, if a tag is given the cli makes a request to tag the image.
Use certificate directory for notary requests
Read certificates using same logic used by daemon for registry requests.
Catch JSON syntax errors from Notary client
When an uncaught error occurs in Notary it may show up in Docker as a JSON syntax error, causing a confusing error message to the user.
Provide a generic error when a JSON syntax error occurs.
Catch expiration errors and wrap in additional context.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
* Add godoc documentation where it was missing
* Change identifier names that don't match Go style, such as INDEX_NAME
* Rename RegistryInfo to PingResult, which more accurately describes
what this structure is for. It also has the benefit of making the name
not stutter if used outside the package.
Updates #14756
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Implement the delete API by implementing soft delete for layers
and blobs by removing link files and updating the blob descriptor
cache. Deletion is configurable - if it is disabled API calls
will return an unsupported error.
We invalidate the blob descriptor cache by changing the linkedBlobStore's
blobStatter to a blobDescriptorService and naming it blobAccessController.
Delete() is added throughout the relevant API to support this functionality.
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
Because the logger was incorrectly replaced while adding hooks, log output did
not include the version and instance ids. The main issue was the the
logrus.Entry was replaced with the logger, which included no context. Replacing
the logger on the context is not necessary when configuring hooks since we are
configuring the contexts logger directly.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
