Commit Graph

82 Commits

Author SHA1 Message Date
sayboras 66809646d9 Migrate to golangci-lint
Signed-off-by: Tam Mach <sayboras@yahoo.com>
2020-02-14 08:11:16 +11:00
Damien Mathieu dd3bdee21c implement Repository Blobs upload resuming
Signed-off-by: Damien Mathieu <dmathieu@salesforce.com>
2019-06-26 09:46:49 +02:00
Ryan Abrams 90dfea7952
Merge pull request #2921 from dmathieu/repository-serve-blob
Implement Repository ServeBlob
2019-06-25 19:07:38 -07:00
Damien Mathieu c5d5f938e3 fast-stop ServeBlob if we're doing a HEAD request
A registry pointing to ECR is having issues if we try loading the blob

Signed-off-by: Damien Mathieu <dmathieu@salesforce.com>
2019-06-25 09:30:22 +02:00
Damien Mathieu 3800c47fd2 Implement Repository ServeBlob
Signed-off-by: Damien Mathieu <dmathieu@salesforce.com>
2019-06-25 09:30:22 +02:00
Damien Mathieu a45e5cb13f handle create blob if the uuid couldn't be retrieved from headers or URL
Signed-off-by: Damien Mathieu <dmathieu@salesforce.com>
2019-06-25 09:29:38 +02:00
Damien Mathieu 8b31a894bd deduce blob UUID from location if it wasn't provided in the headers
Some registries (ECR) don't provide a `Docker-Upload-UUID` when creating
a blob. So we can't rely on that header. Fallback to reading it from the
URL.

Signed-off-by: Damien Mathieu <dmathieu@salesforce.com>
2019-06-25 09:29:38 +02:00
Derek McGowan 32e2260be2
Enable struct check
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-08-06 13:55:43 -07:00
Kevin Lin 1bfbeca726 Properly follow relative links when listing tags
The previous code assumed that the link returned when listing tags was
always absolute. However, some registries, such as quay.io, return the
link as a relative link (e.g. the second page for the quay.io/coreos/etcd
image is /v2/coreos/etcd/tags/list?next_page=<truncated>&n=50). Because
the relative link was retrieved directly, the fetch failed (with the
error `unsupported protocol scheme ""`).

Signed-off-by: Kevin Lin <kevin@kelda.io>
2017-11-18 22:04:19 -08:00
Clayton Coleman 3c5f85abd1
Allow clients to request specific manifest media types
The current registry/client sends the registered manifest types in
random order. Allow clients to request a single specific manifest type
or a preferred order as per the HTTP spec.

Signed-off-by: Clayton Coleman <ccoleman@redhat.com>
2017-10-03 17:06:11 -04:00
Tibor Vass 13e0608bc8 gofmt -w -s registry/client/repository.go
Signed-off-by: Tibor Vass <teabee89@gmail.com>
2017-09-05 13:14:43 -07:00
Tibor Vass 2c58ce1a7f Remove context in NewRegistry and NewRepository
The context parameter was either not used or misused.

Signed-off-by: Tibor Vass <teabee89@gmail.com>
2017-09-05 10:34:56 -07:00
Clayton Coleman a2015272c1
Support HEAD requests without Docker-Content-Digest header
A statically hosted registry that responds correctly to GET with a
manifest will load the right digest (by looking at the manifest body and
calculating the digest). If the registry returns a HEAD without
`Docker-Content-Digest`, then the client Tags().Get() call will return
an empty digest.

This commit changes the client to fallback to loading the tag via GET if
the `Docker-Content-Digest` header is not set.

Signed-off-by: Clayton Coleman <ccoleman@redhat.com>
2017-08-25 17:18:01 -04:00
Stephen J Day 9c88801a12
context: remove definition of Context
Back in the before time, the best practices surrounding usage of Context
weren't quite worked out. We defined our own type to make usage easier.
As this packaged was used elsewhere, it make it more and more
challenging to integrate with the forked `Context` type. Now that it is
available in the standard library, we can just use that one directly.

To make usage more consistent, we now use `dcontext` when referring to
the distribution context package.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-08-11 15:53:31 -07:00
Stephen J Day 532ec9f036
digest: migrate to opencontainers/go-digest
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-01-06 15:42:03 -08:00
Stephen J Day 01dfa0fcb9
digest: use digest.Parse over ParseDigest
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2016-12-15 15:07:42 -08:00
a-palchikov 82609180a1 tag service: properly handle error responses on HEAD requests by (#1918)
* tag service: properly handle error responses on HEAD requests by
re-issuing requests as GET for proper error details.

Fixes #1911.

Signed-off-by: dmitri <deemok@gmail.com>

* Simplify handling of failing HEAD requests in TagService and
make a GET request for cases:
  - if the server does not handle HEAD
  - if the response was an error to get error details

Signed-off-by: dmitri <deemok@gmail.com>

* Add a missing http.Response.Body.Close call for the GET request.

Signed-off-by: dmitri <deemok@gmail.com>
2016-08-26 14:35:04 -07:00
Richard Scothern 857d0f15c0 Merge pull request #1757 from miminar/cross-mount-middleware-repository
Allow to use repo middleware wrapper during cross-repo mounts
2016-07-19 17:43:10 -07:00
Michal Minář 3f1434525b Export storage.CreateOptions in top-level package
Let the options for `BlobStore.Create()` be modified in middleware
wrappers.

Signed-off-by: Michal Minar <miminar@redhat.com>
2016-07-06 16:40:46 +02:00
Brian Bland a1f9f71e67 Changes the client Tags All() method to follow links
This returns all tags even when the registry forces pagination.

Signed-off-by: Brian Bland <brian.t.bland@gmail.com>
2016-06-28 15:49:14 -07:00
Derek McGowan 125f4ff7d7 Add option to get content digest from manifest get
The client may need the content digest to delete a manifest using the digest used by the registry.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-06-08 17:02:29 -07:00
Richard Scothern 87a997249d Merge pull request #1522 from RichardScothern/tag-events
Send tag events to notification listeners
2016-03-23 16:41:29 -07:00
Richard Scothern bc9c820e4b Enable URLs returned from the registry to be configured as relative.
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-03-23 15:14:32 -07:00
Richard Scothern afe2bdd1c5 Propogate tag as a functional argument into the notification system to attach
tags to manifest push and pull event notifications.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-03-23 14:57:52 -07:00
Antonio Murdaca 75b5a1b1f6 registry: client: repository: close response body
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-16 19:53:17 +01:00
Richard Scothern 7144936612 Enable proxying registries to downgrade fetched manifests to Schema 1.
Ensure Accept headers are sent with TagService.Get (which hits manifest
endpoints).  Add support for remote Get and Put for the proxied blobstore.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-02-22 17:56:48 -08:00
Richard Scothern dd58349b35 Merge pull request #1419 from aaronlehmann/safer-header-copying
On redirect, only copy headers when they don't already exist in the redirected request
2016-02-05 12:14:01 -08:00
Aaron Lehmann b0989446eb Rename Name method of Repository to Named
This makes code that gets the name as a string read like
repo.Named().Name() instead of repo.Name().Name().

Requested in
https://github.com/docker/docker/pull/19887#discussion_r51479753

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-02-04 09:55:36 -08:00
Aaron Lehmann 4354f0a107 On redirect, only copy headers when they don't already exist in the redirected request
A changeset under consideration for Go 1.7 would automatically copy
headers on redirect. This change future-proofs our code so we won't make
duplicate copies of the headers if net/http does it automatically in the
future.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-02-03 13:19:44 -08:00
Aaron Lehmann 74c4a0e3e3 Merge pull request #1393 from RichardScothern/putbydgst
Add manifest put by digest to the registry client
2016-01-28 14:41:07 -08:00
Aaron Lehmann 8fd84c4a57 Add a CheckRedirect function to the HTTP client
Use it to preserve Accept and Range headers that were added to the
original request.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-27 16:11:36 -08:00
Richard Scothern 9f72e8442d Add manifest put by digest to the registry client
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-01-26 14:20:23 -08:00
Aaron Lehmann 2b20b0167a Change URLBuilder methods to use references for tags and digests
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-22 14:49:29 -08:00
Aaron Lehmann 4441333912 Use reference package internally
Most places in the registry were using string types to refer to
repository names. This changes them to use reference.Named, so the type
system can enforce validation of the naming rules.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-01-22 14:47:05 -08:00
Brian Bland 8c5a6c13c0 Splits up blob create options definitions to be package-specific
Redefines privately in both storage and client packages

Signed-off-by: Brian Bland <brian.bland@docker.com>
2016-01-14 10:47:33 -08:00
Brian Bland ce88d8a6f4 Adds functional options arguments to the Blobs Create method
Removes the Mount operation and instead implements this behavior as part
of Create a From option is provided, which in turn returns a rich
ErrBlobMounted indicating that a blob upload session was not initiated,
but instead the blob was mounted from another repository

Signed-off-by: Brian Bland <brian.bland@docker.com>
2016-01-13 16:42:59 -08:00
Brian Bland 5df21570a7 Adds cross-repository blob mounting behavior
Extends blob upload POST endpoint to support mount and from query
parameters as described in #634

Signed-off-by: Brian Bland <brian.bland@docker.com>
2016-01-08 13:53:18 -08:00
Kenny Leung c28278f7a2 Print error for failed HTTP auth request.
Signed-off-by: Kenny Leung <kleung@google.com>
2015-12-29 11:39:20 -08:00
Richard Scothern 67d3675d55 Merge pull request #1268 from RichardScothern/manifest-refactor-impl
Implementation of the Manifest Service API refactor.
2015-12-17 17:32:55 -08:00
Richard Scothern cb6f002350 Implementation of the Manifest Service API refactor.
Add a generic Manifest interface to represent manifests in the registry and
remove references to schema specific manifests.

Add a ManifestBuilder to construct Manifest objects. Concrete manifest builders
will exist for each manifest type and implementations will contain manifest
specific data used to build a manifest.

Remove Signatures() from Repository interface.

Signatures are relevant only to schema1 manifests.  Move access to the signature
store inside the schema1 manifestStore.  Add some API tests to verify
signature roundtripping.

schema1
-------

Change the way data is stored in schema1.Manifest to enable Payload() to be used
to return complete Manifest JSON from the HTTP handler without knowledge of the
schema1 protocol.

tags
----

Move tag functionality to a seperate TagService and update ManifestService
to use the new interfaces.  Implement a driver based tagService to be backward
compatible with the current tag service.

Add a proxyTagService to enable the registry to get a digest for remote manifests
from a tag.

manifest store
--------------

Remove revision store and move all signing functionality into the signed manifeststore.

manifest registration
---------------------

Add a mechanism to register manifest media types and to allow different manifest
types to be Unmarshalled correctly.

client
------

Add ManifestServiceOptions to client functions to allow tags to be passed into Put and
Get for building correct registry URLs.  Change functional arguments to be an interface type
to allow passing data without mutating shared state.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>

Signed-off-by: Richard Scothern <richard.scothern@docker.com>
2015-12-17 17:09:14 -08:00
Aaron Lehmann be1f4114c7 Remove unnecessary stat from blob Get method
This calls Stat before Open, which should be unnecessary because Open
can handle the case of a nonexistent blob. Removing the Stat saves a
round trip.

This is similar to the removal of stat in Open in #1226.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-12-11 15:13:03 -08:00
Richard Scothern 95e80a8283 Merge pull request #1243 from RichardScothern/client-header-message
Add clearer messaging around missing content-length headers.
2015-12-08 09:54:18 -08:00
Richard Scothern f18bf5e343 Add clearer messaging around missing content-length headers.
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-12-07 10:17:49 -08:00
Aaron Lehmann 6beeb935cd Avoid stat round-trips when fetching a blob
Without this commit, three round-trips are required to fetch a blob with
a progress bar. The first is a call to Stat (HEAD request), to determine
the size. Then Open is called, which also calls Stat, and finally
performs a GET request.

Only the GET request is actually needed. The size of the blob can be
sniffed from Content-Length in the GET response.

This commit changes HTTPReadSeeker to automatically detect the size from
Content-Length instead of requiring it to be passed in. The Stat call is
removed from Open because it is no longer necessary.

HTTPReadSeeker now takes an additional errorHandler callback argument which
translates an unsuccessful HTTP response into an appropriate API-level
error. Using a callback for this makes it possible to avoid leaking the
repsonse body to Read's caller, which would make lifecycle management
problematic.

Fixes #1223

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-12-02 14:21:13 -08:00
Derek McGowan 31a448a628 Update to provide small and clear interfaces
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-10-09 16:05:49 -07:00
Tibor Vass f8c09b6a7d Add a new reference package abstracting repositories, tags and digests
There seems to be a need for a type that represents a way of pointing
to an image, irrespective of the implementation.

This patch defines a Reference interface and provides 3 implementations:
- TagReference: when only a tag is provided
- DigestReference: when a digest (according to the digest package) is
  provided, can include optional tag as well

Validation of references are purely syntactic.

There is also a strong type for tags, analogous to digests, as well
as a strong type for Repository from which clients can access the
hostname alone, or the repository name without the hostname, or both
together via the String() method.

For Repository, the files names.go and names_test.go were moved from
the v2 package.

Signed-off-by: Tibor Vass <tibor@docker.com>
2015-10-09 16:05:34 -07:00
Richard Scothern f36ab5a834 Don't return a nil array and a nil error if the Tags endpoint cannot be found
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-09-18 16:19:41 -07:00
Richard Scothern 924913b4c3 Avoid returning nil, nil when fetching a manifest by tag by introducing a new
error ErrManifestNotModified which can be checked by clients.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-09-18 16:19:37 -07:00
Stephen Day 1927c6c0f2 Merge pull request #934 from stevvooe/sanitize-url-correctly
Correctly sanitize location url preserving parameters
2015-08-27 22:30:21 -07:00
Stephen J Day 7232daf692 Correctly sanitize location url preserving parameters
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-08-27 17:06:03 -07:00