Commit graph

387 commits

Author SHA1 Message Date
Richard Scothern
00f02b5fbc Buffer writing the scheduler entry state to disk by periodically checking for
changes to the entries index and saving it to the filesystem.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-10-29 11:35:52 -07:00
Richard Scothern
a31e93e1c4 Merge pull request #756 from aibaars/storage-gcs
Storage driver for Google Cloud Storage
2015-10-29 10:41:17 -07:00
Richard Scothern
cbf1f0797c Merge pull request #1096 from aaronlehmann/simplify-proxy-scheduler
Simplify proxy scheduler
2015-10-28 13:23:00 -07:00
Arthur Baars
98ad17f757 Storage driver for: Google Cloud Storage (gcs)
Signed-off-by: Arthur Baars <arthur@semmle.com>
2015-10-28 20:21:56 +00:00
Matt Moore
b38b98c8a8 Add expires_in and issued_at to the auth spec.
This extends the specification for the Bearer token response to include
information pertaining to when an issued Bearer token will expire.

This also allows the client to accept `access_token` as an alias for `token`.

Signed-off-by: Matt Moore <mattmoor@google.com>
2015-10-27 11:05:33 -07:00
Stephen Day
8cc790cccf Merge pull request #1098 from denverdino/master
Redundant digest verification in validateBlob when pushing a new layer
2015-10-21 14:50:18 -07:00
Tianon Gravi
8263cdeb57 Update "type auth.Challenge" comment example code
This interface was changed in 4a2300aaa9, but the comment wasn't ever updated to match.

Signed-off-by: Andrew "Tianon" Page <admwiggin@gmail.com>
2015-10-21 11:54:13 -07:00
Aaron Lehmann
fc5ee720d1 Correct two golint comment issues
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-10-19 16:42:12 -07:00
Richard Scothern
0ce983b4e8 Merge pull request #1006 from RichardScothern/proxy-serialize
Fix a race condition in pull through cache population
2015-10-16 17:13:26 -07:00
Stephen Day
eb484b7ddf Merge pull request #827 from aaronlehmann/read-only-mode-2
Add a read-only mode as a configuration option
2015-10-15 11:50:31 -07:00
Aaron Lehmann
84595fc628 Simplify proxy scheduler
The proxy scheduler implemented its own timer state machine. It's
simpler and more efficient to leverage the Go runtime's timer heap by
using time.AfterFunc.

This commit adds a time.Timer to each scheduler entry, and starts and
stops those timers as necessary. Then the mainloop goroutine and its
associated logic are not needed.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-10-15 09:18:21 -07:00
Li Yi
c40b2e2341 Redundant digest verification in validateBlob
Change-Id: I03764edadae529db2cc3acf7ecca329570f18659
Signed-off-by: Li Yi <denverdino@gmail.com>
2015-10-15 22:21:14 +08:00
Stephen J Day
26762a54fe Correct unmarshal order for SignedManifest
To ensure that we only unmarshal the verified payload into the contained
manifest, we first copy the entire incoming buffer into Raw and then unmarshal
only the Payload portion of the incoming bytes. If the contents is later
verified, the caller can then be sure that the contents of the Manifest fields
can be trusted.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-10-14 14:37:34 -07:00
Richard Scothern
da206d64cb Merge pull request #1035 from aaronlehmann/host-parameter-alternate
Add http.host parameter
2015-10-13 14:03:48 -07:00
Stephen Day
80783c058f Merge pull request #1033 from jlhawn/api_spec_update_auth_errors
[api spec] Update authN and authZ errors
2015-10-13 13:52:18 -07:00
Derek McGowan
582a0661bf Update to provide small and clear interfaces
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-10-09 16:05:49 -07:00
Tibor Vass
b72f1fd2e3 Add a new reference package abstracting repositories, tags and digests
There seems to be a need for a type that represents a way of pointing
to an image, irrespective of the implementation.

This patch defines a Reference interface and provides 3 implementations:
- TagReference: when only a tag is provided
- DigestReference: when a digest (according to the digest package) is
  provided, can include optional tag as well

Validation of references are purely syntactic.

There is also a strong type for tags, analogous to digests, as well
as a strong type for Repository from which clients can access the
hostname alone, or the repository name without the hostname, or both
together via the String() method.

For Repository, the files names.go and names_test.go were moved from
the v2 package.

Signed-off-by: Tibor Vass <tibor@docker.com>
2015-10-09 16:05:34 -07:00
Richard Scothern
36fa22c821 Fix a race condition in pull through cache population by removing the functionality
of readers joining current downloads.  Concurrent requests for the same blob
will not block, but only the first instance will be comitted locally.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-10-08 16:12:29 -07:00
Matt Moore
d5ca577ad1 Allow hostname components in component names.
Fixes https://github.com/docker/distribution/issues/1062

This relaxes the naming restrictions places on Docker images to permit
valid hostnames according to [RFC-2396](https://www.ietf.org/rfc/rfc2396.txt).

It deviates from the RFC in the following ways:
 1) Allow underscores where we allow hyphens (hostnames don't allow
 underscores, which we must for backwards compatibility).
 2) Leave "top-level" name segments unrestricted (domains require an
 alpha character to begin a top-level domain, e.g. "com").
 3) DO NOT allow a trailing dot, as permitted by FQDNs.

Signed-off-by: Matt Moore <mattmoor@google.com>
2015-10-08 11:29:54 -07:00
Richard Scothern
1039e2dc26 Merge pull request #950 from jackpgriffin/master
Skip creating swift container if already exists
2015-10-01 10:15:54 -07:00
Aaron Lehmann
6573d5c119 Add http.host parameter
This allows the administrator to specify an externally-reachable URL for
the registry. It takes precedence over the X-Forwarded-Proto and
X-Forwarded-Host headers, and the hostname in the request.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-09-30 15:12:01 -07:00
Josh Hawn
fa4c33f5f3 [api spec] Update authN and authZ errors
Associate HTTP 401s with Authentication errors rather than Authorization
errors. Changes the meaning of the UNAUTHORIZED error to be authentication
specific.

Defines DENIED error code to be associated with authorization
errors which result in HTTP 403 responses.

Add 'No Such Repository' errors to more endpoints.

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
2015-09-30 09:12:31 -07:00
Stephen Day
3532bc3f2c Merge pull request #1024 from aaronlehmann/avoid-testing-import
Avoid importing "testing" in externally-facing code
2015-09-30 13:14:42 +02:00
Aaron Lehmann
b045aa2a3d Avoid importing "testing" in externally-facing code
The "testing" package adds some flags in its init function, so utilities
that import distribution code may print a page of extra testing flags in
their help output.

This commit solves the issue by moving an import of "testing" in the
registry/storage/cache package to a new
registry/storage/cache/cachecheck package, which is only imported by
tests.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-09-29 10:06:39 -07:00
Aaron Lehmann
8a05f9bb4f Merge pull request #1001 from RichardScothern/fix-atrocity
Don't return nil, nil from functions
2015-09-28 13:34:11 -07:00
Aaron Lehmann
cbf83ecd31 Add an "enabled" parameter under "readonly", and make it as if the mutable handlers don't exist when read-only mode is enabled
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-09-22 15:49:26 -07:00
Aaron Lehmann
df9758ba39 Add a read-only mode as a configuration option
Add "readonly" under the storage/maintenance section. When this is set
to true, uploads and deletions will return 503 Service Unavailable
errors.

Document the parameter and add some unit testing.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-09-22 15:47:48 -07:00
Nuutti Kotivuori
84e7c07c42 Remove initial access check from S3 driver
In the S3 storage driver there is currently an initial access permission check by listing the bucket. If this check fails, registry will panic and exit.

However, this check is broken in two ways. First of all it strips the final slash from the root directory path, meaning that any access permissions which limit access to a single directory will fail, because S3 treats the path as strict prefix match. Secondly it fails to strip any leading slash that might be present, unlike the other access places, which means that the path used is different as a leading slash is allowed and significant in a filename in S3.

Since there is also a periodic health check which correctly checks access permissions and shows the error more cleanly, the best solution seems to be to just remove this initial access check.

Signed-off-by: Nuutti Kotivuori <nuutti.kotivuori@poplatek.fi>
2015-09-19 12:47:12 +03:00
Richard Scothern
9fb5fe4fbb Don't return a nil array and a nil error if the Tags endpoint cannot be found
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-09-18 16:19:41 -07:00
Richard Scothern
b8a1ec4155 Avoid returning nil, nil when fetching a manifest by tag by introducing a new
error ErrManifestNotModified which can be checked by clients.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-09-18 16:19:37 -07:00
Stephen J Day
49f080acc8 Add WithVersion to context and other cleanup
By adding WithVersion to the context package, we can simplify context setup in
the application. This avoids some odd bugs where instantiation order can lead
to missing instance.id or version from log messages.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-09-14 17:14:31 -07:00
Stephen J Day
cabf1fd236 Allow interface{} keys when using logger
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-09-11 11:33:55 -07:00
Aaron Lehmann
6403bf64d5 Only use the distribution/context package in registry.go
This solves a issue from #909 where instance.id was not printed in logs,
because this file was using the background context from
golang.org/x/net/context instead of
github.com/docker/distribution/context.

It's cleaner to standardize on one package, so this commit removes the
import of golang.org/x/net/context entirely. The Context interfaces
defined in both packages are the same, so other code using
golang.org/x/net/context can still pass its context to NewRegistry.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-09-11 09:54:15 -07:00
Aaron Lehmann
045db61784 Add a cobra command that implements the entire main function for registry
Use this command in cmd/registry/main.go.

Move debug server to the main command, and change Serve to be a
ListenAndServe function.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-09-09 14:39:31 -07:00
Aaron Lehmann
8dd51d6460 Move initialization code from main.go to the registry package
This makes it easier to embed a registry instance inside another
application.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-09-09 14:39:31 -07:00
Jack Griffin
8ceca304b0 Skip creating swift container if already exists
Signed-off-by: Jack Griffin <jackpg14@gmail.com>
2015-09-04 18:53:17 +00:00
Hua Wang
8f5f6a4e59 Add TrustId parameter to swift driver
github/ncw/swift has added support for trust, so let's add it.

Signed-off-by: Hua Wang <wanghua.humble@gmail.com>
2015-09-02 15:05:29 +08:00
Stephen Day
0306c5c453 Merge pull request #934 from stevvooe/sanitize-url-correctly
Correctly sanitize location url preserving parameters
2015-08-27 22:30:21 -07:00
Stephen J Day
6e7718dfce Correctly sanitize location url preserving parameters
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-08-27 17:06:03 -07:00
Stephen J Day
bb098c72a2 Move manifest package to schema1
As we begin our march towards multi-arch, we must prepare for the reality of
multiple manifest schemas. This is the beginning of a set of changes to
facilitate this. We are both moving this package into its target position where
it may live peacefully next to other manfiest versions.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-08-21 16:29:47 -07:00
Aaron Lehmann
5b804f7600 Add headers parameter for HTTP checker
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-08-20 15:11:16 -07:00
Aaron Lehmann
ca3d460278 Add a TCP health checker
Also, add timeout and status code parameters to the HTTP checker, and
remove the threshold parameter for the file checker.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-08-20 15:11:15 -07:00
Aaron Lehmann
cdc3143b7e Expose a Registry type in health package, so unit tests can stay isolated from each other
Update docs.

Change health_test.go tests to create their own registries and register
the checks there. The tests now call CheckStatus directly instead of
polling the HTTP handler, which returns results from the default
registry.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-08-20 15:07:54 -07:00
Aaron Lehmann
bbd4699166 Switch tests to import "github.com/docker/distribution/context"
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-08-20 14:50:12 -07:00
Aaron Lehmann
68e8532cef Add storagedriver section to health check configuration
Add default storagedriver health check to example configuration files
with parameters matching the previous hardcoded configuration.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-08-20 14:50:12 -07:00
Aaron Lehmann
c48e460933 Add configurable file-existence and HTTP health checks
Add a section to the config file called "health". Within this section,
"filecheckers" and "httpcheckers" list checks to run. Each check
specifies a file or URI, a time interval for the check, and a threshold
specifying how many times the check must fail to reach an unhealthy
state.

Document the new options in docs/configuration.md.

Add unit testing for both types of checkers. Add an UnregisterAll
function in the health package to support the unit tests, and an
Unregister function for consistency with Register.

Fix a string conversion problem in the health package's HTTP checker.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-08-20 14:50:12 -07:00
Stephen Day
0189e8c218 Merge pull request #893 from aaronlehmann/functional-opts
Functional options for NewRegistryWithDriver
2015-08-19 14:41:08 -07:00
Aaron Lehmann
142b68aaa2 Add a unit test which verifies the ResponseWriter endpoints see implements CloseNotifier
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-08-19 11:37:53 -07:00
Aaron Lehmann
7fb68446cc Functional options for NewRegistryWithDriver
Clean up calling convention for NewRegistryWithDriver to use functional
arguments.

This is a first step towards the refactor described in #215. I plan to
add additional options in the process of moving configurable items from
the App structure to the registry structure.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-08-19 11:06:30 -07:00
Aaron Lehmann
11133181fc Fix CloseNotifier handling and avoid "the ResponseWriter does not implement CloseNotifier" warnings in logs
A change in #763 to address review comments caused problems. Originally,
instrumentedResponseWriter implemented the CloseNotifier interface, and
would panic if it was wrapping something that did not implement that
interface. This was split into a separate instrumentedResponseWriterCN
type that implements CloseNotifier, so there's a fallback if
instrumentedResponseWriter ever needs to wrap something that does not
implement this interface.

instrumentedResponseWriter's Value method would end up upcasting either
type back to instrumentedResponseWriter, which does not implement the
interface. In effect, instrumentedResponseWriterCN was never visible to
the handler.

This fixes the problem by implementing a wrapper Value method for
instrumentedResponseWriterCN.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-08-18 17:28:51 -07:00