Fixes#3223 by bumping logrus to v1.6.0, which in turn bumps
github.com/konsorten/go-windows-terminal-sequences to v1.0.3
wherein the fix to bad pointer is found.
Signed-off-by: Andreas Hassing <andreas@famhassing.dk>
A repository need not contain any unique layers, if its images use only layers
mounted from other repositories. But, the catalog endpoint was looking for the
_layers directory to indicate that a directory was a repository.
Use the _manifests directory as the marker instead, since any repository with
revisions will contain a _manifests directory.
Signed-off-by: Adam Wolfe Gordon <awg@digitalocean.com>
Instead of constructing the list of credential providers manually, if we
use the default list we can take advantage of the AWS SDK checking the
environment and returning either the EC2RoleProvider or the generic HTTP
credentials provider, configured to use the ECS credentials endpoint.
Also, use the `defaults.Config()` function instead of `aws.NewConfig()`,
as this results in an initialised HTTP client which prevents a fatal
error when retrieving credentials from the ECS credentials endpoint.
Fixes#2960
Signed-off-by: Andrew Bulford <andrew.bulford@redmatter.com>
- Modify Travis manifest to use Go 1.14.x
- Hardcode version of golangci-lint in setup script
- Hardcode version of go-md2man in setup script
Signed-off-by: jdolitsky <393494+jdolitsky@users.noreply.github.com>
The metrics tracker in cached blob statter was replaced with prometheus
metrics and no longer needed.
Remove unused log wrapping.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
* Update redis.go
If the dgst key does not exist in the cache when calling HGET, `redis.String` will return an `ErrNil` which we need to translate into `distribution.ErrBlobUnknown` so that the error being returned can be properly handled. This will ensure that `SetDescriptor` is properly called from `cachedBlobStatter::Stat` for `repositoryScopedRedisBlobDescriptorService` which will update the redis cache and be considered as a Miss rather than an Error.
cc @manishtomar
* Update suite.go
Add unit test to ensure missing blobs for scoped repo properly return ErrBlobUnknown when HGET returns redis.ErrNil.
(cherry picked from commit dca6b9526a1d30dd218a9f321c4f84ecc4b5e62e)
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
* redis metrics
it is working but metrics are not very useful since default buckets
start from 5ms and almost all of them are in that range.
* remove extra comment
(cherry picked from commit ba1a1d74e7eb047dd1056548ccf0695e8846782c)
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
* fix redis caching issue
earlier redis cache was updated when there was any error including any
temporary connectivity issue. This would trigger set calls which would
further increase load and possibly connectivity errors from redis
leaving the system with continuous errors and high latency. Now the
cache is updated only when it is genuine cache miss. Other errors do not
trigger a cache update.
* add back tracker Hit() and Miss() calls
*squashed commits*
(cherry picked from commit 6f3e1c10260ef59ba4e9c42e939329fad9fdd8c3)
(cherry picked from commit 6738ff3320cf82cc2df919a95a1bde2f7789a501)
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Update format of governance and bring in language from containerd.
Update maintainers to reflect active maintainers.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Unit test coverge was increased to cover the usages of crypto. This helps to ensure that everything is working fine with fips mode enabled.
Also updated sha1 to sha256 in registry/storage/driver/testsuites/testsuites.go because sha1 is not supported in fips mode.
Signed-off-by: Naveed Jamil <naveed.jamil@tenpearl.com>
Includes 69ecbb4d6d
(forward-port of 8b5121be2f),
which fixes CVE-2020-7919:
- Panic in crypto/x509 certificate parsing and golang.org/x/crypto/cryptobyte
On 32-bit architectures, a malformed input to crypto/x509 or the ASN.1 parsing
functions of golang.org/x/crypto/cryptobyte can lead to a panic.
The malformed certificate can be delivered via a crypto/tls connection to a
client, or to a server that accepts client certificates. net/http clients can
be made to crash by an HTTPS server, while net/http servers that accept client
certificates will recover the panic and are unaffected.
Thanks to Project Wycheproof for providing the test cases that led to the
discovery of this issue. The issue is CVE-2020-7919 and Go issue golang.org/issue/36837.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
full diff: https://github.com/golang/go/compare/go1.13.4...go1.13.7
go1.13.7 (released 2020/01/28) includes two security fixes. One mitigates
the CVE-2020-0601 certificate verification bypass on Windows. The other affects
only 32-bit architectures.
https://github.com/golang/go/issues?q=milestone%3AGo1.13.7+label%3ACherryPickApproved
- X.509 certificate validation bypass on Windows 10
A Windows vulnerability allows attackers to spoof valid certificate chains when
the system root store is in use. These releases include a mitigation for Go
applications, but it’s strongly recommended that affected users install the
Windows security update to protect their system.
This issue is CVE-2020-0601 and Go issue golang.org/issue/36834.
- Panic in crypto/x509 certificate parsing and golang.org/x/crypto/cryptobyte
On 32-bit architectures, a malformed input to crypto/x509 or the ASN.1 parsing
functions of golang.org/x/crypto/cryptobyte can lead to a panic.
The malformed certificate can be delivered via a crypto/tls connection to a
client, or to a server that accepts client certificates. net/http clients can
be made to crash by an HTTPS server, while net/http servers that accept client
certificates will recover the panic and are unaffected.
Thanks to Project Wycheproof for providing the test cases that led to the
discovery of this issue. The issue is CVE-2020-7919 and Go issue golang.org/issue/36837.
This is also fixed in version v0.0.0-20200124225646-8b5121be2f68 of golang.org/x/crypto/cryptobyte.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Ensures that build tags get set in the Dockerfile so that OSS and GCS drivers
are built into the official registry binary.
Closes#2819
Signed-off-by: Ryan Abrams <rdabrams@gmail.com>
(cherry picked from commit bf74e4f91d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Use a synthetic upstream registry when creating the testing mirror configuration
to avoid the test fail when trying to reach http://example.com
Signed-off-by: Fernando Mayo Fernandez <fernando@undefinedlabs.com>
Some registries (ECR) don't provide a `Docker-Upload-UUID` when creating
a blob. So we can't rely on that header. Fallback to reading it from the
URL.
Signed-off-by: Damien Mathieu <dmathieu@salesforce.com>
I've found this logic being in a single method to be quite hard to get.
I believe extracting it makes it easier to read, as we can then more
easily see what the main method does and possibly ignore the intricacies
of `ResumeBlobUpload`.
Signed-off-by: Damien Mathieu <dmathieu@salesforce.com>
When uploading segments to Swift, the registry generates a random file,
by taking the hash of the container path and 32-bytes of random data.
The registry attempts to shard across multiple directory paths, by
taking the first three hex characters as leader.
The implementation in registry, unfortunately, takes the hash of
nothing, and appends it to the path and random data. This results in all
segments being created in one directory.
Fixes: #2407Fixes: #2311
Signed-off-by: Terin Stock <terinjokes@gmail.com>
fetchTokenWithBasicAuth checks if a token is in the token response
but fetchTokenWithOAuth does not
these changes implements the same behaviour for the latter
returning a `ErrNoToken` if a token is not found in the resposne
Signed-off-by: Sevki Hasirci <sevki@cloudflare.com>
Radosgw does not support S3 `GET Bucket` API v2 API but v1.
This API has backward compatibility, so most of this API is working
correctly but we can not get `KeyCount` in v1 API and which is only
for v2 API.
Signed-off-by: Eohyung Lee <liquidnuker@gmail.com>
Use mime.ParseMediaType to parse the media types in Accept header in manifest request. Ignore the failed ones.
Signed-off-by: Yu Wang <yuwa@microsoft.com>
It adds notification related prometheus metrics, including:
- total count for events/success/failure/error
- total count for notification per each status code
- gauge of the pending notification queue
Signed-off-by: tifayuki <tifayuki@gmail.com>
This fixes registry endpoints to return the proper `application/json`
content-type for JSON content, also updating spec examples for that.
As per IETF specification and IANA registry [0], the `application/json`
type is a binary media, so the content-type label does not need any
text-charset selector. Additionally, the media type definition
explicitly states that it has no required nor optional parameters,
which makes the current registry headers non-compliant.
[0]: https://www.iana.org/assignments/media-types/application/json
Signed-off-by: Luca Bruno <lucab@debian.org>
ParseDockerRef normalizes the image reference following the docker
convention. This is added mainly for backward compatibility. The reference
returned can only be either tagged or digested. For reference contains both tag
and digest, the function returns digested reference, e.g.
docker.io/library/busybox:latest@sha256:7cc4b5aefd1d0cadf8d97d4350462ba51c694ebca145b08d7d41b41acc8db5aa
will be returned as
docker.io/library/busybox@sha256:7cc4b5aefd1d0cadf8d97d4350462ba51c694ebca145b08d7d41b41acc8db5aa.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Add an interface alongside TagStore that provides API to retreive
digests of all manifests that a tag historically pointed to. It also
includes currently linked tag.
Signed-off-by: Manish Tomar <manish.tomar@docker.com>
Remove redundant AUTHORS file in favor of relying
on .mailmap for keeping track of author names.
Move older changelogs to release directory.
Move release checklist to releases README.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
It's possible to run into a race condition in which the enumerator lists
lots of repositories and then starts the long process of enumerating through
them. In that time if someone deletes a repo, the enumerator may error out.
Signed-off-by: Ryan Abrams <rdabrams@gmail.com>
This is done by draining the connections for configured time after registry receives a SIGTERM signal.
This adds a `draintimeout` setting under `HTTP`. Registry doesn't drain
if draintimeout is not provided.
Signed-off-by: Manish Tomar <manish.tomar@docker.com>
context.App.repoRemover is single registry instance stored throughout
app run. It was wrapped in another remover when processing each request.
This remover happened to be remover got from previous request. This way
every remover created was stored in infinite linked list causing memory
leak. Fixing it by storing the wrapped remover inside the request context
which will get gced when request context is gced. This was introduced in
PR #2648.
Signed-off-by: Manish Tomar <manish.tomar@docker.com>
Indentation is spelled wrong as intendation everywhere the word occurs
Can either be:
> "found a tab character that violates indentation"
or
> "found tab characters that violate indentation"
Signed-off-by: Parth Mehrotra <parth@mehrotra.me>
by having another interface RepositoryRemover that is implemented by
registry instance and is injected in app context for event tracking
Signed-off-by: Manish Tomar <manish.tomar@docker.com>
OCI Image manifests and indexes are supported both with and without
an embeded MediaType (the field is reserved according to the spec).
Test storing and retrieving both types from the manifest store.
Signed-off-by: Owen W. Taylor <otaylor@fishsoup.net>
In the OCI image specification, the MediaType field is reserved
and otherwise undefined; assume that manifests without a media
in storage are OCI images or image indexes, and determine which
by looking at what fields are in the JSON. We do keep a check
that when unmarshalling an OCI image or image index, if it has
a MediaType field, it must match that media type of the upload.
Signed-off-by: Owen W. Taylor <otaylor@fishsoup.net>
When unmarshalling manifests from JSON, check that the MediaType field
corresponds to the type that we are unmarshalling as. This makes sure
that when we retrieve a manifest from the manifest store, it will have
the same type as it was handled as before storing it in the manifest
store.
Signed-off-by: Owen W. Taylor <otaylor@fishsoup.net>
According golang documentation [1]: no goroutine should expect to be
able to acquire a read lock until the initial read lock is released.
[1] https://golang.org/pkg/sync/#RWMutex
Signed-off-by: Gladkov Alexey <agladkov@redhat.com>
Include rules and roles in file.
Remove inactive maintainers who have already acknowledged
leaving the maintainer role.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
at the first iteration, only the following metrics are collected:
- HTTP metrics of each API endpoint
- cache counter for request/hit/miss
- histogram of storage actions, including:
GetContent, PutContent, Stat, List, Move, and Delete
Signed-off-by: tifayuki <tifayuki@gmail.com>
This adds a configuration setting `HTTP.TLS.LetsEncrypt.Hosts` which can
be set to a list of hosts that the registry will whitelist for retrieving
certificates from Let's Encrypt. HTTPS connections with SNI hostnames
that are not whitelisted will be closed with an "unknown host" error.
It is required to avoid lots of unsuccessful registrations attempts that
are triggered by malicious clients connecting with bogus SNI hostnames.
NOTE: Due to a bug in the deprecated vendored rsc.io/letsencrypt library
clearing the host list requires deleting or editing of the cachefile to
reset the hosts list to null.
Signed-off-by: Felix Buenemann <felix.buenemann@gmail.com>
This removes the old global walk function, and changes all
the code to use the per-driver walk functions.
Signed-off-by: Sargun Dhillon <sargun@sargun.me>
This changes the Walk Method used for catalog enumeration. Just to show
how much an effect this has on our s3 storage:
Original:
List calls: 6839
real 3m16.636s
user 0m0.000s
sys 0m0.016s
New:
ListObjectsV2 Calls: 1805
real 0m49.970s
user 0m0.008s
sys 0m0.000s
This is because it no longer performs a list and stat per item, and instead
is able to use the metadata gained from the list as a replacement to stat.
Signed-off-by: Sargun Dhillon <sargun@sargun.me>
Move the Walk types into registry/storage/driver, and add a Walk method to each
storage driver. Although this is yet another API to implement, there is a fall
back implementation that relies on List and Stat. For some filesystems this is
very slow.
Also, this WalkDir Method conforms better do a traditional WalkDir (a la filepath).
This change is in preparation for refactoring.
Signed-off-by: Sargun Dhillon <sargun@sargun.me>
This upgrade, and vendors aws-sdk-go to version v1.12.36.
This is because it has new API calls accessible to the S3 client,
specifically S3.ListObjectsV2PagesWithContext
Signed-off-by: Sargun Dhillon <sargun@sargun.me>
Fixed the headers. They were rendering incorrectly on GitHub for some weird reason; I copied the strings and rewrote the ## characters at the beginning and now they render as they should.
Signed-off-by: Per Lundberg <per.lundberg@ecraft.com>
If tenant or tenantid are passed as env variables, we systematically use Sprint to make sure they are string and not integer as it would make mapstructure fail.
Signed-off-by: Raphaël Enrici <raphael@root-42.com>
The previous code assumed that the link returned when listing tags was
always absolute. However, some registries, such as quay.io, return the
link as a relative link (e.g. the second page for the quay.io/coreos/etcd
image is /v2/coreos/etcd/tags/list?next_page=<truncated>&n=50). Because
the relative link was retrieved directly, the fetch failed (with the
error `unsupported protocol scheme ""`).
Signed-off-by: Kevin Lin <kevin@kelda.io>
If the overwriteStruct() finds an uninitialized pointer, it tries to initialize it,
but does it incorrectly. It tries to assign a pointer to pointer, instead of pointer.
Signed-off-by: Gladkov Alexey <agladkov@redhat.com>
If htpasswd authentication option is configured but the htpasswd file is
missing, populate it with a default user and automatically generated
password.
The password will be printed to stdout.
Signed-off-by: Liron Levin <liron@twistlock.com>
1. when lookup an entry which is missing, it should say NotFound.
2. when add duplicated entry, the entries size should be increased.
3. when add entry which has different algorithm, it should be allowed.
Signed-off-by: zhouhaibing089 <zhouhaibing089@gmail.com>
the digest cli does not work if we do not import this two packages,
tested in go1.9. basically, we have to make several algorithms to
be available by calling crypto.RegisterHash in init functions.
Signed-off-by: zhouhaibing089 <zhouhaibing089@gmail.com>
The current registry/client sends the registered manifest types in
random order. Allow clients to request a single specific manifest type
or a preferred order as per the HTTP spec.
Signed-off-by: Clayton Coleman <ccoleman@redhat.com>
A statically hosted registry that responds correctly to GET with a
manifest will load the right digest (by looking at the manifest body and
calculating the digest). If the registry returns a HEAD without
`Docker-Content-Digest`, then the client Tags().Get() call will return
an empty digest.
This commit changes the client to fallback to loading the tag via GET if
the `Docker-Content-Digest` header is not set.
Signed-off-by: Clayton Coleman <ccoleman@redhat.com>
AuthorizeRequest() injects the 'pull' scope if `from` is set
unconditionally. If the current token already has that scope, it will
be inserted into the scope list twice and `addedScopes` will be set to
true, resulting in a new token being fetched that has no net new scopes.
Instead, check whether `additionalScopes` are actually new.
Signed-off-by: Clayton Coleman <ccoleman@redhat.com>
To simplify the vendoring story for the client, we have now removed the
requirement for `logrus` and the forked `context` package (usually
imported as `dcontext`). We inject the logger via the metrics tracker
for the blob cache and via options on the token handler. We preserve
logs on the proxy cache for that case. Clients expecting these log
messages may need to be updated accordingly.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Back in the before time, the best practices surrounding usage of Context
weren't quite worked out. We defined our own type to make usage easier.
As this packaged was used elsewhere, it make it more and more
challenging to integrate with the forked `Context` type. Now that it is
available in the standard library, we can just use that one directly.
To make usage more consistent, we now use `dcontext` when referring to
the distribution context package.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Under certain circumstances, the use of `StorageDriver.GetContent` can
result in unbounded memory allocations. In particualr, this happens when
accessing a layer through the manifests endpoint.
This problem is mitigated by setting a 4MB limit when using to access
content that may have been accepted from a user. In practice, this means
setting the limit with the use of `BlobProvider.Get` by wrapping
`StorageDriver.GetContent` in a helper that uses `StorageDriver.Reader`
with a `limitReader` that returns an error.
When mitigating this security issue, we also noticed that the size of
manifests uploaded to the registry is also unlimited. We apply similar
logic to the request body of payloads that are full buffered.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
If the client doesn't support manifest lists, the registry will
rewrite a manifest list into the old format. The Docker-Content-Digest
header should be updated in this case.
Signed-off-by: Oleg Bulatov <oleg@bulatov.me>
In some conditions, regulator.exit may not send a signal to blocked
regulator.enter.
Let's assume we are in the critical section of regulator.exit and r.available
is equal to 0. And there are three more gorotines. One goroutine also executes
regulator.exit and waits for the lock. Rest run regulator.enter and wait for
the signal.
We send the signal, and after releasing the lock, there will be lock
contention:
1. Wait from regulator.enter
2. Lock from regulator.exit
If the winner is Lock from regulator.exit, we will not send another signal to
unlock the second Wait.
Signed-off-by: Oleg Bulatov <obulatov@redhat.com>
Add build args. Defaults to Linux/x64 so no change to existing image, but can build for other platforms - e.g.
```
docker build --build-arg GOOS=windows -t distribution-builder:windows .
```
Signed-off-by: Elton Stoneman <elton@sixeyed.com>
Updates resumable hash implementation to Go 1.8 equivalent. This should
be a major speedup, since it includes a number of optimizations from Go
1.7.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
This updates the grammar to allow more than one digest algorithm
separator, matching the regular expression and intended grammar.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
It turns out that origin/master may not be up to date in CircleCI
checkouts. Fetch origin so that diffing for the vendor check can be done
correctly.
Note that this doesn't help the case of PRs against branches other than
master, but the worst case is that those will waste a few seconds doing
unnecessary vendor validation. Since those PRs are relatively rare, that
seems fine.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
A stray space in the link for the migrator repository wasn't allowing it
to be rendered properly in markdown viewers. Remove such a space to
avoid this.
Signed-off-by: Santiago Torres-Arias <torresariass@gmail.com>
Partially reverts change adding support for X-Forwarded-Port.
Changes the logic to prefer the standard Forwarded header over
X-Forwarded headers. Prefer forwarded "host" over "for" since
"for" represents the client and not the client's request.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Currently, this target makes changes to the vendor directory, which may
overwrite local changes or cause build errors. It's better to restore
the original vendor directory after running the check.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
The dep-validate target appears to never get invoked. In the CircleCI
build environment, "master" points to the commit under test. The
circle.yml fragment needs to compare again "origin/master" instead.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Since there's no default case, if there's not a tag or digest you get
back a confusing error from the router about it not matching the
expected pattern.
Also redoing the tests for URLs a bit so that they can handle checking
for failures.
Signed-off-by: Christy Perez <christy@linux.vnet.ibm.com>
Remove EndpointConfig.Transport from the return value of the
registry.notifications.endpoints expvar.Func. It results in an empty
value for that expvar variable under Go 1.7 because it is a non-nil
*http.Transport, which Go 1.7 can no longer encode as JSON.
Signed-off-by: Noah Treuhaft <noah.treuhaft@docker.com>
A previous inspection of the code surrounding zero-length blobs led to
some interesting question. After inspection, it was found that the hash
was indeed for the empty string (""), and not an empty tar, so the code
was correct. The variable naming and comments have been updated
accordingly.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
The common use case for this function is to add a default tag if the
reference only has a name. The current behavior only adds the default
tag if there is no *tag*, which requires most callers to check for a
digest. Change the behavior to only add default tags to name-only
references, and change the name to reflect this. The documentation
already described the new behavior, so it does not need to be changed.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
The registry uses partial Named values which the named parsers
no longer support. To allow the registry service to continue
to operate without canonicalization, switch to use WithName.
In the future, the registry should start using fully canonical
values on the backend and WithName should no longer support
creating partial values.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
The identifier fields were using spaces while the rest
of the lines were using tabs!
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Add details for release checklist for where to put the update
the changelist and which branch to cherry-pick from. Also add
more details around creating the tag and verifying the tag is
correct.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
The NormalizedNamed interface has shown to not be necessary for
integrating the change downstream. The FamiliarName and FamiliarString
helpers are the only used interface and allow hiding the normalized
completely.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
When get manifest, the handler will try to retrieve it from storage driver. When storage driver is cloud storage, it can fail due to various reasons even if the manifest exists
(like 500, 503, etc. from storage server). Currently manifest handler blindly return 404 which can be confusing to user.
This change will return 404 if the manifest blob doesn't exist, and return 500 UnknownError for all other errors (consistent with the behavior of other handlers).
Signed-off-by: Yu Wang (UC) <yuwa@microsoft.com>
Officials repositories always have 2 part names with the first part
being library and second part being the offical repository name. Names
with more than 2 parts should not hit the special case for official
repositories since they are not valid official repositories.
Add tests for this ambiguity and to ensure that 3 part names are
supports for the default repository, as used by the docker store.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Once upon a time, we referred to manifests and images interchangably.
That simple past is no more. As we grow, we update our nomenclature and
so follows our code.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
If running test behide a proxy, we may get the error code
403 Forbidden which will fail line 135 for the last testcase.
Detail:
```
metrics not as expected: notifications.EndpointMetrics{Pending:0,
Events:0, Successes:4, Failures:0, Errors:0,
Statuses:map[string]int{"307 Temporary Redirect":0, "400 Bad Request":0,
"403 Forbidden":0, "200 OK":4}} !=
notifications.EndpointMetrics{Pending:0, Events:0, Successes:4,
Failures:0, Errors:0, Statuses:map[string]int{"400 Bad Request":0, "200
OK":4, "307 Temporary Redirect":0}}
```
Immediate close will fix that
Signed-off-by: Hu Keping <hukeping@huawei.com>
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
NamedRepository only needs to be used internally. Additionally this
interface may go away in the future and be merged with Named.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Add interface for for a normalized name and corresponding parser for that type.
New normalized versions of all interfaces are not added since all type information is preserved on calls to Familiar.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Allows having other parsers which are capable of unambiguously keeping domain and path separated in a Reference type.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
With token authentication, requiring the "*" action for DELETE requests
makes it impossible to administratively lock a repository against pushes
and pulls but still allow deletion. This change adds a new "delete"
action for DELETE requests to make that possible.
Signed-off-by: Noah Treuhaft <noah.treuhaft@docker.com>
`app.driver.List` on `"/"` is very expensive if registry contains significant amount of images. And the result isn't used anyways.
In most (if not all) storage drivers, `Stat` has a cheaper implementation, so use it instead to achieve the same goal.
Signed-off-by: yixi zhang <yixi@memsql.com>
Modify manifest builder so it can be used to build
manifests with different configuration media types.
Rename config media type const to image config.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Enforces backwards compatibility with older authorization servers
without requiring the client to know about the compatibility
requirements.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
See #2077 for background.
The PR #1438 which was not reviewed by azure folks basically introduced
a race condition around uploads to the same blob by multiple clients
concurrently as it used the "writer" type for PutContent(), introduced in #1438.
This does chunked upload of blobs using "AppendBlob" type, which was not atomic.
Usage of "writer" type and thus AppendBlobs on metadata files is currently not
concurrency-safe and generally, they are not the right type of blob for the job.
This patch fixes PutContent() to use the atomic upload operation that works
for uploads smaller than 64 MB and creates blobs with "BlockBlob" type. To be
backwards compatible, we query the type of the blob first and if it is not
a "BlockBlob" we delete the blob first before doing an atomic PUT. This
creates a small inconsistency/race window "only once". Once the blob is made
"BlockBlob", it is overwritten with a single PUT atomicallly next time.
Therefore, going forward, PutContent() will be producing BlockBlobs and it
will silently migrate the AppendBlobs introduced in #1438 to BlockBlobs with
this patch.
Tested with existing code side by side, both registries with and without this
patch work fine without breaking each other. So this should be good from a
backwards/forward compatiblity perspective, with a cost of doing an extra
HEAD checking the blob type.
Fixes#2077.
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
Vndr has a simpler configuration and allows pointing to forked
packages. Additionally other docker projects are now using
vndr making vendoring in distribution more consistent.
Updates letsencrypt to use fork.
No longer uses sub-vendored packages.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Use whitelist of allowed repository classes to enforce.
By default all repository classes are allowed.
Add authorized resources to context after authorization.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Update grammar to support a resource class. Add
example for plugin repository class.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Golint now checks for new lines at the end of go error strings,
remove these unneeded new lines.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Add a simple helper to create name only references without having
to first get the name as a string and reparse. This method does
not fail so ignoring or checking error to simply trim reference
is not required.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Allow clients to handle errors being set in the WWW-Authenticate
rather than in the body. The WWW-Authenticate errors give a
more precise error describing what is needed to authorize
with the server.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Split challenges into its own package. Avoids possible
import cycle with challenges from client.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
When WithDigest is called on a reference that has a tag, it should
preserve the tag.
When WithTag is called on a reference that has digest, it should
preserve the digest.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Updating to a recent version of Azure Storage SDK to be
able to patch some memory leaks through configurable HTTP client
changes which were made possible by recent patches to it.
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
The current code determines the header order for the
"string-to-sign" payload by sorting on the concatenation
of headers and values, whereas it should only happen on the
key.
During multipart uploads, since `x-amz-copy-source-range` and
`x-amz-copy-source` headers are present, V2 signatures fail to
validate since header order is swapped.
This patch reverts to the expected behavior.
Signed-off-by: Pierre-Yves Ritschard <pyr@spootnik.org>
Prefer non-standard headers like X-Forwarded-Proto, X-Forwarded-Host and
X-Forwarded-Port over the standard Forwarded header to maintain
backwards compatibility.
If a port is not specified neither in Host nor in forwarded headers but
it is specified just with X-Forwarded-Port, use its value in base urls
for redirects.
Forwarded header is defined in rfc7239.
X-Forwarded-Port is a non-standard header. Here's a description copied
from "HTTP Headers and Elastic Load Balancing" of AWS ELB docs:
> The X-Forwarded-Port request header helps you identify the port that
> an HTTP or HTTPS load balancer uses to connect to the client.
Signed-off-by: Michal Minář <miminar@redhat.com>
Driver was passing connections by copying. Storing
`swift.Connection` as pointer to fix the warnings.
Ref: #2030.
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
In GetContent() we read the bytes from a blob but do not close
the underlying response body.
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
To allow generic manifest walking, we define an interface method of
`References` that returns the referenced items in the manifest. The
current implementation does not return the config target from schema2,
making this useless for most applications.
The garbage collector has been modified to show the utility of this
correctly formed `References` method. We may be able to make more
generic traversal methods with this, as well.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Some frontmatter such as the weights, menu stuff, etc is no longer used
'draft=true' becomes 'published: false'
Signed-off-by: Misty Stanley-Jones <misty@docker.com>
Use find instead of wildcard to get go files since wildcard does not
do recursive lookups via `**`, missing most go files.
Do lazy evaluations of variables which are not needed immediately.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Context should use type values instead of strings.
Updated direct calls to WithValue, but still other uses of string keys.
Update Acl to ACL in s3 driver.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Reading the oauth2 token documentation is misleading as it makes
no mention of it being a newer feature which may not be supported
by the token server. Add a note mentioning if it is not supported
to refer to the token documentation for getting a token.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
The Redis tests were failing with a "connection pool exhausted" error
from Redigo. Closing the connection used for FLUSHDB fixes the problem.
Signed-off-by: Noah Treuhaft <noah.treuhaft@docker.com>
The Hub registry generates a large volume of notifications, many of
which are uninteresting based on target media type. Discarding them
within the notification endpoint consumes considerable resources that
could be saved by discarding them within the registry. To that end,
this change adds registry configuration options to restrict the
notifications sent to an endpoint based on target media type.
Signed-off-by: Noah Treuhaft <noah.treuhaft@docker.com>
Access logging is great. Access logging you can turn off is even
better. This change adds a configuration option for that.
Signed-off-by: Noah Treuhaft <noah.treuhaft@docker.com>
Use the same error message as is used in
docker/reference/reference.go to provide
slightly more information about the error.
This change tests if the reference passes
after lowercasing its characters, to determine
if the reference was invalid due to
it containing uppercase characters.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The token auth package logs JWT validation and verification failures at
the `error` level. But from the server's perspective, these aren't
errors. They're the expected response to bad input. Logging them at
the `info` level better reflects that distinction.
Signed-off-by: Noah Treuhaft <noah.treuhaft@docker.com>
Let's Encrypt uses tls-sni to validate the certificate
on the standard https port 443. If the registry is
outwardly listening on a different port Let's Encrypt
will not issue a certificate.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
If a user specifies `mydomain.com:443` in the `Host` configuration, the
PATCH request for the layer upload will fail because the challenge does not
appear to be in the map. To fix this, we normalize the map keys to always
use the Host:Port combination.
Closes https://github.com/docker/docker/issues/18469
Signed-off-by: Stan Hu <stanhu@gmail.com>
Previously, the specificiation incorrectly bound the fates of `urls` and
foreign layers. These are complementary but unrelated features, in that
the `urls` field may be populated for layers that aren't foreign. The
type of the layer only dictates the push behavior of the layer, rather
than involving where it came from.
For example, one may pull a foreign layer from a registry, but they may
not push it back to another registry. Conversely, a layer that has no
restrictions on push/pull behavior may be fetched via `urls` entries.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Running with the race detector may cause some parts
of the code to run slower causing a race in the scheduler
ordering.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Registry handlers tests currently takes up too
many resources for the race test to handle on circle ci.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
* tag service: properly handle error responses on HEAD requests by
re-issuing requests as GET for proper error details.
Fixes#1911.
Signed-off-by: dmitri <deemok@gmail.com>
* Simplify handling of failing HEAD requests in TagService and
make a GET request for cases:
- if the server does not handle HEAD
- if the response was an error to get error details
Signed-off-by: dmitri <deemok@gmail.com>
* Add a missing http.Response.Body.Close call for the GET request.
Signed-off-by: dmitri <deemok@gmail.com>
This change to the S3 Move method uses S3's multipart upload API to copy
objects whose size exceeds a threshold. Parts are copied concurrently.
The level of concurrency, part size, and threshold are all configurable
with reasonable defaults.
Using the multipart upload API has two benefits.
* The S3 Move method can now handle objects over 5 GB, fixing #886.
* Moving most objects, and espectially large ones, is faster. For
example, moving a 1 GB object averaged 30 seconds but now averages 10.
Signed-off-by: Noah Treuhaft <noah.treuhaft@docker.com>
This is already supported by ncw/swift, so we just need to pass the
parameters from the storage driver.
Signed-off-by: Stefan Majewsky <stefan.majewsky@sap.com>
Use the much faster math/rand.Read function where cryptographic
guarantees are not required. The unit test suite should speed up a
little bit but we've already optimized around this, so it may not
matter.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Previous component-wise path comparison is recursive and generates a
large amount of garbage. This more efficient version simply replaces the
path comparison with the zero-value to sort before everything. We do
this by replacing the byte-wise comparison that swaps a single character
inline for the separator comparison, such that separators sort first.
The resulting implementation provides component-wise path comparison
with no cost incurred for allocation or stack frame.
Direction of the comparison is also reversed to match Go style.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
* Allow precomputed stats on cross-mounted blobs
Signed-off-by: Michal Minář <miminar@redhat.com>
* Extended cross-repo mount tests
Signed-off-by: Michal Minář <miminar@redhat.com>
* Add Object ACL Support to the S3 Storage Backend
Signed-off-by: Frank Chen <frankchn@gmail.com>
* Made changes per @RichardScothern's comments
Signed-off-by: Frank Chen <frankchn@gmail.com>
* Fix Typos
Signed-off-by: Frank Chen <frankchn@gmail.com>
Running `apk add` before copying source into the image takes better
adavantage of layer caching when developing and regularly building the
image. This avoids source code changes invalidating the `apk add` layer
and causing that step to run on every image build.
Signed-off-by: Adam Duke <adam.v.duke@gmail.com>
Pass the manifestURL directly into the schema2 manifest handler instead of
accessing through the repository as it has since the reference is now an
interface.
Signed-off-by: Richard Scothern <richard.scothern@docker.com>
Until we have some experience hosting foreign layer manifests, the Hub
operators wish to limit foreign layers on Hub. To that end, this change
adds registry configuration options to restrict the URLs that may appear
in pushed manifests.
Signed-off-by: Noah Treuhaft <noah.treuhaft@docker.com>
Adds a constant leeway (60 seconds) to the nbf and exp claim check to
account for clock skew between the registry servers and the
authentication server that generated the JWT.
The leeway of 60 seconds is a bit arbitrary but based on the RFC
recommendation and hub.docker.com logs/metrics where we don't see
drifts of more than a second on our servers running ntpd.
I didn't attempt to make the leeway configurable as it would add extra
complexity to the PR and I am not sure how Distribution prefer to
handle runtime flags like that.
Also, I am simplifying the exp and nbf check for readability as the
previous `NOT (A AND B)` with cmp operators was not very friendly.
Ref:
https://tools.ietf.org/html/rfc7519#section-4.1.5
Signed-off-by: Marcus Martins <marcus@docker.com>
This fixes errors other than io.EOF from being dropped when a storage driver
lists repositories. For example, filesystem driver may point to a missing
directory and errors, which then gets subsequently dropped.
Signed-off-by: Edgar Lee <edgar.lee@docker.com>
Allows using v2 for v1 endpoints.
The primary use case being for search which does not have a v2 specification.
Added a user scope for allowing v2 search
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Updated to latest version of go aws sdk.
Use vendored sub pakages within aws sdk.
Adds missing vendor packages for letsencrypt
Fixes#1832
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
The EndpointConfig struct in the notifications package has some config
fields for a notification endpoint. This commit adds the ability to pass
in an *http.Transport to use when notifying that endpoint of an event.
This is especially useful for endpoints that use self-signed CAs.
Signed-off-by: Josh Chorlton <josh.chorlton@docker.com>
This is similar to waitForSegmentsToShowUp which is called during
Close/Commit. Intuitively, you wouldn't expect missing segments to be a
problem during read operations, since the previous Close/Commit
confirmed that all segments are there.
But due to the distributed nature of Swift, the read request could be
hitting a different storage node of the Swift cluster, where the
segments are still missing.
Load tests on my team's staging Swift cluster have shown this to occur
about once every 100-200 layer uploads when the Swift proxies are under
high load. The retry logic, borrowed from waitForSegmentsToShowUp, fixes
this temporary inconsistency.
Signed-off-by: Stefan Majewsky <stefan.majewsky@sap.com>
In Go's header parsing, the same header multiple times results in multiple entries in the `r.Header[...]` slice, but Go does no further parsing beyond that (and in https://golang.org/cl/4528086 it was determined that until/unless the stdlib itself needs it, Go will not do so).
The consequence here for parsing of `Accept:` headers is that we support the way Go outputs headers, but not all language HTTP libraries have a facility to output multiple headers instead of a single list header.
This change ensures that the following (valid) header blocks all parse to the same result for the purposes of what is being tested here:
```
Accept: a/b
Accept: b/c
Accept: d/e
```
```
Accept: a/b; q=0.5, b/c
Accept: d/e
```
```
Accept: a/b; q=0.1, b/c; q=0.2, d/e; q=0.8
```
Signed-off-by: Andrew "Tianon" Page <admwiggin@gmail.com>
The client may need the content digest to delete a manifest using the digest used by the registry.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Go will fail to parse the examples since an int is expected rather than a string for the "expires in" value
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
This lets us access registry config within middleware for additional
configuration of whatever it is that you're overriding.
Signed-off-by: Tony Holdstock-Brown <tony@docker.com>
go1.5 doesn't export http.StatusTooManyRequests while
go1.6 does. Fix this by hardcoding the status code for now.
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
This commit refactors base.regulator into the 2.4 interfaces and adds a
filesystem configuration option `maxthreads` to configure the regulator.
By default `maxthreads` is set to 100. This means the FS driver is
limited to 100 concurrent blocking file operations. Any subsequent
operations will block in Go until previous filesystem operations
complete.
This ensures that the registry can never open thousands of simultaneous
threads from os filesystem operations.
Note that `maxthreads` can never be less than 25.
Add test case covering parsable string maxthreads
Signed-off-by: Tony Holdstock-Brown <tony@docker.com>
When we push a schema1 manifest, we encode history information from the
image JSON into v1Compatibility strings for the respective layers. The
"author" field was not being set in these v1Compatibility strings, so if
a parent layer had an author set, it would not be preserved after
pushing through a schema1 manifest and repulling, so the image ID would
change after the pull. This change preserves the authorship information
for parent layers so that the image ID does not change.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
subsequent close.
When a blob upload is cancelled close the blobwriter before removing
upload state to ensure old hashstates don't persist.
Signed-off-by: Richard Scothern <richard.scothern@docker.com>
It's easily possible for a flood of requests to trigger thousands of
concurrent file accesses on the storage driver. Each file I/O call creates
a new OS thread that is not reaped by the Golang runtime. By limiting it
to only 100 at a time we can effectively bound the number of OS threads
in use by the storage driver.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Signed-off-by: Tony Holdstock-Brown <tony@docker.com>
Not just when Commit()ing the result. This fixes some errors I observed
when the layer (i.e. the DLO) is Stat()ed immediately after closing,
and reports the wrong file size because the container listing is not
yet up-to-date.
Signed-off-by: Stefan Majewsky <stefan.majewsky@sap.com>
The kid value can have an arbitrary format according JOSE specification, but Docker distribution expects a specific format (libtrust fingerprint) to work. This is not written in the documentation so far and is only mentioned in the libtrust source code itself.
Signed-off-by: Fabio Huser <fabio@fh1.ch>
Updates the readme to mention running the tests using golem.
Also provides instructions for making test development easier.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
The schema2 manifest builder fills in this part of the manifest based on
the descriptor it gets back from BlobIngester's Put method. It passes
the correct media type to Put, but Put ends up replacing this value with
application/octet-stream in its return value.
This commit works around the issue in the manifest builder. Arguably Put
should not be changing the media type in its return value, but this
commit is a targeted fix to keep it very low-risk for possible inclusion
in Docker 1.11.
Fixes#1621 (but maybe we should open a separate issue for the media
type behavior in the distribution client, and the unnecessary stat).
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Use registry example from golem repository.
Use the golem test runner for the docker integration environment
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Add link to the official list of $GOOS and $GOARCH values and correct
values that were incorrectly listed in the spec examples.
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
Using $GOPATH/bin/godep or $GOPATH/bin/golint is problematic because
$GOPATH can contain multiple colon-separated paths.
We should just run these like normal binaries. The user should make sure
their $PATH contains $GOPATH/bin, if necessary. This is part of normal
Go setup.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
The oauth spec defines using a space to separate parts of a scope.
To better comply with future implementations built on oauth use a space to separate the resource scopes.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
@ -18,7 +11,7 @@ Most people should use the [official Registry docker image](https://hub.docker.c
People looking for advanced operational use cases might consider rolling their own image with a custom Dockerfile inheriting `FROM registry:2`.
OS X users who want to run natively can do so following [the instructions here](osx-setup-guide.md).
OS X users who want to run natively can do so following [the instructions here](https://github.com/docker/docker.github.io/blob/master/registry/recipes/osx-setup-guide.md).
### Gotchas
@ -78,21 +71,19 @@ commands, such as `go test`, should work per package (please see
A `Makefile` has been provided as a convenience to support repeatable builds.
Please install the following into `GOPATH` for it to work:
go get github.com/tools/godep github.com/golang/lint/golint
**TODO(stevvooe):** Add a `make setup` command to Makefile to run this. Have to think about how to interact with Godeps properly.
go get github.com/golang/lint/golint
Once these commands are available in the `GOPATH`, run `make` to get a full
@ -27,7 +34,7 @@ By following these simple rules you will get better and faster feedback on your
- search the bugtracker for an already reported issue
###If you found an issue that describes your problem:
###If you found an issue that describes your problem:
- please read other user comments first, and confirm this is the same issue: a given error condition might be indicative of different problems - you may also find a workaround in the comments
- please refrain from adding "same thing here" or "+1" comments
@ -43,7 +50,7 @@ By following these simple rules you will get better and faster feedback on your
3. copy the command line you used to launch your Registry
4. restart your docker daemon in debug mode (add `-D` to the daemon launch arguments)
5. reproduce your problem and get your docker daemon logs showing the error
@ -51,90 +58,72 @@ By following these simple rules you will get better and faster feedback on your
7. provide any relevant detail about your specific Registry configuration (e.g., storage backend used)
8. indicate if you are using an enterprise proxy, Nginx, or anything else between you and your Registry
## Contributing a patch for a known bug, or a small correction
## Contributing Code
Contributions should be made via pull requests. Pull requests will be reviewed
by one or more maintainers or reviewers and merged when acceptable.
You should follow the basic GitHub workflow:
1. fork
2. commit a change
3. make sure the tests pass
4. PR
1. Use your own [fork](https://help.github.com/en/articles/about-forks)
2. Create your [change](https://github.com/containerd/project/blob/master/CONTRIBUTING.md#successful-changes)
3. Test your code
4. [Commit](https://github.com/containerd/project/blob/master/CONTRIBUTING.md#commit-messages) your work, always [sign your commits](https://github.com/containerd/project/blob/master/CONTRIBUTING.md#commit-messages)
5. Push your change to your fork and create a [Pull Request](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/creating-a-pull-request-from-a-fork)
Additionally, you must [sign your commits](https://github.com/docker/docker/blob/master/CONTRIBUTING.md#sign-your-work). It's very simple:
Refer to [containerd's contribution guide](https://github.com/containerd/project/blob/master/CONTRIBUTING.md#successful-changes)
for tips on creating a successful contribution.
- configure your name with git: `git config user.name "Real Name" && git config user.email mail@example.com`
- sign your commits using `-s`: `git commit -s -m "My commit"`
## Sign your work
Some simple rules to ensure quick merge:
The sign-off is a simple line at the end of the explanation for the patch. Your
signature certifies that you wrote the patch or otherwise have the right to pass
it on as an open-source patch. The rules are pretty simple: if you can certify
the below (from [developercertificate.org](http://developercertificate.org/)):
- clearly point to the issue(s) you want to fix in your PR comment (e.g., `closes #12345`)
- prefer multiple (smaller) PRs addressing individual issues over a big one trying to address multiple issues at once
- if you need to amend your PR following comments, please squash instead of adding more commits
```
Developer Certificate of Origin
Version 1.1
## Contributing new features
Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
660 York Street, Suite 102,
San Francisco, CA 94110 USA
You are heavily encouraged to first discuss what you want to do. You can do so on the irc channel, or by opening an issue that clearly describes the use case you want to fulfill, or the problem you are trying to solve.
Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.
If this is a major new feature, you should then submit a proposal that describes your technical solution and reasoning.
If you did discuss it first, this will likely be greenlighted very fast. It's advisable to address all feedback on this proposal before starting actual work.
Developer's Certificate of Origin 1.1
Then you should submit your implementation, clearly linking to the issue (and possible proposal).
By making a contribution to this project, I certify that:
Your PR will be reviewed by the community, then ultimately by the project maintainers, before being merged.
(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or
It's mandatory to:
(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or
- interact respectfully with other community members and maintainers - more generally, you are expected to abide by the [Docker community rules](https://github.com/docker/docker/blob/master/CONTRIBUTING.md#docker-community-guidelines)
- address maintainers' comments and modify your submission accordingly
- write tests for any new code
(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.
Complying to these simple rules will greatly accelerate the review process, and will ensure you have a pleasant experience in contributing code to the Registry.
(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.
```
Have a look at a great, successful contribution: the [Swift driver PR](https://github.com/docker/distribution/pull/493)
Then you just add a line to every git commit message:
## Coding Style
Signed-off-by: Joe Smith <joe.smith@email.com>
Unless explicitly stated, we follow all coding guidelines from the Go
community. While some of these standards may seem arbitrary, they somehow seem
to result in a solid, consistent codebase.
Use your real name (sorry, no pseudonyms or anonymous contributions.)
It is possible that the code base does not currently comply with these
guidelines. We are not looking for a massive PR that fixes this, since that
goes against the spirit of the guidelines. All new contributions should make a
best effort to clean up and make the code base better than they left it.
Obviously, apply your best judgement. Remember, the goal here is to make the
code base easier for humans to navigate and understand. Always keep that in
mind when nudging others to comply.
The rules:
1. All code should be formatted with `gofmt -s`.
2. All code should pass the default levels of
[`golint`](https://github.com/golang/lint).
3. All code should follow the guidelines covered in [Effective
Go](http://golang.org/doc/effective_go.html) and [Go Code Review
| **registry** | An implementation of the [Docker Registry HTTP API V2](docs/spec/api.md) for use with docker 1.6+. |
| **libraries** | A rich set of libraries for interacting with distribution components. Please see [godoc](https://godoc.org/github.com/docker/distribution) for details. **Note**: These libraries are **unstable**. |
| **specifications** | _Distribution_ related specifications are available in [docs/spec](docs/spec) |
| **documentation** | Docker's full documentation set is available at [docs.docker.com](https://docs.docker.com). This repository [contains the subset](docs/index.md) related just to the registry. |
| **registry** | An implementation of the [OCI Distribution Specification](https://github.com/opencontainers/distribution-spec). |
| **libraries** | A rich set of libraries for interacting with distribution components. Please see [godoc](https://godoc.org/github.com/docker/distribution) for details. **Note**: The interfaces for these libraries are **unstable**. |
| **documentation** | Docker's full documentation set is available at [docs.docker.com](https://docs.docker.com). This repository [contains the subset](docs/) related just to the registry. |
### How does this integrate with Docker engine?
### How does this integrate with Docker, containerd, and other OCI client?
This project should provide an implementation to a V2 API for use in the [Docker
core project](https://github.com/docker/docker). The API should be embeddable
and simplify the process of securely pulling and pushing content from `docker`
daemons.
Clients implement against the OCI specification and communicate with the
registry using HTTP. This project contains an client implementation which
is currently in use by Docker, however, it is deprecated for the
[implementation in containerd](https://github.com/containerd/containerd/tree/master/remotes/docker)
and will not support new features.
### What are the long term goals of the Distribution project?
@ -43,32 +44,20 @@ system that allow users to:
* Implement their own home made solution through good specs, and solid
extensions mechanism.
## More about Registry 2.0
The new registry implementation provides the following benefits:
- faster push and pull
- new, more efficient implementation
- simplified deployment
- pluggable storage backend
- webhook notifications
For information on upcoming functionality, please see [ROADMAP.md](ROADMAP.md).
### Who needs to deploy a registry?
By default, Docker users pull images from Docker's public registry instance.
[Installing Docker](https://docs.docker.com/engine/installation/) gives users this
ability. Users can also push images to a repository on Docker's public registry,
if they have a [Docker Hub](https://hub.docker.com/) account.
if they have a [Docker Hub](https://hub.docker.com/) account.
For some users and even companies, this default behavior is sufficient. For
others, it is not.
others, it is not.
For example, users with their own software products may want to maintain a
registry for private, company images. Also, you may wish to deploy your own
image repository for images used to test or in continuous integration. For these
use cases and others, [deploying your own registry instance](docs/deploying.md)
use cases and others, [deploying your own registry instance](https://github.com/docker/docker.github.io/blob/master/registry/deploying.md)
may be the better choice.
### Migration to Registry 2.0
@ -76,56 +65,27 @@ may be the better choice.
For those who have previously deployed their own registry based on the Registry
1.0 implementation and wish to deploy a Registry 2.0 while retaining images,
data migration is required. A tool to assist with migration efforts has been
created. For more information see [docker/migrator]
(https://github.com/docker/migrator).
created. For more information see [docker/migrator](https://github.com/docker/migrator).
## Contribute
## Contribution
Please see [CONTRIBUTING.md](CONTRIBUTING.md) for details on how to contribute
issues, fixes, and patches to this project. If you are contributing code, see
the instructions for [building a development environment](docs/building.md).
the instructions for [building a development environment](BUILDING.md).
## Support
## Communication
If any issues are encountered while using the _Distribution_ project, several
avenues are available for support:
For async communication and long running discussions please use issues and pull requests on the github repo.
This will be the best place to discuss design and implementation.
