The original implementation wrote to different locations in a shared slice.
While this is theoretically okay, we end up thrashing the cpu cache since
multiple slice members may be on the same cache line. So, even though each
thread has its own memory location, there may be contention over the cache
line. This changes the code to aggregate to a slice in a single goroutine.
In reality, this change likely won't have any performance impact. The theory
proposed above hasn't really even been tested. Either way, we can consider it
and possibly go forward.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Rather than enforce lowercase paths for all drivers, support for
case-sensitivity has been deferred to the driver. There are a few caveats to
this approach:
1. There are possible security implications for tags that only differ in their
case. For instance, a tag "A" may be equivalent to tag "a" on certain file
system backends.
2. All system paths should not use case-sensitive identifiers where possible.
This might be problematic in a blob store that uses case-sensitive ids. For
now, since digest hex ids are all case-insensitive, this will not be an issue.
The recommend workaround is to not run the registry on a case-insensitive
filesystem driver in security sensitive applications.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
To avoid compounded round trips leading to slow retrieval of manifests with a
large number of signatures, the fetch of signatures has been parallelized. This
simply spawns a goroutine for each path, coordinated with a sync.WaitGroup.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
registry/SearchResults was missing the "is_automated" field.
I added it back in.
Pull this 'table' removal one from the others because it fixed
a bug too
Signed-off-by: Doug Davis <dug@us.ibm.com>
For consistency with other systems, the redis and caching monitoring data has
been moved under the "registry" section in expvar. This ensures the entire
registry state is kept to a single section.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
This allows one to better control the usage of the cache and turn it off
completely. The storage configuration module was modified to allow parameters
to be passed to just the storage implementation, rather than to the driver.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
This changeset integrates the layer info cache with the registry webapp and
storage backend. The main benefit is to cache immutable layer meta data,
reducing backend roundtrips. The cache can be configured to use either redis or
an inmemory cache.
This provides massive performance benefits for HEAD http checks on layer blobs
and manifest verification.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
This changeset defines the interface for layer info caches. Layer info caches
speed up access to layer meta data accessed in storage driver backends. The
two main operations are tests for repository membership and resolving path and
size information for backend blobs.
Two implementations are available. The main implementation leverages redis to
store layer info. An alternative implementation simply caches layer info in
maps, which should speed up resolution for less sophisticated implementations.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
By using a resumable digester and storing the state of upload digests between
subsequent upload chunks, finalizing an upload no longer requires reading back
all of the uploaded data to verify the client's expected digest.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
This chnage prevents a crash when moving from a non-existent directory that has
a file as a parent. To prevent this, we simply check that the node is a
directory and throws an error if it is not.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
This change adds strong validation for the uuid variable for v2 routes. This is
a minor specification change but is okay since the uuid field is controlled by
the server. The character set is restricted to avoid path traversal, allowing
for alphanumeric values and urlsafe base64 encoding.
This change has no effect on client implementations.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Redis has been integrated with the web application for use with various
services. The configuraiton exposes connection details, timeouts and pool
parameters. Documentation has been updated accordingly.
A few convenience methods have been added to the context package to get loggers
with certain fields, exposing some missing functionality from logrus.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
This makes `registry.Service` a first class type and does not use jobs
to interact with this type.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Update registry package to use the v2 registry api from distribution. Update interfaces to directly take in digests.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
This enables Azure storage driver to be used with non-default
cloud endpoints like Azure China or Azure Government that does
not use `.blob.core.windows.net` FQDN suffix.
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
No longer add the body to the error when a 404 is received on get repository data.
closes#11510
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Pushing a v2 image layer has two steps:
- POST to get a new upload URL
- PUT to that upload URL
We were previously not checking the response code of
the POST request and the PUT would fail in weird ways.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Currently the http clients used by auth use the default tls config. The config needs to be updated to only support TLS1.0 and newer as well as respect registry insecure configuration.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
When docker push get response with unknown HTTP status, docker daemon
print:
"Error: Status XXX trying to push repository XXX: XXX"
But when docker pull meets response with unknown status code, it gives:
"HTTP code: XXX"
This commit helps docker pull print more detailed error info like push
does, so push and pull can behave consistently when error happens.
Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
Add ability to refer to an image by repository name and digest using the
format repository@digest. Works for pull, push, run, build, and rmi.
Signed-off-by: Andy Goldstein <agoldste@redhat.com>
Currently when registry error strings contain new line characters only the last line is displayed to the client. Quote the string to ensure the client can see the entire body value.
fixes#11346
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Currently when the registry ping is sent, it creates the request directly from http.NewRequest instead of from the http request factory. The request factory adds useful header information such as user agent which is needed by the registry.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
