Commit graph

1444 commits

Author SHA1 Message Date
Olivier Gambier
e56124d343 Merge pull request #312 from stevvooe/add-layer-info-cache
registry: integrate layer info cache with registry and storage
2015-04-03 13:53:27 -07:00
Doug Davis
e5408bd911 Remove engine.Table from docker search and fix missing field
registry/SearchResults was missing the "is_automated" field.
I added it back in.

Pull this 'table' removal one from the others because it fixed
a bug too

Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-04-03 11:27:30 -07:00
Stephen J Day
6b748a74ef Move expvar under the registry section
For consistency with other systems, the redis and caching monitoring data has
been moved under the "registry" section in expvar. This ensures the entire
registry state is kept to a single section.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-02 21:30:27 -07:00
Stephen J Day
4e1ecad6cc Allow control over which storage cache to use
This allows one to better control the usage of the cache and turn it off
completely. The storage configuration module was modified to allow parameters
to be passed to just the storage implementation, rather than to the driver.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-02 20:15:16 -07:00
Stephen J Day
6ab228f798 Integrate layer info cache with registry and storage
This changeset integrates the layer info cache with the registry webapp and
storage backend. The main benefit is to cache immutable layer meta data,
reducing backend roundtrips. The cache can be configured to use either redis or
an inmemory cache.

This provides massive performance benefits for HEAD http checks on layer blobs
and manifest verification.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-02 20:15:16 -07:00
Stephen J Day
a7c2dceea5 Define and implement layer info cache
This changeset defines the interface for layer info caches. Layer info caches
speed up access to layer meta data accessed in storage driver backends. The
two main operations are tests for repository membership and resolving path and
size information for backend blobs.

Two implementations are available. The main implementation leverages redis to
store layer info. An alternative implementation simply caches layer info in
maps, which should speed up resolution for less sophisticated implementations.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-02 20:15:09 -07:00
Stephen Day
dff57726f9 Merge pull request #295 from jlhawn/use_resumable_digest
digest, registry/storage: use resumable digest
2015-04-02 20:12:57 -07:00
Stephen Day
721b39d0cb Merge pull request #314 from stevvooe/lockdown-uuid-route
registry/api/v2: stronger validation for uuid field in urls
2015-04-02 13:57:17 -07:00
Josh Hawn
b96de45be8 Use resumable digest for efficient upload finish
By using a resumable digester and storing the state of upload digests between
subsequent upload chunks, finalizing an upload no longer requires reading back
all of the uploaded data to verify the client's expected digest.

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
2015-04-02 11:20:31 -07:00
Stephen J Day
06acde06cb Avoid crash on invalid Move arguments
This chnage prevents a crash when moving from a non-existent directory that has
a file as a parent. To prevent this, we simply check that the node is a
directory and throws an error if it is not.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-01 19:03:02 -07:00
Stephen J Day
6eb804a1ec Stronger validation for uuid field in urls
This change adds strong validation for the uuid variable for v2 routes. This is
a minor specification change but is okay since the uuid field is controlled by
the server. The character set is restricted to avoid path traversal, allowing
for alphanumeric values and urlsafe base64 encoding.

This change has no effect on client implementations.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-01 18:57:59 -07:00
Stephen J Day
38ae1cb461 Add redis pool to registry webapp
Redis has been integrated with the web application for use with various
services. The configuraiton exposes connection details, timeouts and pool
parameters. Documentation has been updated accordingly.

A few convenience methods have been added to the context package to get loggers
with certain fields, exposing some missing functionality from logrus.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-01 16:27:24 -07:00
Michael Crosby
638ccff564 Remove jobs from registry.Service
This makes `registry.Service` a first class type and does not use jobs
to interact with this type.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2015-03-31 16:38:04 -07:00
Derek McGowan
67e5c940c4 Use vendored v2 registry api
Update registry package to use the v2 registry api from distribution. Update interfaces to directly take in digests.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-03-31 15:02:27 -07:00
Antonio Murdaca
5fa2d814f8 Refactor utils/http.go, fixes #11899
Signed-off-by: Antonio Murdaca <me@runcom.ninja>
2015-03-30 17:59:57 +02:00
Brian Goff
9e9a8add19 Merge pull request #11788 from reikani/pchoi
Changed snake case naming to camelCase.
2015-03-26 23:55:50 -07:00
Antonio Murdaca
d5045d054b Replace aliased imports of logrus, fixes #11762
Signed-off-by: Antonio Murdaca <me@runcom.ninja>
2015-03-26 23:22:04 +01:00
Peter Choi
b085d5556e Changed snake case naming to camelCase
Signed-off-by: Peter Choi <phkchoi89@gmail.com>
2015-03-26 15:05:45 -06:00
Don Kjer
eff5278d12 Fix for issue 9922: private registry search with auth returns 401
Signed-off-by: Don Kjer <don.kjer@gmail.com>
2015-03-26 18:59:16 +00:00
Alexander Morozov
e4afe03dcc Merge pull request #11761 from runcom/11731-remove-engine-status
Remove engine.Status, replace with standard go error
2015-03-25 15:41:02 -07:00
Antonio Murdaca
9c08a43624 Remove engine.Status and replace it with standard go error
Signed-off-by: Antonio Murdaca <me@runcom.ninja>
2015-03-25 22:32:08 +01:00
Stephen Day
9ee35877e3 Merge pull request #294 from fkautz/pr_out_updating_msopentech_azure_sdk_for_go_to_latest_master
Updating MSOpenTech/azure-sdk-for-go to latest master
2015-03-24 13:27:19 -07:00
Frederick F. Kautz IV
dffd1babd2 Updating MSOpenTech/azure-sdk-for-go to latest master
Signed-off-by: Frederick F. Kautz IV <fkautz@alumni.cmu.edu>
2015-03-23 21:59:21 -07:00
Ahmet Alp Balkan
594f733e03 storage/driver/azure: Allow non-default realms
This enables Azure storage driver to be used with non-default
cloud endpoints like Azure China or Azure Government that does
not use `.blob.core.windows.net` FQDN suffix.

Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
2015-03-23 20:41:16 -07:00
Derek McGowan
10128f6e8c Add struct tags on v2 remote tags struct
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-03-23 14:33:52 -07:00
Meaglith Ma
0e7650f958 Fix decode tags value error when call get /v2/<name>/tags/list in registry api v2.
Signed-off-by: Meaglith Ma <genedna@gmail.com>
2015-03-23 14:33:52 -07:00
Derek McGowan
9f5184c111 Add check for 404 on get repository data
No longer add the body to the error when a 404 is received on get repository data.

closes #11510

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-03-22 18:15:18 -07:00
Michael Crosby
50b10850bb Merge pull request #11528 from jlhawn/separate_init_blob_upload
Separate init blob upload
2015-03-20 16:39:31 -07:00
Josh Hawn
bcccf35bb2 Separate init blob upload
Pushing a v2 image layer has two steps:

- POST to get a new upload URL
- PUT to that upload URL

We were previously not checking the response code of
the POST request and the PUT would fail in weird ways.

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
2015-03-20 13:11:05 -07:00
Josh Hawn
11d08c3065 Add verification of image manifest digests
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
2015-03-20 09:34:39 -07:00
Arnaud Porterie
4377a9a3bc Merge pull request #11477 from dmcgowan/fix-auth-http-client
Update auth client configuration to use proper tls config
2015-03-19 14:17:47 -07:00
Derek McGowan
4bf6791328 Update auth client configuration to use proper tls config
Currently the http clients used by auth use the default tls config.  The config needs to be updated to only support TLS1.0 and newer as well as respect registry insecure configuration.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-03-18 14:52:49 -07:00
Doug Davis
59a1d4721d Merge pull request #11314 from HammerOfThor/pull_err_print
print detailed error info for docker pull
2015-03-18 08:08:01 -04:00
Zhang Wei
7d4c1d1e97 print detailed error info for docker pull
When docker push get response with unknown HTTP status, docker daemon
print:
"Error: Status XXX trying to push repository XXX: XXX"
But when docker pull meets response with unknown status code, it gives:
"HTTP code: XXX"

This commit helps docker pull print more detailed error info like push
does, so push and pull can behave consistently when error happens.

Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
2015-03-18 11:35:44 +08:00
Arnaud Porterie
606d538448 Merge pull request #11118 from shishir-a412ed/restriction_username_length
Docker Tag command: Relax the restriction on namespace (username) length from 30 to 255 characters.
2015-03-17 09:41:12 -07:00
Arnaud Porterie
00889e1519 Merge pull request #11109 from ncdc/pull-by-digest
Add support for referring to images by digest
2015-03-17 08:15:45 -07:00
Andy Goldstein
4b813b3847 Add ability to refer to image by name + digest
Add ability to refer to an image by repository name and digest using the
format repository@digest. Works for pull, push, run, build, and rmi.

Signed-off-by: Andy Goldstein <agoldste@redhat.com>
2015-03-17 10:10:42 +00:00
Derek McGowan
1d6ccc1b72 Quote registry error strings
Currently when registry error strings contain new line characters only the last line is displayed to the client. Quote the string to ensure the client can see the entire body value.

fixes #11346

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-03-16 15:32:47 -07:00
Derek McGowan
9879aefa81 Use request factory for registry ping
Currently when the registry ping is sent, it creates the request directly from http.NewRequest instead of from the http request factory. The request factory adds useful header information such as user agent which is needed by the registry.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-03-16 14:21:00 -07:00
Shishir Mahajan
dc4e9c6e90 Docker Tag command: Relax the restriction on namespace (username) length from 30 to 255 characters.
Signed-off-by: Shishir Mahajan <shishir.mahajan@redhat.com>
2015-03-14 16:45:44 -04:00
Stephen Day
af4f5aa7d2 Merge pull request #261 from jlhawn/url_for_method
Refactor Layer interface to return a Handler
2015-03-13 17:27:42 -07:00
Josh Hawn
6d14019368 Refactor Layer interface to return a Handler
... Rather than ServeHTTP directly.

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
2015-03-12 21:59:07 -07:00
Stephen Day
b8cbceb179 Merge pull request #258 from jlhawn/url_for_method
Insert request method option storage driver URLFor
2015-03-12 17:16:54 -07:00
Josh Hawn
fdd6314776 Insert request method option storage driver URLFor
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
2015-03-12 17:06:40 -07:00
Stephen Day
efb24490c7 Merge pull request #256 from ncdc/master
Send WWW-Authenticate header for silly auth
2015-03-11 19:01:23 -07:00
Andy Goldstein
4b5af16fdc Send WWW-Authenticate header for silly auth
Signed-off-by: Andy Goldstein <agoldste@redhat.com>
2015-03-11 15:20:15 -04:00
Stephen Day
10881152ac Merge pull request #244 from endophage/cloudfront_refactor
registry/middleware, registry/storage, configuration: refactoring cloudfront + generic middlewares
2015-03-11 12:10:49 -07:00
David Lawrence
83571e574c don't panic during a request when configuring repository middleware. Return a 500 with an appropriate error
Signed-off-by: David Lawrence <david.lawrence@docker.com> (github: endophage)
2015-03-11 08:59:02 -07:00
Andy Goldstein
5c3f53b70f Fix Godoc typos
Signed-off-by: Andy Goldstein <agoldste@redhat.com>
2015-03-11 08:56:29 -07:00
David Lawrence
6a72d1aefb Final polish to cloudfront and larger middleware refactor
Signed-off-by: David Lawrence <david.lawrence@docker.com> (github: endophage)
2015-03-11 08:56:29 -07:00