Commit graph

2207 commits

Author SHA1 Message Date
Richard Scothern
c9fd26e9ef Merge pull request #1895 from hinshun/improve-enumerate
Improve catalog enumerate runtime by an order of magnitude
2016-08-11 10:53:36 -07:00
Richard Scothern
baca174469 Merge pull request #1892 from stevvooe/use-math-rand-reader
testutil, storage: use math/rand.Read where possible
2016-08-11 10:04:36 -07:00
Stephen J Day
040db51795
testutil, storage: use math/rand.Read where possible
Use the much faster math/rand.Read function where cryptographic
guarantees are not required. The unit test suite should speed up a
little bit but we've already optimized around this, so it may not
matter.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2016-08-10 14:26:12 -07:00
Edgar Lee
2f81b3b058 Improve catalog enumerate runtime by an order of magnitude
Signed-off-by: Edgar Lee <edgar.lee@docker.com>
2016-08-09 17:42:26 -07:00
Richard Scothern
9000745401 Merge pull request #1888 from stevvooe/more-efficient-path-comparison
registry/storage: more efficient path compare in catalog
2016-08-09 13:45:07 -07:00
Stephen Day
c270829c6e Merge pull request #1887 from hinshun/fix-catalog-enumerate
Stop ErrFinishedWalk from escaping from Repositories walk
2016-08-09 13:44:20 -07:00
Derek McGowan
2f356d69c8 Merge pull request #1894 from RichardScothern/circle
Remove godep check from build and increase test timout
2016-08-09 12:06:30 -07:00
Richard Scothern
a8ef842a9b Remove godep check from build and increase test timout
Signed-off-by: Richard Scothern <richard.scothern@docker.com>
2016-08-09 11:50:44 -07:00
Stephen J Day
308faf00f1
catalog: add benchmarks for overridden path comparison
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2016-08-08 17:09:49 -07:00
Stephen J Day
bba5a0d05c
registry/storage: more efficient path compare in catalog
Previous component-wise path comparison is recursive and generates a
large amount of garbage. This more efficient version simply replaces the
path comparison with the zero-value to sort before everything. We do
this by replacing the byte-wise comparison that swaps a single character
inline for the separator comparison, such that separators sort first.

The resulting implementation provides component-wise path comparison
with no cost incurred for allocation or stack frame.

Direction of the comparison is also reversed to match Go style.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2016-08-08 15:55:26 -07:00
Edgar Lee
379312c148 Stop ErrFinishedWalk from escaping from Repositories walk
Signed-off-by: Edgar Lee <edgar.lee@docker.com>
2016-08-05 17:21:48 -07:00
Richard Scothern
bfa0a9c097 Merge pull request #1877 from spacexnice/master
function copyContent() cause socket leak.
2016-08-02 09:15:23 -07:00
yaoyao.xyy
64c8dd3252 when deploy registry as a pull through cache ,function copeContent() may cause a socket leak when docker user canceled its pull operation.
Signed-off-by: yaoyao.xyy <yaoyao.xyy@alibaba-inc.com>
2016-08-02 11:10:10 +08:00
Michal Minář
7365003236 Provide stat descriptor for Create method during cross-repo mount (#1857)
* Allow precomputed stats on cross-mounted blobs

Signed-off-by: Michal Minář <miminar@redhat.com>

* Extended cross-repo mount tests

Signed-off-by: Michal Minář <miminar@redhat.com>
2016-08-01 19:07:11 -07:00
Frank Chen
87917f3052 Add 'objectAcl' Option to the S3 Storage Backend (#1867)
* Add Object ACL Support to the S3 Storage Backend

Signed-off-by: Frank Chen <frankchn@gmail.com>

* Made changes per @RichardScothern's comments

Signed-off-by: Frank Chen <frankchn@gmail.com>

* Fix Typos

Signed-off-by: Frank Chen <frankchn@gmail.com>
2016-07-27 12:26:57 -07:00
Richard Scothern
4abae2ae56 Merge pull request #1861 from adamvduke/adamvduke/reorder-dockerfile-for-layer-caching-optimization
reorder Dockerfile steps for better layer caching
2016-07-22 11:07:24 -07:00
Adam Duke
41c87559c6 reorder Dockerfile steps for better layer caching
Running `apk add` before copying source into the image takes better
adavantage of layer caching when developing and regularly building the
image. This avoids source code changes invalidating the `apk add` layer
and causing that step to run on every image build.

Signed-off-by: Adam Duke <adam.v.duke@gmail.com>
2016-07-22 13:51:31 -04:00
Richard Scothern
c2a201dabf Merge pull request #1860 from Seb-Solon/master
Fix #1854
2016-07-22 10:04:01 -07:00
Richard Scothern
08d0d3756c Merge pull request #1862 from RichardScothern/fix-build
Fix the build.
2016-07-22 09:38:27 -07:00
Richard Scothern
2287f1c42f Fix the build.
Pass the manifestURL directly into the schema2 manifest handler instead of
accessing through the repository as it has since the reference is now an
interface.

Signed-off-by: Richard Scothern <richard.scothern@docker.com>
2016-07-21 17:16:47 -07:00
Richard Scothern
2b72dd3927 Merge pull request #1829 from nwt/foreign-layer-host-whitelist
Add a foreign layer URL host whitelist
2016-07-21 16:02:20 -07:00
Aaron Lehmann
f56b143dc8 Merge pull request #1858 from RichardScothern/decouple
Decouple storage components by redefining dependencies as interfaces
2016-07-21 11:12:05 -07:00
Sebastien Coavoux
a9c9daf0f2 Fix: Compare path properly when list repository in catalog. #1854
Signed-off-by: Sebastien Coavoux <alignak@pyseb.cx>
2016-07-21 11:46:00 -04:00
Sebastien Coavoux
3290e2bef5 Enh: Test - Edit catalog_test to reproduce #1854
Signed-off-by: Sebastien Coavoux <alignak@pyseb.cx>
2016-07-21 11:23:19 -04:00
Richard Scothern
2a5fcacdf0 Decouple storage components by redefining dependencies as interfaces instead of concrete types
Signed-off-by: Richard Scothern <richard.scothern@docker.com>
2016-07-20 15:09:11 -07:00
Richard Scothern
857d0f15c0 Merge pull request #1757 from miminar/cross-mount-middleware-repository
Allow to use repo middleware wrapper during cross-repo mounts
2016-07-19 17:43:10 -07:00
Richard Scothern
dd355e95af Merge pull request #1686 from marcusmartins/nbf-leeway
Add leeway to JWT nbf checking
2016-07-19 17:41:57 -07:00
Noah Treuhaft
61e5803b56 Add control over validation of URLs in pushed manifests
Until we have some experience hosting foreign layer manifests, the Hub
operators wish to limit foreign layers on Hub. To that end, this change
adds registry configuration options to restrict the URLs that may appear
in pushed manifests.

Signed-off-by: Noah Treuhaft <noah.treuhaft@docker.com>
2016-07-19 14:38:42 -07:00
Richard Scothern
7b5958798e Merge pull request #1852 from jfindley/load-middleware-redirect
Load the middleware/redirect driver
2016-07-19 09:10:25 -07:00
Richard Scothern
96a1aec65d Merge pull request #1850 from runcom/typo
manifest: versioned: fix typo
2016-07-19 09:08:42 -07:00
Richard Scothern
f27ceb7ab5 Merge pull request #1710 from majewsky/swift/wait-for-dlo-segments-during-read
[Swift] add simple heuristic to detect incomplete DLOs during read ops
2016-07-19 09:07:44 -07:00
James Findley
6b98c68d12 Load the middleware/redirect driver
Signed-off-by: James Findley <jfindley@fastmail.com>
2016-07-19 15:37:10 +01:00
Antonio Murdaca
2ea0a41872
manifest: versioned: fix typo
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-07-19 13:54:41 +02:00
Marcus Martins
db1bf93098
Add leeway to JWT nbf and exp checking
Adds a constant leeway (60 seconds) to the nbf and exp claim check to
account for clock skew between the registry servers and the
authentication server that generated the JWT.

The leeway of 60 seconds is a bit arbitrary but based on the RFC
recommendation and hub.docker.com logs/metrics where we don't see
drifts of more than a second on our servers running ntpd.

I didn't attempt to make the leeway configurable as it would add extra
complexity to the PR and I am not sure how Distribution prefer to
handle runtime flags like that.

Also, I am simplifying the exp and nbf check for readability as the
previous `NOT (A AND B)` with cmp operators was not very friendly.

Ref:
https://tools.ietf.org/html/rfc7519#section-4.1.5

Signed-off-by: Marcus Martins <marcus@docker.com>
2016-07-18 17:47:30 -07:00
Richard Scothern
aa3de8fd4f Merge pull request #1847 from RichardScothern/go-1.7
Fix vet errors with go 1.7
2016-07-18 14:01:47 -07:00
Richard Scothern
fcea6145f5 Fix vet errors with go 1.7
Signed-off-by: Richard Scothern <richard.scothern@docker.com>
2016-07-18 13:41:19 -07:00
Derek McGowan
f8083b7ff3 Merge pull request #1836 from hinshun/catalog-walk-return-error
Fix storage drivers dropping non EOF errors when listing repositories
2016-07-15 14:29:50 -07:00
Derek McGowan
b1ab3bfde5 Merge pull request #1838 from dmcgowan/search-v2-auth-test
Search v2 auth test
2016-07-15 14:27:56 -07:00
Derek McGowan
42a7e6e171 Merge pull request #1837 from RichardScothern/catalog-test
Use registry APIs to generate catalog test environment
2016-07-15 14:23:22 -07:00
Edgar Lee
a82f661ef0 Use typecast over reflect for error type checking
Signed-off-by: Edgar Lee <edgar.lee@docker.com>
2016-07-14 15:03:18 -07:00
Edgar Lee
3bfd03cbe6 Refactor errVal named parameter for catalog repositories to err
Signed-off-by: Edgar Lee <edgar.lee@docker.com>
2016-07-14 13:28:08 -07:00
Edgar Lee
aeb9a29499 Handle new errors returned from catalog repository listing
Signed-off-by: Edgar Lee <edgar.lee@docker.com>
2016-07-14 13:24:16 -07:00
Richard Scothern
b126d6643e Use distribution APIs to generate catalog test environment
Signed-off-by: Richard Scothern <richard.scothern@docker.com>
2016-07-14 09:58:19 -07:00
Derek McGowan
fc07e0380e Add v1 search with v2 token auth test
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-07-13 17:03:14 -07:00
Edgar Lee
5a0b35ca10 Fix storage drivers dropping non EOF errors when listing repositories
This fixes errors other than io.EOF from being dropped when a storage driver
lists repositories. For example, filesystem driver may point to a missing
directory and errors, which then gets subsequently dropped.

Signed-off-by: Edgar Lee <edgar.lee@docker.com>
2016-07-13 16:41:51 -07:00
Derek McGowan
37b5e3e81b Merge pull request #1833 from aaronlehmann/document-toomanyrequests
Document TOOMANYREQUESTS error code
2016-07-13 10:41:01 -07:00
Richard Scothern
07f32ac183 Merge pull request #1830 from dmcgowan/allow-v2-auth-on-v1-search
Add support for using v2 ping challenges for v1
2016-07-12 17:28:54 -07:00
Derek McGowan
145abeea7b Add support for using v2 ping challenges for v1
Allows using v2 for v1 endpoints.
The primary use case being for search which does not have a v2 specification.
Added a user scope for allowing v2 search

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-07-12 17:15:56 -07:00
Derek McGowan
022416c502 Add support for registry type in scope
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-07-12 17:13:43 -07:00
Richard Scothern
c3c5277007 Merge pull request #1834 from dmcgowan/aws-vendor-update
Vendor update for aws sdk
2016-07-12 15:26:16 -07:00