Commit graph

115 commits

Author SHA1 Message Date
Joseph Schorr
171c7e5238 Further fixes for unicode handling in manifests
We were occasionally trying to compute schema 2 version 1 signatures on the *unicode* representation, which was failing the signature check. This PR adds a new wrapper type called `Bytes`, which all manifests must take in, and which handles the unicodes vs encoded utf-8 stuff in a central location. This PR also adds a test for the manifest that was breaking in production.
2019-01-09 15:14:41 -05:00
Joseph Schorr
6b5064aba4 Convert V2's manifest endpoints to use the new data model interface 2018-10-18 14:25:40 -04:00
Joseph Schorr
4520f9e842 Change V1 to use the manifest builder and new registry data model 2018-09-28 17:30:04 -04:00
Joseph Schorr
f252b0b16f Move verbs endpoint to use new registry data model 2018-08-29 16:38:54 -04:00
Joseph Schorr
65c1594a81 Remove ACI tests because they are brittle 2018-07-24 17:47:52 -04:00
Joseph Schorr
3ce8a0279a Increase liveserver timeout on older registry tests 2018-07-24 15:51:21 -04:00
Joseph Schorr
188ea98441 Add new decorator to prevent reflected text attacks
Instead of disabling repo names with periods in them, we simply disallow calls to the API when they are GET requests, whose path ends in a dot, and that do not have a referrer from the frontend.
2018-02-20 11:33:45 -05:00
Joseph Schorr
b342111edb Add registry tests for pushing and pulling previously bad repo names 2018-02-20 11:31:36 -05:00
Joseph Schorr
bbdf9e074c Add metrics for tracking when instance key renewal succeeds and fails, as well as when instance key *lookup* fails 2018-02-02 11:14:42 -05:00
Joseph Schorr
524d77f527 Add an AppSpecificAuthToken data model for app-specific auth tokens. These will be used for the Docker CLI in place of username+password 2018-01-04 15:27:41 -05:00
Joseph Schorr
3ce9d68a3e Fix broken registry test
Flask now returns a 404 error, rather than redirecting like it used to do
2017-12-20 11:43:55 -05:00
Joseph Schorr
f9bd7ef42b Add validation of Docker V2_1 schemas and add a test for PUTing an invalid schema 2017-12-20 11:43:03 -05:00
Joseph Schorr
11e3724919 Return an http 415 (manifest version not supported) for OCI manifest content types
This was breaking skopeo, as it first tries to send the *OCI* manifest type, which we didn't say we didn't support, thus breaking the tool
2017-12-20 11:02:34 -05:00
Joseph Schorr
b2485934ed Enable caching of blobs in V2 registry protocol, to avoid DB connections after the cache has been loaded
This should help for bursty pull traffic, as it will avoid DB connections on a huge % of requests
2017-12-14 13:38:24 -05:00
Joseph Schorr
a706d99849 Add additional logs and an additional test for verbs 2017-12-07 15:22:20 -05:00
Joseph Schorr
2677720577 Fix exception raised for certain non-JSON strings given to is_json
This is breaking pushes in production for certain manifests

Fixes https://jira.prod.coreos.systems/browse/QS-60
2017-11-14 13:46:06 -05:00
Joseph Schorr
c5d8b5f86b Add support for tag expiration based on a quay.expires-after label 2017-07-19 17:13:06 -04:00
Joseph Schorr
555041876d Make public catalog only enabled via a feature flag 2017-06-02 15:30:51 -07:00
Joseph Schorr
065e327190 Make sure catalog always returns public repositories 2017-05-19 16:24:08 -04:00
Jimmy Zelinskie
e2c25ce9bc registry tests: assert audit log metadata 2017-05-16 17:05:31 -04:00
Joseph Schorr
f9e6110f73 Add basic user interface for application repos
Adds support for creating app repos, viewing app repos and seeing the list of app repos in the Quay UI.
2017-04-05 11:30:09 -04:00
Joseph Schorr
dcb970b783 Add registry app repository failure test 2017-03-22 17:26:59 -04:00
Joseph Schorr
b5bb76cdea Optimize repository search by changing our lookup strategy
Previous to this change, repositories were looked up unfiltered in six different queries, and then filtered using the permissions model, which issued a query per repository found, making search incredibly slow. Instead, we now lookup a chunk of repositories unfiltered and then filter them via a single query to the database. By layering the filtering on top of the lookup, each as queries, we can minimize the number of queries necessary, without (at the same time) using a super expensive join.

Other changes:
- Remove the 5 page pre-lookup on V1 search and simply return that there is one more page available, until there isn't. While technically not correct, it is much more efficient, and no one should be using pagination with V1 search anyway.
- Remove the lookup for repos without entries in the RAC table. Instead, we now add a new RAC entry when the repository is created for *the day before*, with count 0, so that it is immediately searchable
- Remove lookup of results with a matching namespace; these aren't very relevant anyway, and it overly complicates sorting
2017-03-09 19:47:55 -05:00
Joseph Schorr
a0bc0e9488 Implement the full spec for the old Docker V1 registry search API
This API is still (apparently) being used by the Docker CLI for `docker search` (why?!) and we therefore have customers expecting this to work the same way as the DockerHub.
2017-02-16 14:45:33 -05:00
Joseph Schorr
b3a74b94b3 Fix flaky port selection in registry tests
Depends on https://github.com/jarus/flask-testing/pull/98

[Fixes #136705135]
2017-01-18 15:06:11 -05:00
Joseph Schorr
481cebe46b Fix pytests and enable parallel registry tests 2016-12-20 15:42:04 -05:00
Joseph Schorr
bab17932ac Fix namespace lookup in V1 registry search
Fixes #2053
2016-10-31 13:24:40 -04:00
Joseph Schorr
3439f814b6 Fix quoting of scopes in WWW-Authenticate header
Fixes part of #2002
2016-10-17 14:32:43 -04:00
Joseph Schorr
18097a1bd6 Fix Link headers for pagination to match RFC
Fixes part of #2002
2016-10-17 13:57:05 -04:00
Joseph Schorr
c3ce491f02 Add a sleep to fix ACI conversion tests
This ensures that the cache is cleared before the second request is made
2016-10-14 13:36:47 -04:00
Jake Moshenko
df1f35e9f9 Always use absolute URLs in Location headers.
This works around docker/docker#15048
2016-10-10 16:30:24 -04:00
Joseph Schorr
a046141708 Add parent mis-ordered registry test 2016-10-04 19:26:12 +03:00
Joseph Schorr
f4e1748bb7 Fix parent rewrite bug in schema1 manifest code and add a bunch more tests
Before this change, if you ended up writing a middle layer whose parent is not in the database, the manifest would fail to rewrite. We now just lookup the parent image in the manifest given to us, ignoring whether it is in the database or not (as it doesn't actually matter if not present; it'll be created if necessary).
2016-10-04 09:15:27 -04:00
Joseph Schorr
f72cb1d2ba Fix tags API pagination and add a test 2016-10-03 22:06:31 +03:00
Joseph Schorr
0b7bb6d6c6 Fix issue in V1 registry code with accessing locations under HEAD
Fixes #1922
2016-10-03 17:09:12 +03:00
Joseph Schorr
95b7b47501 Add a registry tests for numeric tags 2016-10-03 16:06:49 +03:00
josephschorr
684ace3b5a Merge pull request #1761 from coreos-inc/nginx-direct-download
Add feature flag to force all direct download URLs to be proxied
2016-09-29 22:46:57 +02:00
Joseph Schorr
6ae3faf7fc Add explicit config parameter to the JWT auth methods 2016-09-29 11:15:20 +02:00
Joseph Schorr
dd2e086a20 Add feature flag to force all direct download URLs to be proxied
Fixes #1667
2016-09-29 11:13:41 +02:00
Jimmy Zelinskie
fc7301be0d *: fix legacy imports
This change reorganizes imports and renames the legacy flask extensions.
2016-09-28 20:17:14 -04:00
Joseph Schorr
3c8b87e086 Fix verbs in manifestlist
All registry_tests now pass
2016-09-26 14:49:58 -04:00
Joseph Schorr
db60df827d Implement V2 interfaces and remaining V1 interfaces
Also adds some tests to registry tests for V1 stuff.
Note: All *registry* tests currently pass, but as verbs are not yet converted, the verb tests in registry_tests.py currently fail.
2016-09-26 14:49:04 -04:00
Joseph Schorr
1864196d8d TAR creation is not deterministic, so we can't test repush consistently 2016-08-29 16:38:12 -04:00
Joseph Schorr
357005e33f Raise a 409 if we try to insert a tag twice at the same time
Also fixes handling of labels for existing manifests

Fixes #1775
2016-08-29 16:03:35 -04:00
Joseph Schorr
608ffd9663 Basic labels support
Adds basic labels support to the registry code (V2), and the API. Note that this does not yet add any UI related support.
2016-08-26 15:24:26 -04:00
Joseph Schorr
c4daf1cc3d Change permissions model so that non-admins do not get org-wide read
Fixes #1684
2016-08-04 16:47:28 -04:00
Joseph Schorr
3d558f6090 Disable ACI tests when build under Docker
We need to figure out why they fail on our build fleet
2016-07-13 14:23:30 -04:00
Joseph Schorr
a43b741f1b Add a uniqueness hash to derived image storage to break caching over tags
This allows converted ACIs and squashed images to be unique based on the specified tag.

Fixes #92
2016-06-20 16:34:52 -04:00
Joseph Schorr
fea47bdaed Increase test counter to 10 2016-06-13 17:31:42 -04:00
Joseph Schorr
4747dea395 Fix flaky tests 2016-06-13 16:00:55 -04:00