Commit graph

56 commits

Author SHA1 Message Date
Joseph Schorr
eb9ca8e8a8 Fix handling of four byte utf8 manifests
- Adds the charset: utf-8 to all the manifest responses
- Makes sure we connect to MySQL in utf8mb4 mode, to ensure we can properly read and write 4-byte utf8 strings
- Adds tests for all of the above
2019-01-10 16:34:56 -05:00
Joseph Schorr
b1dd053b02 Fix an NPE when trying to pull a manifest without a legacy image via V1 2019-01-10 13:59:35 -05:00
Joseph Schorr
171c7e5238 Further fixes for unicode handling in manifests
We were occasionally trying to compute schema 2 version 1 signatures on the *unicode* representation, which was failing the signature check. This PR adds a new wrapper type called `Bytes`, which all manifests must take in, and which handles the unicodes vs encoded utf-8 stuff in a central location. This PR also adds a test for the manifest that was breaking in production.
2019-01-09 15:14:41 -05:00
Joseph Schorr
48e584905a Make sure to decode manifests into utf-8 when necessary
This fixes a decoding error
2018-12-17 16:31:24 -05:00
Joseph Schorr
d49eec18a3 Add automatic retry to the registry test suite to make it less flaky 2018-12-11 13:30:30 -05:00
Joseph Schorr
3c2e050593 Support pulling of schema2 manifests directly via a manifest list tag
This change ensures that if a manifest list is requested with an accepts header for a *schema 2* manifest, the legacy manifest (if any) is returned as schema 2 if it was pushed as a schema 2 manifest (rather than being auto-converted to schema 1)
2018-12-07 11:57:13 -05:00
Joseph Schorr
c3710a6a5e Add ability for specific geographic regions to be blocked from pulling images within a namespace 2018-12-06 17:14:45 -05:00
Joseph Schorr
4e1ff90cb2 Unify the get_layers calls across all implements of manifest schemas to ensure we have a common type returned
Also renames some methods to make it more clear what kind of information they return
2018-11-26 17:58:48 +02:00
Joseph Schorr
180d8847db Fix image replication for images with remote layers 2018-11-26 16:15:48 +02:00
Joseph Schorr
0eb84f8077 Add a squashing test for images with empty layers 2018-11-26 12:48:20 +02:00
Joseph Schorr
4985040d31 Properly handle the empty layer when pushing schema 2 manifests
Docker doesn't send us the contents of this layer, so we are forced to synthesize it ourselves
2018-11-25 16:16:59 +02:00
Joseph Schorr
49ea2d1fa1 Add a schema 2 push pull verification test 2018-11-21 21:02:13 +02:00
Joseph Schorr
6207bb5e6f Fix registry test due to recent name change 2018-11-21 18:06:16 +02:00
Joseph Schorr
1f03fdb27e Fix pulling of squashed versions of the legacy image in a manifest lists 2018-11-20 16:37:15 +02:00
Joseph Schorr
17d7828f1a Add registry test for manifestlist with the same child manifest twice 2018-11-20 11:22:27 +02:00
Joseph Schorr
e972e4088b Fix lookup of manifests referenced solely by a manifest list
We need to ensure we can find them if there is an active tag pointing to the parent list
2018-11-19 23:31:41 +02:00
Joseph Schorr
54904cfd6e Fix bug around pushing manifest lists that refer to the same manifest twice as children 2018-11-19 18:26:22 +02:00
Joseph Schorr
45db1d27e7 Update registry tests 2018-11-19 14:01:42 +02:00
Joseph Schorr
1eaf5b18dd Adjustments based on code review feedback 2018-11-15 13:51:48 +02:00
Joseph Schorr
276d0d571d Fix support for pulling manifest lists via Docker V1 protocol where applicable 2018-11-14 14:05:06 +02:00
Joseph Schorr
37b20010aa Add support for pushing and pulling schema 2 manifests with remote layers
This is required for windows image support
2018-11-14 13:21:50 +02:00
Joseph Schorr
3b4002877a Fix mimetype handling in registry tests 2018-11-14 08:51:01 +02:00
Joseph Schorr
7a794e29c0 Add tests for manifest lists and fix some issues encountered while testing 2018-11-13 21:03:20 +02:00
Joseph Schorr
e752a9a73f Update registry tests to test schema 2 manifest pushes and pulls
Note that tests for manifest *lists* will be in a following commit
2018-11-13 17:15:00 +02:00
Joseph Schorr
8985d8f106 Have registry tests run against both data models (old and new) 2018-11-13 12:00:55 +02:00
Joseph Schorr
fdcb8bad23 Implement the new OCI-based registry data model
Note that this change does *not* enable the new data model by default, but does allow it to be used when a special environment variable is specified.
2018-11-07 22:07:58 -05:00
Joseph Schorr
6b5064aba4 Convert V2's manifest endpoints to use the new data model interface 2018-10-18 14:25:40 -04:00
Joseph Schorr
4520f9e842 Change V1 to use the manifest builder and new registry data model 2018-09-28 17:30:04 -04:00
Joseph Schorr
3dc9ddc6b3 Add an additional multilayer push/pull test 2018-09-24 12:54:56 -04:00
Joseph Schorr
7567808a10 Fix bug in V1 delete tag code and add a test 2018-09-24 12:54:56 -04:00
Joseph Schorr
3bd1b21ca9 Fix ACI conversion and add a registry test for it 2018-09-13 14:12:23 -04:00
Joseph Schorr
7424a6d73a Add additional multi-layer complex squashing test 2018-09-12 15:26:57 -04:00
Joseph Schorr
f252b0b16f Move verbs endpoint to use new registry data model 2018-08-29 16:38:54 -04:00
Joseph Schorr
f297249100 Move manifest backfill for V1 tags into the new registry model interface 2018-08-27 15:01:27 -04:00
Joseph Schorr
e13baa9041
Merge pull request #3206 from quay/fix-unicode-schemas
Fix unicode manifests in the backfill
2018-08-16 13:24:17 -04:00
Jimmy Zelinskie
6d4300a92a test/registry: add accept header test 2018-08-15 14:49:26 -04:00
Joseph Schorr
fa58f3b1d2 Fix handling of manifests with unicode in the backfill
Also adds a bunch of tests around manifests to ensure we get the same information in and out
2018-08-15 11:41:15 -04:00
Joseph Schorr
701eac5466 Fix registry tests to not terminate prematurely when looking for errors
Also reorders a check in the V1 protocol to have better error messaging for the users
2018-08-10 13:36:20 -04:00
Joseph Schorr
9669320df2 Fix pushing of manifests whose layers share blobs
If a blob was repeated previously, we would get a constraint error from the ManifestBlob table
2018-08-07 13:06:30 -04:00
Joseph Schorr
c34eacb4fa Fix issue accessing a null tagmanifest under a tag in a repository 2018-07-25 16:14:20 -04:00
Joseph Schorr
a572fd33c7 Ensure that we limit the length of repository names
Until now, they'd simply be truncated by the database. Now, we properly check their lengths.

Fixes https://jira.coreos.com/browse/QUAY-963
2018-07-13 13:22:33 +03:00
Joseph Schorr
2b34ae74fe Add test for trying to pull the tags of a repository under a disabled namespace 2018-06-21 14:41:27 -04:00
Joseph Schorr
fdd0db7a7f Add test for trying to pull the catalog under a disabled namespace 2018-06-21 14:41:27 -04:00
Joseph Schorr
892cc82b6a Ensure that verbs cannot be performed on disabled namespaces or by disabled users 2018-06-21 14:41:27 -04:00
Joseph Schorr
2caaf84f31 Add caching support to catalog
We will now cache the results of the catalog for 60s and not hit the database at all if cached
2018-06-20 14:58:01 -04:00
Joseph Schorr
3161b60522 Switch V2 pagination back to using IDs, which should be much faster and easier on the DB
Also adds a test for the tags endpoint
2018-06-18 16:11:26 -04:00
Joseph Schorr
e22b0ce004
Merge pull request #3057 from quay/joseph.schorr/QUAY-893/blob-mount
Implement support for blob mounting via the `mount` parameter on blob uploads
2018-06-06 13:13:39 -04:00
Joseph Schorr
55eb04e622
Merge pull request #3101 from quay/parallel-testing
Enable parallel testing of all the Quay unit tests
2018-06-06 13:13:14 -04:00
Joseph Schorr
f6eaf7ce9d Add blob mounting tests to the new registry test suite 2018-06-06 11:31:28 -04:00
Joseph Schorr
06cb4bddaa Add a test to ensure we don't hit the database when accessing a private catalog with no auth 2018-06-05 19:42:16 -04:00