Commit graph

36 commits

Author SHA1 Message Date
Joseph Schorr
f0f2d9cdf4 Add architecture validation to manifest lists that contain schema 1 manifests
Fixes https://jira.coreos.com/browse/QUAY-1266
2019-02-22 13:39:54 -05:00
Joseph Schorr
bacf074219 Fix backfills of super large manifests by stripping metadata from all but the final layer
This is semantically valid because Docker only uses the leaf layer as the image config when reading a V2_1 manifest

Fixes https://jira.coreos.com/browse/QUAY-1351
2019-02-20 12:08:20 -05:00
Joseph Schorr
097311f360 Handle additional error cases in the manifest schema 1 implementation 2019-02-06 11:41:50 -05:00
Joseph Schorr
c1ac12ff8d Some small cleanup to the tag backfill worker 2019-01-21 13:37:11 -05:00
Joseph Schorr
570d974067 Add support for retargeting a tag to all schema 1 manifests
Schema version 1 manifests contain the tag name, and we have a check to ensure we don't point a tag at a manifest with the wrong name embedded. However, this also means that we cannot retarget to that manifest, which will break the UI once we get rid of legacy images.

This change means we can retarget to those manifests, and the OCI model does the work of rewriting the manifest when necessary.
2019-01-15 16:00:06 -05:00
Joseph Schorr
171c7e5238 Further fixes for unicode handling in manifests
We were occasionally trying to compute schema 2 version 1 signatures on the *unicode* representation, which was failing the signature check. This PR adds a new wrapper type called `Bytes`, which all manifests must take in, and which handles the unicodes vs encoded utf-8 stuff in a central location. This PR also adds a test for the manifest that was breaking in production.
2019-01-09 15:14:41 -05:00
Joseph Schorr
feee49be9e Ensure we encode the config in manifest schema 2 via the canonical JSON format 2018-12-18 14:52:19 -05:00
Joseph Schorr
48e584905a Make sure to decode manifests into utf-8 when necessary
This fixes a decoding error
2018-12-17 16:31:24 -05:00
Joseph Schorr
71b7a2b3a2 Make command optional in schema 2 manifests (as per OCI spec) and pull out additional information
Also updates the manifest view page to show the comment or digest if there is no command defined
2018-12-11 17:23:39 -05:00
Joseph Schorr
3c2e050593 Support pulling of schema2 manifests directly via a manifest list tag
This change ensures that if a manifest list is requested with an accepts header for a *schema 2* manifest, the legacy manifest (if any) is returned as schema 2 if it was pushed as a schema 2 manifest (rather than being auto-converted to schema 1)
2018-12-07 11:57:13 -05:00
Joseph Schorr
63f9043312 Code review small fixes 2018-11-28 12:56:16 +02:00
Joseph Schorr
4e1ff90cb2 Unify the get_layers calls across all implements of manifest schemas to ensure we have a common type returned
Also renames some methods to make it more clear what kind of information they return
2018-11-26 17:58:48 +02:00
Joseph Schorr
4985040d31 Properly handle the empty layer when pushing schema 2 manifests
Docker doesn't send us the contents of this layer, so we are forced to synthesize it ourselves
2018-11-25 16:16:59 +02:00
Joseph Schorr
c233760007 Fix conversion of schema 2 manifests to schema 1 manifests
Also adds a number of conversion tests and clarify the interfaces a bit more
2018-11-21 18:06:16 +02:00
Joseph Schorr
cbfb6054e5 Switch content retrieval in manifests to be behind an interface
This allows for easy separation of retrieval of config blobs vs manifests
2018-11-19 11:55:52 +02:00
Joseph Schorr
1eaf5b18dd Adjustments based on code review feedback 2018-11-15 13:51:48 +02:00
Joseph Schorr
c46b11bac1 Implement UI support for manifest lists 2018-11-14 18:40:57 +02:00
Joseph Schorr
37b20010aa Add support for pushing and pulling schema 2 manifests with remote layers
This is required for windows image support
2018-11-14 13:21:50 +02:00
Joseph Schorr
d97055e2ba Code review fixes 2018-11-14 09:15:58 +02:00
Joseph Schorr
1b3daac3c3 Add method for retrieving a V1-compatible manifest for any manifest or manifest list
This is used to serve older clients that don't support the V2 schema format
2018-11-13 10:40:57 +02:00
Joseph Schorr
e344d4a5cf Enhancements for Docker schema implementations in preparing for supporting schema 2 in the OCI model
This adds additional required properties and methods to the Docker schema interface to allow us to treat both schema1 and schema2 manifests and lists logically equivalent from the OCI mode perspective
2018-11-12 23:27:01 +02:00
Joseph Schorr
0ae062be62 Add manifest creation to new registry data model interface 2018-10-18 14:25:39 -04:00
Joseph Schorr
c92294c218 Add compressed_size to both schema formats on their layers 2018-08-29 16:38:54 -04:00
Joseph Schorr
7fa3506723 Add created_datetime to schema1 format 2018-08-27 18:03:58 -04:00
Joseph Schorr
fa58f3b1d2 Fix handling of manifests with unicode in the backfill
Also adds a bunch of tests around manifests to ensure we get the same information in and out
2018-08-15 11:41:15 -04:00
Joseph Schorr
36c7482385 Define a formal manifest interface and implement in the schema1 and schema2 manifests
This will allow us to pass arbitrary manifests to the model
2018-08-06 13:01:11 -04:00
Joseph Schorr
a73cd9170a Add schema2 list support 2018-06-15 16:00:39 -04:00
Joseph Schorr
4bd70eab3c Add basic tests for schema1 2018-06-15 16:00:39 -04:00
Joseph Schorr
9e16a989f5 Audit the number of SQL queries we make in writing manifests, and significantly reduce in the common case
Instead of 41 queries now for the simple manifest, we are down to 14.

The biggest changes:
  - Only synthesize the V1 image rows if we haven't already found them in the database
  - Thread the repository object through to the other model method calls, and use it instead of loading again and again
2018-01-25 11:10:43 -05:00
Joseph Schorr
f9bd7ef42b Add validation of Docker V2_1 schemas and add a test for PUTing an invalid schema 2017-12-20 11:43:03 -05:00
Joseph Schorr
f4e1748bb7 Fix parent rewrite bug in schema1 manifest code and add a bunch more tests
Before this change, if you ended up writing a middle layer whose parent is not in the database, the manifest would fail to rewrite. We now just lookup the parent image in the manifest given to us, ignoring whether it is in the database or not (as it doesn't actually matter if not present; it'll be created if necessary).
2016-10-04 09:15:27 -04:00
Jimmy Zelinskie
ca883e5662 port label support to refactored v2 registry 2016-09-26 14:49:58 -04:00
Joseph Schorr
3c8b87e086 Fix verbs in manifestlist
All registry_tests now pass
2016-09-26 14:49:58 -04:00
Joseph Schorr
db60df827d Implement V2 interfaces and remaining V1 interfaces
Also adds some tests to registry tests for V1 stuff.
Note: All *registry* tests currently pass, but as verbs are not yet converted, the verb tests in registry_tests.py currently fail.
2016-09-26 14:49:04 -04:00
Jimmy Zelinskie
56951397a9 content type lists -> sets 2016-09-26 14:48:05 -04:00
Jimmy Zelinskie
32a6c22b43 mv data/types image
This change also merges formats into the new image module.
2016-09-26 14:48:05 -04:00
Renamed from data/types.py (Browse further)