Joseph Schorr
77d7600b64
Add ability for proportional rollout of the OCI data model
2019-02-18 16:34:43 -05:00
Joseph Schorr
4a22a52aee
Make sure to delete the image placements in the manifest builder cleanup code
2019-02-16 00:28:27 -05:00
Joseph Schorr
bbfb0211dc
Fix bug where we weren't properly caching enum values and we were always looking up the media type when constructing manifests
2019-01-22 14:46:38 -05:00
Joseph Schorr
0035a1ec6d
Fix a bug in manifest backfill that made it only work on active tags
2019-01-21 16:59:37 -05:00
Joseph Schorr
7c241a93f6
Allow namespaces to be whitelisted for OCI, but not V22
2019-01-21 14:40:26 -05:00
Joseph Schorr
570d974067
Add support for retargeting a tag to all schema 1 manifests
...
Schema version 1 manifests contain the tag name, and we have a check to ensure we don't point a tag at a manifest with the wrong name embedded. However, this also means that we cannot retarget to that manifest, which will break the UI once we get rid of legacy images.
This change means we can retarget to those manifests, and the OCI model does the work of rewriting the manifest when necessary.
2019-01-15 16:00:06 -05:00
Joseph Schorr
eb9ca8e8a8
Fix handling of four byte utf8 manifests
...
- Adds the charset: utf-8 to all the manifest responses
- Makes sure we connect to MySQL in utf8mb4 mode, to ensure we can properly read and write 4-byte utf8 strings
- Adds tests for all of the above
2019-01-10 16:34:56 -05:00
Joseph Schorr
171c7e5238
Further fixes for unicode handling in manifests
...
We were occasionally trying to compute schema 2 version 1 signatures on the *unicode* representation, which was failing the signature check. This PR adds a new wrapper type called `Bytes`, which all manifests must take in, and which handles the unicodes vs encoded utf-8 stuff in a central location. This PR also adds a test for the manifest that was breaking in production.
2019-01-09 15:14:41 -05:00
Joseph Schorr
3c2e050593
Support pulling of schema2 manifests directly via a manifest list tag
...
This change ensures that if a manifest list is requested with an accepts header for a *schema 2* manifest, the legacy manifest (if any) is returned as schema 2 if it was pushed as a schema 2 manifest (rather than being auto-converted to schema 1)
2018-12-07 11:57:13 -05:00
Joseph Schorr
d59bea3569
Enable a configurable whitelist of namespaces for V22
...
If a namespace is present in the whitelist, all calls are sent to the OCI model instead of the Pre OCI model
Note that this does increase overhead for registry calls (since we need to lookup the namespace for every single call), but it should only be temporary until we've migrated all users over to the OCI data model
2018-12-03 17:37:28 -05:00
Joseph Schorr
4e1ff90cb2
Unify the get_layers calls across all implements of manifest schemas to ensure we have a common type returned
...
Also renames some methods to make it more clear what kind of information they return
2018-11-26 17:58:48 +02:00
Joseph Schorr
180d8847db
Fix image replication for images with remote layers
2018-11-26 16:15:48 +02:00
Joseph Schorr
adccdd30ca
Read blobs from new manifest blob table where relevant
2018-11-25 17:31:09 +02:00
Joseph Schorr
1f03fdb27e
Fix pulling of squashed versions of the legacy image in a manifest lists
2018-11-20 16:37:15 +02:00
Joseph Schorr
8a3427e55a
Change registry data model to use a content retriever where applicable
2018-11-19 14:01:41 +02:00
Joseph Schorr
276d0d571d
Fix support for pulling manifest lists via Docker V1 protocol where applicable
2018-11-14 14:05:06 +02:00
Joseph Schorr
37b20010aa
Add support for pushing and pulling schema 2 manifests with remote layers
...
This is required for windows image support
2018-11-14 13:21:50 +02:00
Joseph Schorr
7b9f56eff3
Fixes to ensuring existing code can process schema 2 manifests
2018-11-13 17:13:51 +02:00
Joseph Schorr
30f072aeff
Add support for creating schema 2 manifests and manifest lists via the OCI model
2018-11-12 23:27:49 +02:00
Joseph Schorr
fdcb8bad23
Implement the new OCI-based registry data model
...
Note that this change does *not* enable the new data model by default, but does allow it to be used when a special environment variable is specified.
2018-11-07 22:07:58 -05:00
Joseph Schorr
e8ed43b46f
Fix name of reversion field in new Tag table and add a test
2018-11-01 17:59:10 -04:00
Joseph Schorr
114e2c3bf2
Have all tag code add, modify and delete both old and new style tags
...
This preps us for being able to use the new data model with existing repositories
2018-11-01 17:59:10 -04:00
Joseph Schorr
053d918d67
Rename pre oci model tests in preparation for the new model (the tests will be applicable to both models)
2018-11-01 17:59:09 -04:00
Joseph Schorr
366bddd20a
Fix NPE bug in torrentinfo set call
...
Fixes https://sentry.io/coreos/backend-production/issues/747494304
2018-10-30 16:47:59 -04:00
Joseph Schorr
6b5064aba4
Convert V2's manifest endpoints to use the new data model interface
2018-10-18 14:25:40 -04:00
Joseph Schorr
a172de4fdc
Reimplement cache support for blobs in the registry data model
2018-10-18 14:25:40 -04:00
Joseph Schorr
0ae062be62
Add manifest creation to new registry data model interface
2018-10-18 14:25:39 -04:00
Joseph Schorr
818ed32f87
Add function in data interface for mounting blobs into other repositories
2018-10-18 14:25:39 -04:00
Joseph Schorr
03789b2210
Add interface for batch creation of labels on a manifest
...
This cannot be a true batch operation right now because of the current mapping table entries needed, but we can create and use the interface now and change the underlying implementation later
2018-10-18 14:25:39 -04:00
Joseph Schorr
8cfb3f4fe8
Add interface function for deleting tags pointing to a manifest
2018-10-18 14:25:39 -04:00
Joseph Schorr
4520f9e842
Change V1 to use the manifest builder and new registry data model
2018-09-28 17:30:04 -04:00
Joseph Schorr
65d5be23c7
Implement a manifest builder, to allow for easier management of state around constructing manifests
2018-09-28 15:52:43 -04:00
Joseph Schorr
ba39737c3a
Add blob uploader helper class, for easy management of all the state associated with uploading blobs
2018-09-24 12:54:56 -04:00
Joseph Schorr
5d2fe535b4
Add ability to commit blob uploads into blobs
2018-09-14 17:15:25 -04:00
Joseph Schorr
f68fbb8028
Add blob upload to new registry data model
2018-09-14 17:15:25 -04:00
Joseph Schorr
1bbe41bb36
Add blob support to get_legacy_image
2018-09-14 17:15:24 -04:00
Joseph Schorr
f252b0b16f
Move verbs endpoint to use new registry data model
2018-08-29 16:38:54 -04:00
Joseph Schorr
6c5c2f1a75
Add new methods to registry data model interface in prep for moving verbs to using it
2018-08-29 16:38:54 -04:00
Joseph Schorr
f297249100
Move manifest backfill for V1 tags into the new registry model interface
2018-08-27 15:01:27 -04:00
Joseph Schorr
a96c5a7f64
Optimize the new registry data model to avoid unnecessary queries
2018-08-24 11:25:24 -04:00
Joseph Schorr
46edebe6b0
Change secscan API endpoints to use new registry model interface
2018-08-23 16:36:04 -04:00
Joseph Schorr
8225c61a1f
Implement tag functions in new registry model interface
2018-08-22 18:03:57 -04:00
Joseph Schorr
a0a6a3d67d
Change manifest API endpoints to use new registry data interface
2018-08-21 17:26:32 -04:00
Joseph Schorr
584a18c0e1
Change datatype base to raise AttributeError on missing keys
2018-08-20 15:49:51 -04:00
Joseph Schorr
254f06e634
Implement legacy image portion of the data model
...
This also makes use of the newly created input system
2018-08-20 15:49:51 -04:00
Joseph Schorr
9f96e595ac
Change build component labeling to use new registry interface
2018-08-20 15:49:51 -04:00
Joseph Schorr
301532279c
Fix broken registry data interface tests
2018-08-20 13:01:12 -04:00
Joseph Schorr
c30214c7a8
Start on a basic registry_model interface and change a single module to use it. This will allow us to completely abstract out how we deal with registry-related tables and ensure that transitioning to the new OCI-like model will be easier to do.
2018-08-14 13:48:59 -04:00
