Commit graph

9158 commits

Author SHA1 Message Date
Joseph Schorr
1eaf5b18dd Adjustments based on code review feedback 2018-11-15 13:51:48 +02:00
Joseph Schorr
c46b11bac1 Implement UI support for manifest lists 2018-11-14 18:40:57 +02:00
Joseph Schorr
276d0d571d Fix support for pulling manifest lists via Docker V1 protocol where applicable 2018-11-14 14:05:06 +02:00
Joseph Schorr
37b20010aa Add support for pushing and pulling schema 2 manifests with remote layers
This is required for windows image support
2018-11-14 13:21:50 +02:00
Joseph Schorr
d97055e2ba Code review fixes 2018-11-14 09:15:58 +02:00
Joseph Schorr
3b4002877a Fix mimetype handling in registry tests 2018-11-14 08:51:01 +02:00
Joseph Schorr
7a794e29c0 Add tests for manifest lists and fix some issues encountered while testing 2018-11-13 21:03:20 +02:00
Joseph Schorr
9994f0ae61 Disable GC for repositories with manifest lists
This is a temporary measure so we can roll V2_2 out for internal customer testing without having to worry about GC interacting oddly with it
2018-11-13 21:03:20 +02:00
Joseph Schorr
e752a9a73f Update registry tests to test schema 2 manifest pushes and pulls
Note that tests for manifest *lists* will be in a following commit
2018-11-13 17:15:00 +02:00
Joseph Schorr
7b9f56eff3 Fixes to ensuring existing code can process schema 2 manifests 2018-11-13 17:13:51 +02:00
Joseph Schorr
9474fb7833 Fix bug with json.dumps changing the order of keys
We write the literal received JSON string into the image data, to ensure the checksums computed will always match
2018-11-13 16:50:13 +02:00
Joseph Schorr
8985d8f106 Have registry tests run against both data models (old and new) 2018-11-13 12:00:55 +02:00
Joseph Schorr
849e613386 Implement support for schema 2 manifests 2018-11-13 11:49:12 +02:00
Joseph Schorr
1b3daac3c3 Add method for retrieving a V1-compatible manifest for any manifest or manifest list
This is used to serve older clients that don't support the V2 schema format
2018-11-13 10:40:57 +02:00
Joseph Schorr
d77d383e46 Add schema2 media types 2018-11-13 09:24:04 +02:00
Joseph Schorr
30f072aeff Add support for creating schema 2 manifests and manifest lists via the OCI model 2018-11-12 23:27:49 +02:00
Joseph Schorr
e344d4a5cf Enhancements for Docker schema implementations in preparing for supporting schema 2 in the OCI model
This adds additional required properties and methods to the Docker schema interface to allow us to treat both schema1 and schema2 manifests and lists logically equivalent from the OCI mode perspective
2018-11-12 23:27:01 +02:00
Joseph Schorr
6b86b87a16
Merge pull request #3290 from quay/joseph.schorr/QUAY-1124/implement-new-data-model
Implement the new OCI-based registry data model
2018-11-08 13:39:29 -05:00
Joseph Schorr
527c44087b
Merge pull request #3292 from vbatts/comments
dockerclients: comment of the BOX tuple
2018-11-08 13:38:05 -05:00
c312ad1a1b dockerclients: comment of the BOX tuple
Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2018-11-08 13:21:23 -05:00
Joseph Schorr
fdcb8bad23 Implement the new OCI-based registry data model
Note that this change does *not* enable the new data model by default, but does allow it to be used when a special environment variable is specified.
2018-11-07 22:07:58 -05:00
Sida Chen
7f72fd2f80
Merge pull request #3291 from KeyboardNerd/bug/utc
Fix wrong utc timestamp in tag expiration view
2018-11-07 16:54:54 -05:00
Sida Chen
d30045a424 Fix wrong utc timestamp in tag expiration view
* changed expiration date response to parse UTC timestamp from the
database. This problem only happens when testing using Sqlite.
2018-11-07 16:51:59 -05:00
Joseph Schorr
3e63b08731
Merge pull request #3288 from quay/faster-gc
Change garbage collection queries to be far smaller by GCing per tag and per image
2018-11-07 14:35:38 -05:00
Joseph Schorr
5124422332 Change garbage collection queries to be far smaller by GCing per tag and per image
While this will require far more iterations and queries, each query itself will be quite small, thus preventing us from locking up the database
2018-11-07 00:50:34 -05:00
Joseph Schorr
924b386437
Merge pull request #3286 from quay/extra-gc-check
Add an extra check in garbage collection to ensure we don't delete manifests referenced by other new-style tags
2018-11-05 12:43:02 -05:00
Joseph Schorr
5b089653d1 Add an extra check in garbage collection to ensure we don't delete manifests referenced by other new-style tags 2018-11-02 15:56:18 -04:00
Joseph Schorr
d0e1f464ff
Merge pull request #3285 from quay/fix-change-tag-expiration
Fix NPE in changing the expiration of a tag
2018-11-02 14:55:00 -04:00
Joseph Schorr
4779e05f55 Fix NPE in changing the expiration of a tag 2018-11-02 14:54:13 -04:00
Joseph Schorr
417b3f9fc2
Merge pull request #3284 from quay/switch-tag-basic-to-ascii
Switch Tag name field back to ASCII
2018-11-02 14:39:45 -04:00
Joseph Schorr
0c83e1272b Switch Tag name field back to ASCII
Because we run MySQL 5.6 in production, making this field a utf-8 went outside the 767 byte max size for indexes on the `tag_repository_id_name_lifetime_end_ms` index.
2018-11-02 14:35:39 -04:00
Joseph Schorr
876ebc9b2f
Merge pull request #3278 from quay/joseph.schorr/QUAY-1124/new-data-model
Begin work on the new registry data model implementation
2018-11-02 14:04:14 -04:00
Joseph Schorr
3cd6a4ae84
Merge pull request #3282 from kleesc/handle-missing-build-phase-case
Build phase message: add missing case
2018-11-02 13:46:38 -04:00
Joseph Schorr
e8ed43b46f Fix name of reversion field in new Tag table and add a test 2018-11-01 17:59:10 -04:00
Joseph Schorr
114e2c3bf2 Have all tag code add, modify and delete both old and new style tags
This preps us for being able to use the new data model with existing repositories
2018-11-01 17:59:10 -04:00
Joseph Schorr
36db133b86 Pull out common shared code in the data model interface 2018-11-01 17:59:10 -04:00
Joseph Schorr
c0653ef2ad Add Tag, TagKind and ManifestChild tables in prep for new data model 2018-11-01 17:59:09 -04:00
Joseph Schorr
053d918d67 Rename pre oci model tests in preparation for the new model (the tests will be applicable to both models) 2018-11-01 17:59:09 -04:00
Joseph Schorr
6c29baf3fc
Merge pull request #3281 from quay/georep-fix
Fixes for georeplication
2018-11-01 17:58:32 -04:00
Joseph Schorr
ebc57da843 Tiny fixes found in review 2018-11-01 17:45:39 -04:00
Joseph Schorr
0fb3d6847e Add a script for verifying the placements of blobs in storage. If a blob is not present in storage, its placement will be removed. 2018-11-01 11:44:49 -04:00
Kenny Lee Sin Cheong
8805d64eca Build phase message: add missing case 2018-11-01 11:22:59 -04:00
Joseph Schorr
859bbec3dc
Merge pull request #3272 from quay/joseph.schorr/QUAY-1171/registry-jwt-tests
Implement a basic test suite for jwtutil and add extra checks to the decode method
2018-10-31 17:03:49 -04:00
Joseph Schorr
6a9634dffb Harden the storage replication worker to failures by explicitly catching certain errors, having better backoff and explicitly ensuring a layer has been replicated to a region before adding the placement into the database
Also adds tests for the various failure cases
2018-10-31 14:13:25 -04:00
Joseph Schorr
088a301754 Add extra exception handling around stream_write in verbs 2018-10-31 12:46:58 -04:00
Joseph Schorr
3a0adfcb11 Fix stream_write to properly raise an exception on failure, instead of just silently failing
This was causing problems for customers using georeplication over unstable storage engines

Also adds tests for stream_write and copy,  to ensure we detect failure
2018-10-31 12:46:49 -04:00
Joseph Schorr
a048ff3633
Merge pull request #3279 from quay/fix-torrent-info-bug
Fix NPE bug in torrentinfo set call
2018-10-31 11:33:11 -04:00
Joseph Schorr
366bddd20a Fix NPE bug in torrentinfo set call
Fixes https://sentry.io/coreos/backend-production/issues/747494304
2018-10-30 16:47:59 -04:00
Joseph Schorr
4868f17832 Implement a basic test suite for jwtutil and add extra checks to the decode method 2018-10-30 16:45:30 -04:00
Kenny Lee Sin Cheong
436e8cb760
Merge pull request #3275 from kleesc/fix-tag-ui-function-name
Fix tag ui function name
2018-10-30 14:07:55 -04:00