Joseph Schorr
442312402f
Merge pull request #3299 from quay/joseph.schorr/QUAY-1233/batch-logs-download
...
Batch download of logs
2018-12-19 15:38:22 -05:00
Joseph Schorr
9190c046c7
Remove unusable sitemap endpoint
...
It calls a data model method which is extremely slow and heavy, and it isn't even used anyway
2018-12-18 16:37:09 -05:00
Joseph Schorr
8a212728a3
Implement a worker for batch exporting of usage logs
...
This will allow customers to request their usage logs for a repository or an entire namespace, and we can export the logs in a manner that doesn't absolutely destroy the database, with every step along the way timed.
2018-12-18 15:33:03 -05:00
Joseph Schorr
6a94eba1a2
Remove all references to LogEntry2 from the code
...
The migration to actually drop the table will go in after this code has been pushed to production
2018-12-11 15:54:16 -05:00
Joseph Schorr
c3710a6a5e
Add ability for specific geographic regions to be blocked from pulling images within a namespace
2018-12-06 17:14:45 -05:00
Joseph Schorr
492934de3c
Remove redundant call
2018-11-27 18:44:34 +02:00
Joseph Schorr
4e1ff90cb2
Unify the get_layers calls across all implements of manifest schemas to ensure we have a common type returned
...
Also renames some methods to make it more clear what kind of information they return
2018-11-26 17:58:48 +02:00
Joseph Schorr
adccdd30ca
Read blobs from new manifest blob table where relevant
2018-11-25 17:31:09 +02:00
Joseph Schorr
4985040d31
Properly handle the empty layer when pushing schema 2 manifests
...
Docker doesn't send us the contents of this layer, so we are forced to synthesize it ourselves
2018-11-25 16:16:59 +02:00
Joseph Schorr
ffd567e2c2
Rename accessed items in the data model
2018-11-21 18:06:16 +02:00
Joseph Schorr
e972e4088b
Fix lookup of manifests referenced solely by a manifest list
...
We need to ensure we can find them if there is an active tag pointing to the parent list
2018-11-19 23:31:41 +02:00
Joseph Schorr
54904cfd6e
Fix bug around pushing manifest lists that refer to the same manifest twice as children
2018-11-19 18:26:22 +02:00
Joseph Schorr
e6c2ddfa93
Add support for direct pushing of schema 2 manifests without tags
...
This is required for manifest lists
2018-11-19 14:01:41 +02:00
Joseph Schorr
82ee21bfbd
Change OCI data model to use a content retriever for the various manifest operations
2018-11-19 12:24:04 +02:00
Joseph Schorr
1eaf5b18dd
Adjustments based on code review feedback
2018-11-15 13:51:48 +02:00
Joseph Schorr
37b20010aa
Add support for pushing and pulling schema 2 manifests with remote layers
...
This is required for windows image support
2018-11-14 13:21:50 +02:00
Joseph Schorr
d97055e2ba
Code review fixes
2018-11-14 09:15:58 +02:00
Joseph Schorr
7a794e29c0
Add tests for manifest lists and fix some issues encountered while testing
2018-11-13 21:03:20 +02:00
Joseph Schorr
9994f0ae61
Disable GC for repositories with manifest lists
...
This is a temporary measure so we can roll V2_2 out for internal customer testing without having to worry about GC interacting oddly with it
2018-11-13 21:03:20 +02:00
Joseph Schorr
849e613386
Implement support for schema 2 manifests
2018-11-13 11:49:12 +02:00
Joseph Schorr
30f072aeff
Add support for creating schema 2 manifests and manifest lists via the OCI model
2018-11-12 23:27:49 +02:00
Joseph Schorr
6b86b87a16
Merge pull request #3290 from quay/joseph.schorr/QUAY-1124/implement-new-data-model
...
Implement the new OCI-based registry data model
2018-11-08 13:39:29 -05:00
Joseph Schorr
fdcb8bad23
Implement the new OCI-based registry data model
...
Note that this change does *not* enable the new data model by default, but does allow it to be used when a special environment variable is specified.
2018-11-07 22:07:58 -05:00
Joseph Schorr
3e63b08731
Merge pull request #3288 from quay/faster-gc
...
Change garbage collection queries to be far smaller by GCing per tag and per image
2018-11-07 14:35:38 -05:00
Joseph Schorr
5124422332
Change garbage collection queries to be far smaller by GCing per tag and per image
...
While this will require far more iterations and queries, each query itself will be quite small, thus preventing us from locking up the database
2018-11-07 00:50:34 -05:00
Joseph Schorr
5b089653d1
Add an extra check in garbage collection to ensure we don't delete manifests referenced by other new-style tags
2018-11-02 15:56:18 -04:00
Joseph Schorr
4779e05f55
Fix NPE in changing the expiration of a tag
2018-11-02 14:54:13 -04:00
Joseph Schorr
e8ed43b46f
Fix name of reversion field in new Tag table and add a test
2018-11-01 17:59:10 -04:00
Joseph Schorr
114e2c3bf2
Have all tag code add, modify and delete both old and new style tags
...
This preps us for being able to use the new data model with existing repositories
2018-11-01 17:59:10 -04:00
Joseph Schorr
366bddd20a
Fix NPE bug in torrentinfo set call
...
Fixes https://sentry.io/coreos/backend-production/issues/747494304
2018-10-30 16:47:59 -04:00
Joseph Schorr
4a7b4ad06a
Fix disabled namespace check
2018-10-18 14:25:40 -04:00
Joseph Schorr
e91ba98e1b
Convert V2's tag endpoints to use the new data model interface
2018-10-18 14:25:40 -04:00
Joseph Schorr
6b5064aba4
Convert V2's manifest endpoints to use the new data model interface
2018-10-18 14:25:40 -04:00
Joseph Schorr
a172de4fdc
Reimplement cache support for blobs in the registry data model
2018-10-18 14:25:40 -04:00
Joseph Schorr
0ae062be62
Add manifest creation to new registry data model interface
2018-10-18 14:25:39 -04:00
Joseph Schorr
65d5be23c7
Implement a manifest builder, to allow for easier management of state around constructing manifests
2018-09-28 15:52:43 -04:00
Kenny Lee Sin Cheong
8e643ce5d9
Repository endpoint tags pagination ( #3238 )
...
* endpoint/api/repository: limit the number of tags returned
- Limit the number of tags returned by /api/v1/repository/<ns:repo> to 500.
- Uses the tag history endpoint instead, with an active tag filte.
- Update UI to use tag history endpoint instead.
2018-09-14 15:30:54 -04:00
Joseph Schorr
a4f1475de4
Merge pull request #3234 from quay/joseph.schorr/QUAY-1030/interfacing-part-8
...
Change verbs endpoint to use the new registry data model interface
2018-09-13 12:50:02 -04:00
Joseph Schorr
f252b0b16f
Move verbs endpoint to use new registry data model
2018-08-29 16:38:54 -04:00
Joseph Schorr
6c5c2f1a75
Add new methods to registry data model interface in prep for moving verbs to using it
2018-08-29 16:38:54 -04:00
Joseph Schorr
ce61ec6668
Fix bug where GC attempts to delete manifests that are shared by multiple tag manifests
...
Also adds a test for this case
All of this will be moot once we get rid of tag manifests, but for now, its causing exceptions
2018-08-29 11:11:52 -04:00
Joseph Schorr
70e233ab5c
Disallow access in the new registry model to hidden tags
...
This was causing problems with certain build cache pulls
2018-08-28 13:34:58 -04:00
Joseph Schorr
581ec47798
Handle the cases where an existing tag manifest or manifest has already been created
2018-08-27 15:02:03 -04:00
Joseph Schorr
a96c5a7f64
Optimize the new registry data model to avoid unnecessary queries
2018-08-24 11:25:24 -04:00
Joseph Schorr
8225c61a1f
Implement tag functions in new registry model interface
2018-08-22 18:03:57 -04:00
Joseph Schorr
9f96e595ac
Change build component labeling to use new registry interface
2018-08-20 15:49:51 -04:00
Joseph Schorr
6414e580f6
Merge pull request #3204 from quay/joseph.schorr/QUAY-1030/registry-interface
...
Start on a basic registry_model interface
2018-08-17 15:31:11 -04:00
Sam Chow
cc9bedbeb9
refactor approval service key to not need approver
2018-08-15 17:18:41 -04:00
Joseph Schorr
70df10dade
Change manifest backfill to lookup missing content checksums in the repository
...
After discussion, we decided the best solution for the missing content checksum problem was to lookup the proper blobs in the repository and, if not present, mark the manifest as broken, as this would reflect the actual issue the user faces if they pull the repository tag today via V2
2018-08-14 14:36:19 -04:00
Joseph Schorr
c30214c7a8
Start on a basic registry_model interface and change a single module to use it. This will allow us to completely abstract out how we deal with registry-related tables and ensure that transitioning to the new OCI-like model will be easier to do.
2018-08-14 13:48:59 -04:00
