Commit graph

8753 commits

Author SHA1 Message Date
Joseph Schorr
6c494f4917
Merge pull request #3213 from quay/joseph.schorr/QUAY-1030/continue-interfacing
Continue work on changing registry data access to be behind an interface
2018-08-21 15:32:20 -04:00
Joseph Schorr
5a997eb4e3 Change label creation function to return the label created 2018-08-21 14:27:10 -04:00
Sam Chow
b56899aa96
Merge pull request #3214 from quay/project/mod-cluster
Add ability to populate q.e. config from kube cluster
2018-08-21 10:24:34 -04:00
Sam Chow
df25ec0061 Add ability to populate config from kube cluster 2018-08-20 16:47:42 -04:00
Joseph Schorr
23ff49f0c1 Adjust usage of image model in manifest model 2018-08-20 16:47:38 -04:00
Joseph Schorr
584a18c0e1 Change datatype base to raise AttributeError on missing keys 2018-08-20 15:49:51 -04:00
Joseph Schorr
7b95082a99 Change image API endpoint to use new registry model 2018-08-20 15:49:51 -04:00
Joseph Schorr
254f06e634 Implement legacy image portion of the data model
This also makes use of the newly created input system
2018-08-20 15:49:51 -04:00
Joseph Schorr
8aafbf8b8c Switch the registry data model types to our own class constructor
This allows us to hide the DB ID from external-to-the-package users of the types and will allow us to add model-level caching as well
2018-08-20 15:49:51 -04:00
Joseph Schorr
9f96e595ac Change build component labeling to use new registry interface 2018-08-20 15:49:51 -04:00
Joseph Schorr
b096e793fd
Merge pull request #3211 from quay/backfill-shared-manifests
Fix TagManifests with shared digests under the same repository.
2018-08-20 15:31:24 -04:00
Joseph Schorr
301532279c Fix broken registry data interface tests 2018-08-20 13:01:12 -04:00
Joseph Schorr
ed897626a2 Handle data model exceptions when loading parent images in manifest backfill 2018-08-20 11:32:59 -04:00
Joseph Schorr
e30b746aef Fix TagManifests with shared digests under the same repository.
TagManifests can (apparently, in very rare scenarios) share manifests with the exact same digests, so we need to support that case in the backfill worker. We also need to remove a unique constraint on the manifest column in the mapping table to support this case.
2018-08-20 11:32:59 -04:00
Sam Chow
8a83bf7c81
Merge pull request #3210 from quay/project/kube-tls
Add ability to add extra certs for config tool
2018-08-20 10:07:10 -04:00
Sam Chow
ff294d6c52 Add init script to download extra ca certs 2018-08-17 15:42:42 -04:00
Joseph Schorr
6414e580f6
Merge pull request #3204 from quay/joseph.schorr/QUAY-1030/registry-interface
Start on a basic registry_model interface
2018-08-17 15:31:11 -04:00
Sam Chow
5b400f4c22 Add one-action transplant of kube secret 2018-08-17 11:00:04 -04:00
Sam Chow
8eb7d73f22
Merge pull request #3208 from quay/project/2-spaces
Fix formatting for the config app
2018-08-17 10:30:22 -04:00
Sam Chow
efa66d84e4 Change spacing from 4 spaces to 2 spaces 2018-08-16 16:49:10 -04:00
Sam Chow
ec14007268
Merge pull request #3207 from quay/project/gen-sec-key
Add the service key creation to config tool
2018-08-16 16:48:15 -04:00
Sam Chow
9faba7f5c3 Add backend ability to generate sec keys in config app 2018-08-16 16:45:41 -04:00
Joseph Schorr
e13baa9041
Merge pull request #3206 from quay/fix-unicode-schemas
Fix unicode manifests in the backfill
2018-08-16 13:24:17 -04:00
Sam Chow
0bc22d810a Add components for generating sec keys 2018-08-15 17:18:41 -04:00
Sam Chow
cc9bedbeb9 refactor approval service key to not need approver 2018-08-15 17:18:41 -04:00
Jimmy Zelinskie
7edf679670
Merge pull request #3195 from jzelinskie/v2-accept-headers
V2 accept headers
2018-08-15 16:54:47 -04:00
Sam Chow
3c8252d808
Merge pull request #3200 from quay/project/kube-config
Kube QE user can create/modify deployment with config tool
2018-08-15 16:34:45 -04:00
Jimmy Zelinskie
9f8b1e2831 gitignore: ignore all of the pytest cache 2018-08-15 14:49:26 -04:00
Jimmy Zelinskie
6d4300a92a test/registry: add accept header test 2018-08-15 14:49:26 -04:00
Jimmy Zelinskie
b7573a8c88 endpoints/v2: fail clients not accepting schema v1 2018-08-15 14:49:26 -04:00
Sam Chow
eea5fe3391 Add cycling deployments and updating config
Add kube config with refactor to kube accessor

Add tests for k8s accessor, some styling changes
2018-08-15 14:17:12 -04:00
Jimmy Zelinskie
568ca1d7ba delete trailing whitespace 2018-08-15 13:21:26 -04:00
Joseph Schorr
2b4d52a357 Migrate the manifest bytes column to UTF8 2018-08-15 11:41:26 -04:00
Joseph Schorr
fa58f3b1d2 Fix handling of manifests with unicode in the backfill
Also adds a bunch of tests around manifests to ensure we get the same information in and out
2018-08-15 11:41:15 -04:00
Joseph Schorr
a66a4d6a10
Merge pull request #3205 from quay/backfill-storages-from-manifest
Change manifest backfill to lookup missing content checksums in the repository
2018-08-14 16:42:25 -04:00
Joseph Schorr
70df10dade Change manifest backfill to lookup missing content checksums in the repository
After discussion, we decided the best solution for the missing content checksum problem was to lookup the proper blobs in the repository and, if not present, mark the manifest as broken, as this would reflect the actual issue the user faces if they pull the repository tag today via V2
2018-08-14 14:36:19 -04:00
Joseph Schorr
c30214c7a8 Start on a basic registry_model interface and change a single module to use it. This will allow us to completely abstract out how we deal with registry-related tables and ensure that transitioning to the new OCI-like model will be easier to do. 2018-08-14 13:48:59 -04:00
Joseph Schorr
c0d25d3149
Merge pull request #3202 from quay/fix-backfill-manifest-for-non-checksum-images
Fix manifest backfill for manifests pointing to V1 images
2018-08-13 18:19:02 -04:00
Joseph Schorr
96e0fc4ad6 Fix manifest backfill for manifests pointing to V1 images
V1 images don't have checksums, so we just always use the full storage set for the manifest, rather than a checksum map
2018-08-13 15:51:18 -04:00
Joseph Schorr
73e585df49
Merge pull request #3194 from quay/joseph.schorr/QUAY-1013-1015/backfill-worker
Add worker to backfill the new manifest tables from the tagmanifest table
2018-08-13 14:59:03 -04:00
Joseph Schorr
f2d50b3f8e Add run commands for backfill worker 2018-08-13 14:56:32 -04:00
Joseph Schorr
03ea3a3250 Add worker to backfill the new manifest tables from the tagmanifest table 2018-08-13 14:56:31 -04:00
Joseph Schorr
805e928dff
Merge pull request #3199 from quay/fix-registry-tests
Fix registry tests to not terminate prematurely when looking for errors
2018-08-10 18:06:22 -04:00
Sam Chow
d387ba171f Adds ability to upload config dir to k8s in config 2018-08-10 17:43:30 -04:00
Sam Chow
3d4e43c8d1 Add fetching of qe deployments in config tool 2018-08-10 17:42:41 -04:00
Joseph Schorr
b39f53ee38
Merge pull request #3198 from quay/batch-blob-inserts
Change ManifestBlob row creation to be a single batch operation
2018-08-10 15:11:26 -04:00
Joseph Schorr
701eac5466 Fix registry tests to not terminate prematurely when looking for errors
Also reorders a check in the V1 protocol to have better error messaging for the users
2018-08-10 13:36:20 -04:00
Joseph Schorr
2ab7ca30d0 Change ManifestBlob row creation to be a single batch operation
Should reduce DB roundtrip times and the overall time we spend under the transaction
2018-08-09 17:04:16 -04:00
Joseph Schorr
1f04b3ea03
Merge pull request #3197 from quay/remove-transaction
Remove the transaction around creating the tag and the tag manifest
2018-08-09 16:34:31 -04:00
Joseph Schorr
ad03404d31
Merge pull request #3196 from quay/joseph.schorr/QUAY-1021/manifest-links
Allow lookup of "dead" manifests so manifest links can be clicked in the tag history
2018-08-09 16:32:25 -04:00