8784 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
Sam Chow	efa66d84e4	Change spacing from 4 spaces to 2 spaces	2018-08-16 16:49:10 -04:00
Sam Chow	ec14007268	Merge pull request #3207 from quay/project/gen-sec-key ... Add the service key creation to config tool	2018-08-16 16:48:15 -04:00
Sam Chow	9faba7f5c3	Add backend ability to generate sec keys in config app	2018-08-16 16:45:41 -04:00
Joseph Schorr	e13baa9041	Merge pull request #3206 from quay/fix-unicode-schemas ... Fix unicode manifests in the backfill	2018-08-16 13:24:17 -04:00
Sam Chow	0bc22d810a	Add components for generating sec keys	2018-08-15 17:18:41 -04:00
Sam Chow	cc9bedbeb9	refactor approval service key to not need approver	2018-08-15 17:18:41 -04:00
Jimmy Zelinskie	7edf679670	Merge pull request #3195 from jzelinskie/v2-accept-headers ... V2 accept headers	2018-08-15 16:54:47 -04:00
Sam Chow	3c8252d808	Merge pull request #3200 from quay/project/kube-config ... Kube QE user can create/modify deployment with config tool	2018-08-15 16:34:45 -04:00
Jimmy Zelinskie	9f8b1e2831	gitignore: ignore all of the pytest cache	2018-08-15 14:49:26 -04:00
Jimmy Zelinskie	6d4300a92a	test/registry: add accept header test	2018-08-15 14:49:26 -04:00
Jimmy Zelinskie	b7573a8c88	endpoints/v2: fail clients not accepting schema v1	2018-08-15 14:49:26 -04:00
Sam Chow	eea5fe3391	Add cycling deployments and updating config ... Add kube config with refactor to kube accessor Add tests for k8s accessor, some styling changes	2018-08-15 14:17:12 -04:00
Jimmy Zelinskie	568ca1d7ba	delete trailing whitespace	2018-08-15 13:21:26 -04:00
Joseph Schorr	2b4d52a357	Migrate the manifest bytes column to UTF8	2018-08-15 11:41:26 -04:00
Joseph Schorr	fa58f3b1d2	Fix handling of manifests with unicode in the backfill ... Also adds a bunch of tests around manifests to ensure we get the same information in and out	2018-08-15 11:41:15 -04:00
Joseph Schorr	a66a4d6a10	Merge pull request #3205 from quay/backfill-storages-from-manifest ... Change manifest backfill to lookup missing content checksums in the repository	2018-08-14 16:42:25 -04:00
Joseph Schorr	70df10dade	Change manifest backfill to lookup missing content checksums in the repository ... After discussion, we decided the best solution for the missing content checksum problem was to lookup the proper blobs in the repository and, if not present, mark the manifest as broken, as this would reflect the actual issue the user faces if they pull the repository tag today via V2	2018-08-14 14:36:19 -04:00
Joseph Schorr	c30214c7a8	Start on a basic registry_model interface and change a single module to use it. This will allow us to completely abstract out how we deal with registry-related tables and ensure that transitioning to the new OCI-like model will be easier to do.	2018-08-14 13:48:59 -04:00
Joseph Schorr	c0d25d3149	Merge pull request #3202 from quay/fix-backfill-manifest-for-non-checksum-images ... Fix manifest backfill for manifests pointing to V1 images	2018-08-13 18:19:02 -04:00
Joseph Schorr	96e0fc4ad6	Fix manifest backfill for manifests pointing to V1 images ... V1 images don't have checksums, so we just always use the full storage set for the manifest, rather than a checksum map	2018-08-13 15:51:18 -04:00
Joseph Schorr	73e585df49	Merge pull request #3194 from quay/joseph.schorr/QUAY-1013-1015/backfill-worker ... Add worker to backfill the new manifest tables from the tagmanifest table	2018-08-13 14:59:03 -04:00
Joseph Schorr	f2d50b3f8e	Add run commands for backfill worker	2018-08-13 14:56:32 -04:00
Joseph Schorr	03ea3a3250	Add worker to backfill the new manifest tables from the tagmanifest table	2018-08-13 14:56:31 -04:00
Joseph Schorr	805e928dff	Merge pull request #3199 from quay/fix-registry-tests ... Fix registry tests to not terminate prematurely when looking for errors	2018-08-10 18:06:22 -04:00
Sam Chow	d387ba171f	Adds ability to upload config dir to k8s in config	2018-08-10 17:43:30 -04:00
Sam Chow	3d4e43c8d1	Add fetching of qe deployments in config tool	2018-08-10 17:42:41 -04:00
Joseph Schorr	b39f53ee38	Merge pull request #3198 from quay/batch-blob-inserts ... Change ManifestBlob row creation to be a single batch operation	2018-08-10 15:11:26 -04:00
Joseph Schorr	701eac5466	Fix registry tests to not terminate prematurely when looking for errors ... Also reorders a check in the V1 protocol to have better error messaging for the users	2018-08-10 13:36:20 -04:00
Joseph Schorr	2ab7ca30d0	Change ManifestBlob row creation to be a single batch operation ... Should reduce DB roundtrip times and the overall time we spend under the transaction	2018-08-09 17:04:16 -04:00
Joseph Schorr	1f04b3ea03	Merge pull request #3197 from quay/remove-transaction ... Remove the transaction around creating the tag and the tag manifest	2018-08-09 16:34:31 -04:00
Joseph Schorr	ad03404d31	Merge pull request #3196 from quay/joseph.schorr/QUAY-1021/manifest-links ... Allow lookup of "dead" manifests so manifest links can be clicked in the tag history	2018-08-09 16:32:25 -04:00
Joseph Schorr	8cdddd365b	Remove the transaction around creating the tag and the tag manifest ... The tag manifest creation operation is now quite a bit heavier (because it is populating a bunch of new-model tables as well), and it is not necessary for the transaction to include this operation, because tags are valid without manifests currently.	2018-08-09 16:32:07 -04:00
Joseph Schorr	590f8f65a3	Allow lookup of "dead" manifests so manifest links can be clicked in the tag history ... Fixes https://jira.coreos.com/browse/QUAY-1021	2018-08-09 16:05:10 -04:00
Joseph Schorr	f6902f8096	Merge pull request #3193 from quay/ipresolver-perf ... Make IP resolver performance a bit better by skipping an unnecessary set	2018-08-08 17:28:47 -04:00
Sida Chen	549b286650	Merge pull request #3190 from KeyboardNerd/sidchen/boto_change ... Changes boto2 pip source to point to the fixed	2018-08-08 14:51:37 -04:00
Sida Chen	bccff26e69	Changes boto2 pip source ... Changes requirements file to point boto2 to devtable repo, which contains the fix for SigV4 connection problem.	2018-08-08 14:49:57 -04:00
Joseph Schorr	b7d8bb227e	Make IP resolver performance a bit better by skipping an unnecessary set ... This will still hang for a second or two on first call	2018-08-08 14:40:26 -04:00
Joseph Schorr	c7baf6cf58	Merge pull request #3192 from quay/fix-blobs-stored-for-manifest ... Fix blobs stored for manifest	2018-08-08 12:53:16 -04:00
Joseph Schorr	56222f95dc	Change manifest creation to take in the map of blobs that form the manifest ... We need to lookup the blobs *specific to the images in that manifest*, so we now pass them in from the locations in which we know that information	2018-08-07 16:28:50 -04:00
Joseph Schorr	e3da522d26	Rename store_tag_manifest to indicate clearly it is only to be used for testing	2018-08-07 14:52:59 -04:00
Joseph Schorr	e33ccff8cb	Fix query count in test	2018-08-07 14:52:33 -04:00
Joseph Schorr	fc57f09ac7	Merge pull request #3191 from quay/fix-shared-blobs-under-manifest ... Fix pushing of manifests whose layers share blobs	2018-08-07 13:27:41 -04:00
Joseph Schorr	9669320df2	Fix pushing of manifests whose layers share blobs ... If a blob was repeated previously, we would get a constraint error from the ManifestBlob table	2018-08-07 13:06:30 -04:00
Joseph Schorr	92743ca57b	Merge pull request #3189 from quay/fix-v22-phase1-migration ... Fix the V22 phase 1 migrations to use new tables for mapping rather than editing existing tables	2018-08-07 11:38:00 -04:00
Joseph Schorr	89582438cd	Fix the V22 phase 1 migrations to use new tables for mapping rather than editing existing tables ... The ALTER TABLE operations previously used were causing the DB to die when run on the production TagManifest table which has 7 million rows. We instead now use new mapping tables, which is less nice, but these are temporary anyway, so hopefully we only have to deal with their ugliness for a short duration.	2018-08-06 16:58:27 -04:00
Joseph Schorr	7068010751	Merge pull request #3186 from quay/joseph.schorr/QUAY-1013-1015/manifest-migration ... First phase of V2_2 data model migrations	2018-08-06 13:56:14 -04:00
Joseph Schorr	a46660a06f	Add new Manifest, ManifestLabel, ManifestLegacyImage and ManifestBlob tables and start writing and GCing to/from them ... This change also starts passing in the manifest interface, rather than the raw data, to the model for writing. Note that this change does *not* backfill the existing rows in to the new tables; that will occur in a followup PR. The new columns in `tagmanifest` and `tagmanifestlabel` will be used to track the backfill, as it will occur in a worker.	2018-08-06 13:01:11 -04:00
Joseph Schorr	36c7482385	Define a formal manifest interface and implement in the schema1 and schema2 manifests ... This will allow us to pass arbitrary manifests to the model	2018-08-06 13:01:11 -04:00
Joseph Schorr	cf5a6e1adc	Remove old audit manifests tool	2018-08-06 13:01:11 -04:00
Sida Chen	6efe262217	Merge pull request #3188 from KeyboardNerd/sidchen/notification_datetime_bug ... Fixes empty "created" field in notification	2018-08-06 12:58:38 -04:00