vbatts/quay
Archived
1
0
Fork 0
Code Issues Pull requests Releases Wiki Activity
8744 commits 2 branches 62 tags 205 MiB
Commit graph

1543 commits

Author SHA1 Message Date
Joseph Schorr
9f96e595ac Change build component labeling to use new registry interface 2018-08-20 15:49:51 -04:00
Joseph Schorr
301532279c Fix broken registry data interface tests 2018-08-20 13:01:12 -04:00
Joseph Schorr
e30b746aef Fix TagManifests with shared digests under the same repository. 
TagManifests can (apparently, in very rare scenarios) share manifests with the exact same digests, so we need to support that case in the backfill worker. We also need to remove a unique constraint on the manifest column in the mapping table to support this case.
 2018-08-20 11:32:59 -04:00
Joseph Schorr
6414e580f6
Merge pull request #3204 from quay/joseph.schorr/QUAY-1030/registry-interface 
Start on a basic registry_model interface
 2018-08-17 15:31:11 -04:00
Sam Chow
ec14007268
Merge pull request #3207 from quay/project/gen-sec-key 
Add the service key creation to config tool
 2018-08-16 16:48:15 -04:00
Sam Chow
cc9bedbeb9 refactor approval service key to not need approver 2018-08-15 17:18:41 -04:00
Joseph Schorr
2b4d52a357 Migrate the manifest bytes column to UTF8 2018-08-15 11:41:26 -04:00
Joseph Schorr
70df10dade Change manifest backfill to lookup missing content checksums in the repository 
After discussion, we decided the best solution for the missing content checksum problem was to lookup the proper blobs in the repository and, if not present, mark the manifest as broken, as this would reflect the actual issue the user faces if they pull the repository tag today via V2
 2018-08-14 14:36:19 -04:00
Joseph Schorr
c30214c7a8 Start on a basic registry_model interface and change a single module to use it. This will allow us to completely abstract out how we deal with registry-related tables and ensure that transitioning to the new OCI-like model will be easier to do. 2018-08-14 13:48:59 -04:00
Joseph Schorr
96e0fc4ad6 Fix manifest backfill for manifests pointing to V1 images 
V1 images don't have checksums, so we just always use the full storage set for the manifest, rather than a checksum map
 2018-08-13 15:51:18 -04:00
Joseph Schorr
03ea3a3250 Add worker to backfill the new manifest tables from the tagmanifest table 2018-08-13 14:56:31 -04:00
Joseph Schorr
2ab7ca30d0 Change ManifestBlob row creation to be a single batch operation 
Should reduce DB roundtrip times and the overall time we spend under the transaction
 2018-08-09 17:04:16 -04:00
Joseph Schorr
1f04b3ea03
Merge pull request #3197 from quay/remove-transaction 
Remove the transaction around creating the tag and the tag manifest
 2018-08-09 16:34:31 -04:00
Joseph Schorr
8cdddd365b Remove the transaction around creating the tag and the tag manifest 
The tag manifest creation operation is now quite a bit heavier (because it is populating a bunch of new-model tables as well), and it is not necessary for the transaction to include this operation, because tags are valid without manifests currently.
 2018-08-09 16:32:07 -04:00
Joseph Schorr
590f8f65a3 Allow lookup of "dead" manifests so manifest links can be clicked in the tag history 
Fixes https://jira.coreos.com/browse/QUAY-1021
 2018-08-09 16:05:10 -04:00
Joseph Schorr
56222f95dc Change manifest creation to take in the map of blobs that form the manifest 
We need to lookup the blobs *specific to the images in that manifest*, so we now pass them in from the locations in which we know that information
 2018-08-07 16:28:50 -04:00
Joseph Schorr
e3da522d26 Rename store_tag_manifest to indicate clearly it is only to be used for testing 2018-08-07 14:52:59 -04:00
Joseph Schorr
9669320df2 Fix pushing of manifests whose layers share blobs 
If a blob was repeated previously, we would get a constraint error from the ManifestBlob table
 2018-08-07 13:06:30 -04:00
Joseph Schorr
89582438cd Fix the V22 phase 1 migrations to use new tables for mapping rather than editing existing tables 
The ALTER TABLE operations previously used were causing the DB to die when run on the production TagManifest table which has 7 million rows. We instead now use new mapping tables, which is less nice, but these are temporary anyway, so hopefully we only have to deal with their ugliness for a short duration.
 2018-08-06 16:58:27 -04:00
Joseph Schorr
7068010751
Merge pull request #3186 from quay/joseph.schorr/QUAY-1013-1015/manifest-migration 
First phase of V2_2 data model migrations
 2018-08-06 13:56:14 -04:00
Joseph Schorr
a46660a06f Add new Manifest, ManifestLabel, ManifestLegacyImage and ManifestBlob tables and start writing and GCing to/from them 
This change also starts passing in the manifest interface, rather than the raw data, to the model for writing.

Note that this change does *not* backfill the existing rows in to the new tables; that will occur in a followup PR. The new columns in `tagmanifest` and `tagmanifestlabel` will be used to track the backfill, as it will occur in a worker.
 2018-08-06 13:01:11 -04:00
Sida Chen
6efe262217
Merge pull request #3188 from KeyboardNerd/sidchen/notification_datetime_bug 
Fixes empty "created" field in notification
 2018-08-06 12:58:38 -04:00
Sida Chen
415631153a Fixes empty "created" field in notification 
Fixes https://jira.coreos.com/browse/QUAY-1022
 2018-08-01 17:17:47 -04:00
Joseph Schorr
7325b22c90 Change id column in LogEntry to a BigInt and migrate back to using that table 2018-07-30 12:42:56 -04:00
Joseph Schorr
8e425ee559 Fix for merge 2018-07-23 12:43:23 -04:00
Joseph Schorr
9a40e99a8f Enhancements for validation of DB urls 
We now turn off retries and DB pooling, and make sure to always close the connection
 2018-07-23 12:43:23 -04:00
Joseph Schorr
d15dcae505 Reenable retry wrapper on database operations 2018-07-23 12:43:23 -04:00
Brad Ison
d3d9cca182 Upgrade Peewee to latest 3.x 
This requires a number of small changes in the data model code, as well as additional testing.
 2018-07-23 12:43:23 -04:00
Joseph Schorr
487edf0ba1 Phase 3 of Appr migration 
Deletes the old models and their code
 2018-07-21 15:43:00 -04:00
Joseph Schorr
acb7d626a4 Phase 2 of Appr migration 
Backfills the new tables from the old ones, and switches all code to read and write from the new tables
 2018-07-21 15:25:04 -04:00
Joseph Schorr
113bb96f29 Phase 1 of migrating APPR-specific tables to tables with the Appr prefix 
Fixes https://jira.coreos.com/browse/QUAY-950
 2018-07-20 13:19:01 -04:00
Joseph Schorr
6622f27c93 Rename oci_model to appr_model 2018-07-20 12:46:19 -04:00
Joseph Schorr
3f3a0946f6 Move repo visibility tests into pytest 2018-07-19 11:23:11 -04:00
Joseph Schorr
f84cb05792
Merge pull request #3164 from quay/move-queue-tests 
Move queue tests to pytest
 2018-07-18 17:27:38 -04:00
Joseph Schorr
2d7b4dd1bf Move public repo test to pytest 2018-07-18 17:22:05 -04:00
Joseph Schorr
f648b3311b Move queue tests to pytest 2018-07-18 15:22:38 -04:00
Joseph Schorr
928a5d604a Move build query tests to pytest 2018-07-18 11:26:18 -04:00
Sam Chow
f5a8116f5a Remove password confirmation in config app 
Small fix to manually clean up temp dir when creating new temp dir,
small fix to font awesome icons, change the jwt/keystone
validators to not use username/password
 2018-07-17 16:00:55 -04:00
Sam Chow
496d94138c
Merge pull request #3139 from quay/spike/install-certs 
Install certs in the config app, small refactor to LDAP validation
 2018-07-16 12:50:36 -04:00
Joseph Schorr
eb8e325f94 Move image sharing model tests to pytest 2018-07-16 09:58:21 -04:00
Joseph Schorr
dda083abb1 Move blob model tests to pytest 2018-07-13 13:19:46 +03:00
Sam Chow
9024419896 Modify ldap validator to just check user existence 
Remove auth user check from updating config app config

remove duplicate certs install script
 2018-07-11 16:49:13 -04:00
Joseph Schorr
924dda296f Fully migrate API security tests into the pytest test suite 
Also adds an additional test that ensures that at least one security test exists for every (api endpoint, http method) pair.
 2018-07-08 18:33:21 +03:00
Brad Ison
73cb7f3228
endpoints/api: Allow null fields in user metadata 
The user metadata fields are nullable in the database, but were not in
the json sechema.  This prevented users from updating some of their
information on the site if they hadn't set the metadata fields.
 2018-06-27 15:34:55 -04:00
Sam Chow
872be8605a Fix error case in uploading tar, more comments 2018-06-22 13:23:08 -04:00
Sam Chow
aff1a08a83 Fix main app migration pathway 2018-06-21 15:33:26 -04:00
Sam Chow
b5f630ba29 Fix alembic migrations importing app 
Ensure we connect to loaded config db
 2018-06-20 17:17:35 -04:00
Joseph Schorr
371f6f8946
Merge pull request #3118 from quay/joseph.schorr/QUAY-977/catalog-efficiency 
Catalog efficiency improvements
 2018-06-20 16:40:37 -04:00
Joseph Schorr
2caaf84f31 Add caching support to catalog 
We will now cache the results of the catalog for 60s and not hit the database at all if cached
 2018-06-20 14:58:01 -04:00
Joseph Schorr
7604e9842b Change repo filtering for users to use a user ID reference, rather than the username 
While this means we need an additional query for initial lookup, it makes the *filtering* query (which is the heavy part) require far fewer joins, thus making it more efficient.

Also adds a new unit test to verify that our filter filters to the correct set of repositories.
 2018-06-19 10:51:30 -04:00