vbatts/quay
Archived
1
0
Fork 0
Code Issues Pull requests Releases Wiki Activity
9163 commits 2 branches 62 tags 205 MiB
Commit graph

9163 commits

This branch
This branch
All branches
Author SHA1 Message Date
Joseph Schorr
66ddf66fb0
Merge pull request #3372 from quay/small-exception-fixes 
Small exception fixes
 2019-02-18 14:59:46 -05:00
Joseph Schorr
bcbcb1e197
Merge pull request #3373 from quay/remove-head-check 
Remove the HEAD check for a file being read in cloud storage
 2019-02-18 14:59:36 -05:00
Joseph Schorr
f07d7e0a73 Remove the HEAD check for a file being read in cloud storage 
In addition to being unnecessary, it also causes read-after-write issues occasionally on S3
 2019-02-18 14:27:28 -05:00
Joseph Schorr
15ec09cb58
Merge pull request #3365 from quay/joseph.schorr/QUAY-1332/anon-access-header 
Have the V2 registry endpoints raise Unauthorized with the proper header when anonymous access is disabled
 2019-02-18 13:08:25 -05:00
Joseph Schorr
eef4094f56 Catch parsing errors in image API 
Fixes https://sentry.io/organizations/coreos/issues/664440779/
 2019-02-18 13:02:46 -05:00
Joseph Schorr
ca5ec6f10f
Merge pull request #3366 from quay/joseph.schorr/QUAY-1333/creds-dialog-fix 
Remove dex from the CLI username whitelist
 2019-02-18 12:16:01 -05:00
Joseph Schorr
68300b2644 Catch exceptions when trying to read HTTP body in logs 
Fixes https://sentry.io/organizations/coreos/issues/627586398/events/9ea9873f11e6456abc58f5de10951e44/?project=52148
 2019-02-18 12:11:35 -05:00
Joseph Schorr
166fd671c8 Validate the namespace before looking it up 
This will prevent unicode errors when trying to lookup unicode in the database

Fixes https://sentry.io/organizations/coreos/issues/628269769/?project=52148
 2019-02-18 12:09:25 -05:00
Joseph Schorr
2cebf828a2
Merge pull request #3368 from quay/security-scan-set-opt 
Change the security scan update query to only change the indexed row
 2019-02-18 11:59:45 -05:00
Joseph Schorr
410aa151db
Merge pull request #3371 from quay/further-storage-fixes 
Make absolutely sure we don't overwrite an existing storage checksum
 2019-02-16 13:11:31 -05:00
Joseph Schorr
59e4896dbf Make absolutely sure we don't overwrite an existing storage checksum 
Old V1 code would occasionally do so, which was never correct, but we never hit it because we didn't (practically) share storage rows before. Now that we explicitly do, we were occasionally de-checksum-ing storage rows on V1 pushes (which are extremely rare as-is). This change makes sure we don't do that, and makes sure we always set a proper digest and checksum on storage rows.
 2019-02-16 12:40:53 -05:00
Joseph Schorr
127174fcf7
Merge pull request #3370 from quay/delete-image-placements 
Make sure to delete the image placements in the manifest builder cleanup code
 2019-02-16 12:39:19 -05:00
Joseph Schorr
4a22a52aee Make sure to delete the image placements in the manifest builder cleanup code 2019-02-16 00:28:27 -05:00
Joseph Schorr
f2a351183a
Merge pull request #3369 from quay/fix-broken-manifest-builder 
Fix broken manifest builder
 2019-02-15 21:46:02 -05:00
Joseph Schorr
d878232070 Make sure the manifest builder only deletes temporary storage rows it created 
We were accidentally deleting other image storage rows in the repository, which is really, really bad
 2019-02-15 21:18:02 -05:00
Joseph Schorr
3c526c957a Catch exceptions if trying to build an invalid manifest in the manifest backfill code 2019-02-15 21:17:33 -05:00
Joseph Schorr
10ee6be442 Change the security scan update query to only change the indexed row 
This will mean we try to re-index other similar rows, but that's a no-op on the Clair side, and this update query is extremely heavy and easily interleaving locks
 2019-02-15 18:13:42 -05:00
Joseph Schorr
154c17bd3c
Merge pull request #3367 from quay/add-additional-missing-indexes 
Add an index on the logs_archived boolean on a RepositoryBuild
 2019-02-15 16:52:17 -05:00
Joseph Schorr
f4a1646a8b Add an index on the logs_archived boolean on a RepositoryBuild 
Addresses the slow query:

```
SELECT `candidates` . `id` FROM ( SELECT `t1` . `id` FROM `repositorybuild` AS `t1` WHERE ( ( ( `t1` . `phase` IN (...) ) OR ( `t1` . `started` < ? ) ) AND ( `t1` . `logs_archived` = ? ) ) LIMIT ? ) AS `candidates` ORDER BY `Rand` ( ) LIMIT ? OFFSET ?
```

While the cardinality on `logs_archived` will be low, it should also only be in the `false` state for a very small number of records, so it should make this query very, very fast.
 2019-02-15 16:11:25 -05:00
Joseph Schorr
7b733a75cc Remove dex from the CLI username whitelist 
Dex isn't an internal auth service (one that replaces the database), but an *external* one (one that supplements the internal auth). Therefore, it should not be included in this list.

Fixes https://jira.coreos.com/browse/QUAY-1333
 2019-02-15 15:37:15 -05:00
Joseph Schorr
7690d5d495 Have the V2 registry endpoints raise Unauthorized with the proper header when anonymous access is disabled 
Before this change, we'd raise a generic 401, which was breaking containerd and cri-o.

Fixes https://jira.coreos.com/browse/QUAY-1332
 2019-02-15 15:29:57 -05:00
Joseph Schorr
c4f7b28dc6
Merge pull request #3364 from quay/appr-search-fix 
Extend appr search to return private repositories
 2019-02-15 11:24:08 -05:00
Joseph Schorr
f8b19e798f Change appr search to return private repos when logged in 2019-02-14 17:39:40 -05:00
Joseph Schorr
cb677c5711
Merge pull request #3363 from quay/add-missing-indexes 
Add some missing indexes that should make queries significantly faster
 2019-02-14 17:27:24 -05:00
Joseph Schorr
331a7a640b Add some missing indexes that should make queries significantly faster 2019-02-14 17:23:30 -05:00
Sida Chen
ea9707b969
Merge pull request #3361 from KeyboardNerd/fix_es_test 
Fix elasticsearch test
 2019-02-14 15:17:49 -05:00
Joseph Schorr
39db907172
Merge pull request #3362 from quay/shared-layer-fixes 
Shared blob fixes and optimizations
 2019-02-14 14:45:54 -05:00
Joseph Schorr
7688a3e348 Make sure _lookup_repo_storages_by_content_checksum operates over a set 2019-02-14 14:43:23 -05:00
Sida Chen
2e7725cfa4 Fix elasticsearch test 
dictionary object was serialized in random key order, fix that by
comparing the deserialized dict directly.
 2019-02-14 14:19:23 -05:00
Joseph Schorr
69c306e740 Have blob mounting use the blob cache 2019-02-14 13:06:18 -05:00
Joseph Schorr
f75f315037 Optimize lookup of shared global blobs 
Currently, we only have one (the shared empty layer), but this should make the blob lookups for repositories significantly faster, as we won't need to do the massive join.
 2019-02-14 12:46:42 -05:00
Joseph Schorr
7beac643ec Add an additional test for lookup repo blobs by digest 2019-02-14 11:22:06 -05:00
Joseph Schorr
d24127d8ce
Merge pull request #3360 from thomasmckay/1290-nginx-vts-redo 
1290 - fix reverted nginx vts removal
 2019-02-14 11:06:52 -05:00
Joseph Schorr
ec35750f2c Add a lookup test for get_or_create_shared_blob 2019-02-14 10:27:34 -05:00
Tom McKay
a4700d75c1 1290 - fix reverted nginx vts removal 2019-02-14 09:40:34 -05:00
Joseph Schorr
f917de8c70
Merge pull request #3357 from quay/joseph.schorr/QUAY-1327/dont-scan-uploading 
Make sure images in the process of being uploaded are not scanned by Clair
 2019-02-13 16:04:34 -05:00
Sida Chen
d67bbd4917
Merge pull request #3352 from kleesc/elastic_logs_model_implementation 
Implement logs model using Elasticsearch
 2019-02-13 15:51:13 -05:00
Kenny Lee Sin Cheong
035541c6f2 Implement logs model using Elasticsearch 
- Implement logs model using Elasticsearch with tests
- Implement transition model using both elasticsearch and database model
- Add LOGS_MODEL configuration to choose which to use.

Co-authored-by: Sida Chen <sidchen@redhat.com>
Co-authored-by: Kenny Lee Sin Cheong <kenny.lee@redhat.com>
 2019-02-12 16:33:42 -05:00
Sida Chen
5808a84279
Merge pull request #3358 from KeyboardNerd/fix_license 
Fix python license checking and uninstall docutils on build
 2019-02-12 13:32:32 -05:00
Sida Chen
8434b65fcb Fix python license checking and uninstall docutils on build 2019-02-12 12:50:35 -05:00
Joseph Schorr
0b706bc7c4 Make sure images in the process of being uploaded are not scanned by Clair 2019-02-08 16:20:44 -05:00
Joseph Schorr
8d722dee81
Merge pull request #3356 from quay/tag-backfill-skip-improvements 
Tag backfill improvements
 2019-02-07 13:52:32 -05:00
Joseph Schorr
0bfbccdd44 Add a handler for broken tags in the tag backfill system 
This will generate a tag pointing to an empty manifest; the tag will be broken, but as it is *already* broken, at least the backfill can complete
 2019-02-07 13:36:47 -05:00
Joseph Schorr
bbaee0d3f2 Add additional logging for missing parent images 2019-02-06 17:34:18 -05:00
Joseph Schorr
606b08906c Fix typo 2019-02-06 11:41:50 -05:00
Joseph Schorr
097311f360 Handle additional error cases in the manifest schema 1 implementation 2019-02-06 11:41:50 -05:00
Joseph Schorr
82897a2bba Add a log for an integrity error in writing a manifest 2019-02-06 11:41:50 -05:00
Joseph Schorr
f8dd8b2494 Have tag backfill worker continue processing ranges that have errors 2019-02-06 11:41:50 -05:00
thomasmckay
c8bb6e0ec5
Merge pull request #3355 from thomasmckay/1316-exportactionlogsworker 
1316 - exportactionlogsworker period to 60sec
 2019-02-06 08:17:48 -05:00
Tom McKay
56ceef823a 1316 - exportactionlogsworker period to 60sec 2019-02-05 16:06:29 -05:00