vbatts/quay
Archived
1
0
Fork 0
Code Issues Pull requests Releases Wiki Activity
6953 commits 2 branches 62 tags 205 MiB
Commit graph

60 commits

Author SHA1 Message Date
Joseph Schorr
eff1827d9d Batch QSS notifications after initial scan 2017-03-01 15:42:49 -05:00
Jimmy Zelinskie
cbb2fff0e2 util.secscan.api: raise exception for !200 status 2017-03-01 00:40:47 -05:00
Joseph Schorr
3db4c15459 Pull out security scanner validation into validator class 2017-02-24 12:23:16 -05:00
Jimmy Zelinskie
c8034deab4 util.secscan.api: failover connection failures 2017-02-23 15:01:32 -05:00
Jimmy Zelinskie
1d6339e644 test.test_api_usage: fix secscan tests 2017-02-14 15:21:18 -05:00
Jimmy Zelinskie
3286566478 util.secscan.api: reorg try/catch 2017-02-14 15:21:17 -05:00
Jimmy Zelinskie
d2909c0e4d failover: store result in FailoverException 2017-02-14 14:36:36 -05:00
Jimmy Zelinskie
c2c6bc1e90 test: add qss read failover case 2017-02-03 19:20:13 -05:00
Jimmy Zelinskie
1d59095460 utils.secscan: linter fixes 2017-02-03 19:20:13 -05:00
Jimmy Zelinskie
e81926fcba util.secscan.api: init read-only failover 2017-02-03 19:20:13 -05:00
Joseph Schorr
7c1bb886db Security scanner ordered tuplize bug fix 
If only the old list is present, we still need to tuplize the entries.

Fixes https://sentry.io/coreos/backend-production/issues/207196561/
 2017-01-24 13:16:44 -05:00
josephschorr
aafcb592a6 Merge pull request #2257 from coreos-inc/clair-gc-take2 
feat(gc): Garbage collection for security scanning
 2017-01-17 14:49:36 -05:00
Joseph Schorr
d609e6a1c4 Security scanner garbage collection support 
Adds support for calling GC in the security scanner for any layers+storage removed by GC on the Quay side
 2016-12-22 14:55:26 -05:00
Joseph Schorr
5b3212ea0e Change security notification code to use the new stream diff reporters 
This ensures that even if security scanner pagination sends Old and New layer IDs on different pages, they will properly be handled across the entire notification.

Fixes https://www.pivotaltracker.com/story/show/136133657
 2016-12-20 12:50:19 -05:00
Joseph Schorr
ced0149520 Implement helper classes for tracking streaming diffs, both indexed and non-indexed 
These classes will be used to handle the Layer ID paginated diffs from Clair.
 2016-12-20 12:50:18 -05:00
Joseph Schorr
405eca074c Security scanner flow changes and auto-retry 
Changes the security scanner code to raise exceptions now for non-successful operations. One of the new exceptions raised is MissingParentLayerException, which, when raised, will cause the security worker to perform a full rescan of all parent images for the current layer, before trying once more to scan the current layer. This should allow the system to be "self-healing" in the case where the security scanner engine somehow loses or corrupts a parent layer.
 2016-12-16 15:38:09 -05:00
Joseph Schorr
15041ac5ed Add a fake security scanner class for easier testing 
The FakeSecurityScanner mocks out all calls that Quay is expected to make to the security scanner API, and returns faked data that can be adjusted by the calling test case
 2016-12-14 17:11:45 -05:00
Joseph Schorr
6871eb95b1 Send notifications for previously unscannable layers in QSS 
Following this change, if an image was previously indexed unsuccessfully, then we will send notifications once successfully indexed
 2016-12-14 11:25:45 -05:00
Joseph Schorr
624b2a8385 Have security scanner analyze only send notifications for *new* layers 
Following this change, anytime a layer is indexed by the security scanner, we only send notifications out if the layer previously had a security_indexed_engine value of `-1`, thus ensuring it has *never* been indexed previously. This will allow us to change to version of the security scanner upwards, and have all the images be re-indexed, without firing off notifications in a spammy manner.
 2016-12-13 23:17:11 -05:00
Evan Cordell
5686c80af1 Revert "Add GC of layers in Clair" 
This reverts 49872838ab
 2016-12-13 18:40:58 -05:00
Joseph Schorr
49872838ab Add GC of layers in Clair 
Fixes https://www.pivotaltracker.com/story/show/135583207
 2016-12-06 19:52:56 -05:00
Jake Moshenko
21e3001446 Add a bulk insert for queue and notifications. 
Use it for Clair spawned notifications.
 2016-12-06 14:00:16 -05:00
Joseph Schorr
3c8b87e086 Fix verbs in manifestlist 
All registry_tests now pass
 2016-09-26 14:49:58 -04:00
Joseph Schorr
541764d87b Fix get_priority_for_index method for non-int values 
Fixes #1607
 2016-07-11 15:04:50 -04:00
Joseph Schorr
8887f09ba8 Use the instance service key for registry JWT signing 2016-06-07 11:58:10 -04:00
josephschorr
d572a45a57 Merge pull request #1441 from coreos-inc/fastesttests 
Make security scan testing much faster
 2016-05-05 13:57:05 -04:00
Joseph Schorr
343a080833 Make security scan testing much faster 2016-05-05 13:55:24 -04:00
Jake Moshenko
75f5df6369 Add clair auth header in generalized interface 2016-05-05 13:28:06 -04:00
Joseph Schorr
232fa42897 Add testing of the new secscan-for-local endpoint and fix a bug 2016-05-04 21:47:03 -04:00
Jake Moshenko
9221a515de Use the registry API for security scanning 
when the storage engine doesn't support direct download url
 2016-05-04 18:04:06 -04:00
Joseph Schorr
2cbdecb043 Implement setup tool support for Clair 
Fixes #1387
 2016-05-04 13:40:50 -04:00
Evan Cordell
0c2ecec9a9 Don't check for client certs when talking to clair 2016-04-29 14:10:33 -04:00
Evan Cordell
f30a9e56f3 Be really sure about proxy protocol 2016-04-29 14:10:33 -04:00
Evan Cordell
8595140f38 Use signer proxy for all http(s) requests 2016-04-29 14:10:33 -04:00
Evan Cordell
f4d2fae5d8 Separate jwtproxy signer config from secscan config 2016-04-29 14:10:33 -04:00
Evan Cordell
474884acd7 Don't require certs for clair anymore 2016-04-29 14:10:33 -04:00
Evan Cordell
e499c4a8ef Actually go through signer proxy 2016-04-29 14:10:33 -04:00
Evan Cordell
9e7a501dae Authenticate in the other direction with jwtproxy 2016-04-29 14:10:33 -04:00
josephschorr
d63ec8c6b0 Merge pull request #1402 from coreos-inc/clairbugfixes 
Fix handling of Clair notifications without `New` block
 2016-04-22 15:11:51 -04:00
Joseph Schorr
34a8090328 Fix handling of Defcon 1 
Fixes #1397
 2016-04-22 13:21:35 -04:00
Joseph Schorr
3f8d51ebd7 Fix handling of Clair notifications without New block 
Fixes #1398
 2016-04-22 13:05:34 -04:00
Joseph Schorr
0e84a94146 Make analyzer handle images without features or vulnerabilities 2016-03-29 15:16:22 -04:00
Joseph Schorr
aa5587c93c Fixes and added tests for the security notification worker 
Fixes #1301

- Ensures that the worker uses pagination properly
- Ensures that the worker handles failure as expected
- Moves marking the notification as read to after the worker processes it
- Increases the number of layers requested to 100
 2016-03-18 20:28:06 -04:00
Quentin Machu
d093a7bde5 Merge pull request #1290 from Quentin-M/split_clair_clusters 
Split clair clusters
 2016-03-15 11:09:51 -04:00
Quentin Machu
81fe315171 Add ability to use another Clair stack for batch tasks 2016-03-14 14:28:34 -04:00
Joseph Schorr
821b09daaf Update Quay Sec UI as per feedback from design team 
Fixes #1281
 2016-03-10 14:49:36 -05:00
Quentin Machu
d36528a77a Increase POST timeout in secscan API 2016-03-04 11:59:00 -05:00
Quentin Machu
4f7a66ab0e Repair secscan's analyze_layer API call 2016-03-02 16:05:11 -05:00
Quentin Machu
888f976e8d Use a feature flag to toggle security notifications 2016-03-01 15:54:18 -05:00
Quentin Machu
672168ce78 Close Clair API connections 
This forces every API calls to be load-balanced properly.
 2016-02-29 14:52:38 -05:00