Commit graph

548 commits

Author SHA1 Message Date
Jake Moshenko
668a8edc50 Refactor prometheus integration
Move prometheus to SaaS and make it a plugin
Move static callers to use metrics_queue plugin
Change local-docker to support different quay clone dirnames
Change prom_aggregator to use logrus
2016-07-01 14:16:50 -04:00
Matt Jibson
3d9acf2fff Use prometheus as a metric backend
This entails writing a metric aggregation program since each worker has its
own memory, and thus own metrics because of python gunicorn. The python
client is a simple wrapper that makes web requests to it.
2016-07-01 14:16:50 -04:00
Joseph Schorr
9558c0e937 Fix handling of Github API paths and add tests 2016-06-30 14:10:22 -04:00
Joseph Schorr
ab1756306b Switch to using the leeway parameter on JWT validation 2016-06-27 14:42:44 -04:00
Joseph Schorr
2983195a4a Fix OAuth key not found error for Dex
Fixes #1582
2016-06-27 13:38:11 -04:00
Joseph Schorr
2653d213c9 Add an allowed amount of clock skew to registry JWTs 2016-06-24 15:08:26 -04:00
Joseph Schorr
30ede029d5 Fix GeneratorFile for working with BufferedReader
The user files system uses a BufferedReader along with the magic library to determine the mime type of the user file being served. Currently, BufferedReader fails with an exception on Swift storage, because Swift storage returns a GeneratorFile, which is missing the `readable()` method.
2016-06-23 13:40:57 -04:00
josephschorr
7173d53030 Merge pull request #1549 from coreos-inc/certs
Switch to install custom LDAP cert by name
2016-06-21 15:13:44 -04:00
Joseph Schorr
66ec1d81ce Switch to install custom LDAP cert by name 2016-06-21 15:10:26 -04:00
josephschorr
9e6a264f5f Merge pull request #1523 from coreos-inc/verb-tag-cache-fix
Add a uniqueness hash to derived image storage to break caching over …
2016-06-20 16:38:25 -04:00
Joseph Schorr
a43b741f1b Add a uniqueness hash to derived image storage to break caching over tags
This allows converted ACIs and squashed images to be unique based on the specified tag.

Fixes #92
2016-06-20 16:34:52 -04:00
Jake Moshenko
22562b0156 Merge pull request #1559 from jakedt/finishthejob
Finish removing the AJAX indexing support.
2016-06-20 13:42:05 -04:00
Joseph Schorr
986d20bcad Switch to generic RedisError
Fixes #1558
2016-06-20 11:20:17 -04:00
Jake Moshenko
4130054ef3 Finish removing the AJAX indexing support. 2016-06-20 10:15:21 -04:00
Jake Moshenko
746728ba24 Remove escaped_fragment snapshot rendering. 2016-06-14 12:53:10 -04:00
josephschorr
58bef472d9 Merge pull request #1526 from coreos-inc/superuser-grant
Add ability for super users to take ownership of namespaces
2016-06-13 16:23:10 -04:00
Joseph Schorr
20816804e5 Add ability for super users to take ownership of namespaces
Fixes #1395
2016-06-13 16:22:52 -04:00
Jimmy Zelinskie
f15e5483e7 fix identation according to lint 2016-06-08 15:55:47 -04:00
Jimmy Zelinskie
9fb8b585b5 fix broken import 2016-06-08 15:55:29 -04:00
Joseph Schorr
71b2853f40 Make sure to iterate over a copy of the public_keys dictionary 2016-06-07 18:20:42 -04:00
Joseph Schorr
8887f09ba8 Use the instance service key for registry JWT signing 2016-06-07 11:58:10 -04:00
josephschorr
cad8746f9d Merge pull request #1502 from coreos-inc/image-replication
Enable storage replication for V2 and add backfill tool
2016-06-02 15:02:53 -04:00
Joseph Schorr
12924784ce Enable storage replication for V2 and add backfill tool
Fixes #1501
2016-06-02 14:36:08 -04:00
Jimmy Zelinskie
2317938bfa Merge pull request #1496 from jzelinskie/ripRMS
dockerfile: add check for GPL pip packages
2016-06-02 12:28:18 -04:00
Jimmy Zelinskie
8810157586 remove GPL'd timeparse library 2016-06-02 12:27:49 -04:00
Joseph Schorr
c61c3db728 Remove unused safetar file 2016-05-31 16:50:16 -04:00
Joseph Schorr
4ec3a6c231 Make ACI generation consistent across calls
This will ensure that no matter which signature we write for the generated ACI, it is correct for that image.
2016-05-26 17:09:19 -04:00
Joseph Schorr
f02d295dd8 Fix missing argument change 2016-05-23 17:44:22 -04:00
Joseph Schorr
f670c4c7a9 Change Signer to use the config provider and fix tests
Fixes the broken ACI tests
2016-05-23 17:10:03 -04:00
Jimmy Zelinskie
5568cc77b8 remove all default keys (#1485)
This change:
- Generates a new BitTorrent pepper by default
- Generates a new pagination key by default
- Changes the pagination key format to base64
- Removes selfsigned JWT certs
- Moves test keys to test/data
2016-05-23 16:00:48 -04:00
Jake Moshenko
4266ae7ce5 Fix the x5c header in our registry jwts. 2016-05-23 15:05:54 -04:00
Joseph Schorr
64fe11a5f1 Add ACI signing tests 2016-05-13 18:29:57 -04:00
josephschorr
d572a45a57 Merge pull request #1441 from coreos-inc/fastesttests
Make security scan testing much faster
2016-05-05 13:57:05 -04:00
Joseph Schorr
343a080833 Make security scan testing much faster 2016-05-05 13:55:24 -04:00
Jake Moshenko
75f5df6369 Add clair auth header in generalized interface 2016-05-05 13:28:06 -04:00
Joseph Schorr
232fa42897 Add testing of the new secscan-for-local endpoint and fix a bug 2016-05-04 21:47:03 -04:00
Jake Moshenko
9221a515de Use the registry API for security scanning
when the storage engine doesn't support direct download url
2016-05-04 18:04:06 -04:00
Joseph Schorr
73fa593d02 Various small fixes in prep for QE release 2016-05-04 15:20:27 -04:00
josephschorr
f55fd2049f Merge pull request #1433 from coreos-inc/ldapoptions
Add additional options for LDAP
2016-05-04 14:06:29 -04:00
Joseph Schorr
42515ed9ec Add additional options for LDAP
Fixes #1420
2016-05-04 13:59:20 -04:00
Joseph Schorr
2cbdecb043 Implement setup tool support for Clair
Fixes #1387
2016-05-04 13:40:50 -04:00
Jimmy Zelinskie
437ec84c9f torrent: use quay.pem to mint JWT (#1425) 2016-05-02 18:10:16 -04:00
Evan Cordell
af4106e5c0 Fix generatepresharedkey script 2016-04-29 15:21:19 -05:00
Evan Cordell
2242c6773d Add 'Automatic' ServiceKeyApprovalType 2016-04-29 14:10:33 -04:00
Evan Cordell
c766727d1d address review comments
- more inline documentation
 - don't explicitly specify audience
 - approver is optional in `generate_key`
 - ADD -> RUN for better caching of jwtproxy
2016-04-29 14:10:33 -04:00
Evan Cordell
0c2ecec9a9 Don't check for client certs when talking to clair 2016-04-29 14:10:33 -04:00
Evan Cordell
9ffc32f680 Generate preshared key on boot 2016-04-29 14:10:33 -04:00
Evan Cordell
f30a9e56f3 Be really sure about proxy protocol 2016-04-29 14:10:33 -04:00
Evan Cordell
8595140f38 Use signer proxy for all http(s) requests 2016-04-29 14:10:33 -04:00
Evan Cordell
f4d2fae5d8 Separate jwtproxy signer config from secscan config 2016-04-29 14:10:33 -04:00
Evan Cordell
474884acd7 Don't require certs for clair anymore 2016-04-29 14:10:33 -04:00
Evan Cordell
e499c4a8ef Actually go through signer proxy 2016-04-29 14:10:33 -04:00
Evan Cordell
9e7a501dae Authenticate in the other direction with jwtproxy 2016-04-29 14:10:33 -04:00
Joseph Schorr
dc9bcec9ce Add pre shared generation tool 2016-04-29 14:09:37 -04:00
Jimmy Zelinskie
6577ac3e62 mv JWK-canonicalization util.security.fingerprint 2016-04-29 14:05:16 -04:00
Joseph Schorr
11ff3e9b59 keys ui WIP 2016-04-29 14:05:16 -04:00
Jimmy Zelinskie
97ae800e6c canonicalize json 2016-04-29 13:38:25 -04:00
josephschorr
d63ec8c6b0 Merge pull request #1402 from coreos-inc/clairbugfixes
Fix handling of Clair notifications without `New` block
2016-04-22 15:11:51 -04:00
Joseph Schorr
34a8090328 Fix handling of Defcon 1
Fixes #1397
2016-04-22 13:21:35 -04:00
Joseph Schorr
3f8d51ebd7 Fix handling of Clair notifications without New block
Fixes #1398
2016-04-22 13:05:34 -04:00
josephschorr
affb600423 Merge pull request #1328 from coreos-inc/queuefilefix
Fix QueueFile to support read-to-end semantics and add some tests
2016-04-08 18:07:06 -04:00
Jake Moshenko
45e7c94586 Initialize the db for fixsequences 2016-04-01 14:26:19 -04:00
Jake Moshenko
bd5b44cbd2 Move the sequence fixer to a separate tool which can be run 2016-04-01 13:46:13 -04:00
josephschorr
b9f47f6761 Merge pull request #1285 from coreos-inc/configmaildefaults
Fix mail and signing defaults
2016-03-31 12:31:26 -04:00
Joseph Schorr
6251e63e0e Fix QueueFile to support read-to-end semantics and add some tests 2016-03-31 12:06:49 -04:00
Joseph Schorr
0e84a94146 Make analyzer handle images without features or vulnerabilities 2016-03-29 15:16:22 -04:00
Joseph Schorr
dc8f9713f8 Change logs worker to use a global lock in the inner loop and move storage out of the transaction 2016-03-24 14:09:48 -04:00
Joseph Schorr
aa5587c93c Fixes and added tests for the security notification worker
Fixes #1301

- Ensures that the worker uses pagination properly
- Ensures that the worker handles failure as expected
- Moves marking the notification as read to after the worker processes it
- Increases the number of layers requested to 100
2016-03-18 20:28:06 -04:00
Jimmy Zelinskie
8af0b887ef fix broken tests 2016-03-18 15:48:41 -04:00
Jimmy Zelinskie
5094e1f712 move slash_join to prevent local imports 2016-03-18 15:09:25 -04:00
Jimmy Zelinskie
e5d8a431f4 replace use of URL joining with slash_join 2016-03-18 14:56:10 -04:00
Jimmy Zelinskie
bf477b6b9c add slash_join helper and tests 2016-03-18 14:56:10 -04:00
Jimmy Zelinskie
0dcfcebe34 remove unused imports and lint 2016-03-18 14:56:09 -04:00
Jimmy Zelinskie
bcea268fcb use app.gitlab_trigger for config data
This includes defaults and makes the structure of the Gitlab trigger
parallel the GitHub trigger.
2016-03-18 14:56:09 -04:00
Quentin Machu
d093a7bde5 Merge pull request #1290 from Quentin-M/split_clair_clusters
Split clair clusters
2016-03-15 11:09:51 -04:00
Quentin Machu
81fe315171 Add ability to use another Clair stack for batch tasks 2016-03-14 14:28:34 -04:00
Joseph Schorr
821b09daaf Update Quay Sec UI as per feedback from design team
Fixes #1281
2016-03-10 14:49:36 -05:00
Joseph Schorr
8e1727b6d3 Fix mail and signing defaults 2016-03-08 18:08:40 -05:00
Quentin Machu
897df4de32 Merge pull request #1271 from coreos-inc/allocator_bs
Repair allocator (min/max swapped)
2016-03-04 12:06:04 -05:00
Quentin Machu
d36528a77a Increase POST timeout in secscan API 2016-03-04 11:59:00 -05:00
Quentin Machu
4f7a66ab0e Repair secscan's analyze_layer API call 2016-03-02 16:05:11 -05:00
Quentin Machu
c8bf55c2bb Repair allocator (min/max swapped) 2016-03-02 14:51:54 -05:00
Quentin Machu
c29ce8e1a1 Merge pull request #1268 from Quentin-M/secnotif_feature_flag
Use a feature flag to toggle security notifications
2016-03-01 15:54:37 -05:00
Quentin Machu
888f976e8d Use a feature flag to toggle security notifications 2016-03-01 15:54:18 -05:00
Quentin Machu
ea013b8066 make min_index optionnal in allocator's constructor 2016-03-01 14:54:38 -05:00
Quentin Machu
672168ce78 Close Clair API connections
This forces every API calls to be load-balanced properly.
2016-02-29 14:52:38 -05:00
Joseph Schorr
ae9140caae Implement new vulnerabilities and packages tabs.
Fixes https://github.com/coreos-inc/design/issues/268
2016-02-25 17:09:29 -05:00
Joseph Schorr
f498e92d58 Implement against new Clair paginated notification system 2016-02-25 15:58:42 -05:00
Joseph Schorr
c0374d71c9 Refactor the security worker and API calls and add a bunch of tests 2016-02-25 12:29:41 -05:00
Quentin Machu
0183c519f7 Merge pull request #1253 from Quentin-M/clair2
Adapt securityworker, secscan API and Quay UI for Clair 1.0
2016-02-19 18:21:25 -05:00
josephschorr
11af123ba5 Merge pull request #1244 from coreos-inc/enableaci
Add UI to the setup tool for enabling ACI conversion
2016-02-17 12:29:48 -05:00
Joseph Schorr
1940fd9939 Add UI to the setup tool for enabling ACI conversion
Fixes #1211
2016-02-17 12:05:48 -05:00
josephschorr
6f9fc7fc08 Merge pull request #1225 from coreos-inc/setuptooltest
Add tests for superuser config API calls
2016-02-16 17:01:43 -05:00
josephschorr
81a36ee3b8 Merge pull request #1217 from coreos-inc/v2pagination
Fix V2 catalog and tag pagination
2016-02-16 15:34:49 -05:00
Quentin Machu
c8d825c232 expose min_id in allocator.py 2016-02-16 15:16:22 -05:00
Jake Moshenko
88d84aa182 Fixes for content checksum and torrent pieces backfill
Remove null handler from app.py, was silencing other logs
2016-02-11 16:53:18 -05:00
Joseph Schorr
03533db5a3 Add tests for superuser config API calls 2016-02-11 11:04:37 +02:00
Joseph Schorr
db0eab0461 Fix V2 catalog and tag pagination 2016-02-10 00:25:33 +02:00
Jimmy Zelinskie
5828d8e716 private swarms torrents 2016-02-08 13:56:31 -05:00
Joseph Schorr
1536709c02 Small fixes 2016-01-29 20:01:17 +02:00