Commit graph

643 commits

Author SHA1 Message Date
Jimmy Zelinskie
5fee4d6d19 *: misc formatting cleanup 2016-10-17 21:43:45 -04:00
Jimmy Zelinskie
a42eb09a3e util.license: make bp-modification a method 2016-10-17 21:43:45 -04:00
Jimmy Zelinskie
6eb26d7998 configproviders: pass filemode when opening volume 2016-10-17 21:43:45 -04:00
Jimmy Zelinskie
0c5400b7d1 enforce license across registry blueprints 2016-10-17 21:43:45 -04:00
Joseph Schorr
8fe29c5b89 Add license upload step to the setup flow
Fixes #853
2016-10-17 21:43:15 -04:00
Joseph Schorr
5211c407ff Add license checking to Quay
Based off of mjibson's changes

Fixes #499
2016-10-17 21:43:15 -04:00
josephschorr
78f87d96bc Merge pull request #1986 from coreos-inc/external-tls
Add option to properly handle external TLS
2016-10-15 16:05:28 -04:00
Jake Moshenko
f04b018805 Write our users to Marketo as leads. 2016-10-14 16:29:11 -04:00
Jake Moshenko
013e27f7d5 Clean up mixpanel analytics a bit. 2016-10-13 15:03:04 -04:00
Joseph Schorr
5a8200f17a Add option to properly handle external TLS
Fixes #1984
2016-10-13 14:49:29 -04:00
Joseph Schorr
6ea51afa66 Add a configurable prometheus namespace for all metrics
Fixes #1918
2016-10-05 10:33:35 +03:00
josephschorr
684ace3b5a Merge pull request #1761 from coreos-inc/nginx-direct-download
Add feature flag to force all direct download URLs to be proxied
2016-09-29 22:46:57 +02:00
Evan Cordell
832ee89923 Add duration metric collector decorator (#1885)
Track time-to-start for builders
Track time-to-build for builders
Track ec2 builder fallbacks
Track build time
2016-09-29 15:44:06 -04:00
Joseph Schorr
6ae3faf7fc Add explicit config parameter to the JWT auth methods 2016-09-29 11:15:20 +02:00
Joseph Schorr
dd2e086a20 Add feature flag to force all direct download URLs to be proxied
Fixes #1667
2016-09-29 11:13:41 +02:00
Jimmy Zelinskie
fc7301be0d *: fix legacy imports
This change reorganizes imports and renames the legacy flask extensions.
2016-09-28 20:17:14 -04:00
Jimmy Zelinskie
ae16d24fd1 license: validate via key instance rather than PEM 2016-09-28 15:44:28 -04:00
josephschorr
e1771abe58 Merge pull request #739 from coreos-inc/license
Add license checking to Quay
2016-09-27 16:52:08 +02:00
Joseph Schorr
476576bb70 Add license checking to Quay
Based off of mjibson's changes

Fixes #499
2016-09-27 10:31:34 +02:00
Joseph Schorr
3c8b87e086 Fix verbs in manifestlist
All registry_tests now pass
2016-09-26 14:49:58 -04:00
Jimmy Zelinskie
59529569dc reorder imports 2016-09-26 14:48:05 -04:00
josephschorr
ad4efba802 Merge pull request #1830 from coreos-inc/superuser-dashboard
Add prometheus stats to enable better dashboarding
2016-09-26 17:19:22 +02:00
Joseph Schorr
25ed99f9ef Add feature flag to turn off requirement for team invitations
Fixes #1804
2016-09-20 16:45:00 -04:00
Joseph Schorr
c7beea2032 Fix handling of custom LDAP cert
This change moves the LDAP cert installation into a common script and reorganizes the startup scripts for creating and installing these certs

Fixes #1846
2016-09-19 17:55:08 -04:00
Joseph Schorr
1571b2867a Add executor name to the build metric 2016-09-16 16:26:04 -04:00
Joseph Schorr
30af8aef1a Add a worker for reporting global stats to Prometheus
Fixes #1789
2016-09-12 16:19:19 -04:00
Joseph Schorr
818ea38dac Add repo-specific reporting of repository builds 2016-09-09 15:36:54 -04:00
Joseph Schorr
c8a1b8abab Add prom stats for repository push, pull and verb actions 2016-09-09 15:13:58 -04:00
Jake Moshenko
1d8b72235a Add a helper method to Image to parse ancestor string. 2016-09-07 10:48:58 -04:00
josephschorr
480d890442 Merge pull request #1771 from coreos-inc/kubernetes-save-error
Make sure the Quay Enterprise Kubernetes namespace exists
2016-08-30 12:59:00 -04:00
Joseph Schorr
3f9c82462f Make sure the Quay Enterprise Kubernetes namespace exists
Prevents config from failing to save. Also clarifies any other errors that do occur.

Fixes #1449
2016-08-30 12:58:39 -04:00
Joseph Schorr
aa7c87d765 Fix locking via RedLock
Fixes #1777
2016-08-29 16:06:26 -04:00
Joseph Schorr
608ffd9663 Basic labels support
Adds basic labels support to the registry code (V2), and the API. Note that this does not yet add any UI related support.
2016-08-26 15:24:26 -04:00
Joseph Schorr
193040a473 Fix tag links
Fixes #1741
2016-08-17 15:06:10 -04:00
Joseph Schorr
afc2705b1c Have email read the enterprise logo 2016-08-09 12:18:35 -04:00
Ben Spoon
b0e34692cf Merge pull request #1674 from coreos-inc/new-quay-emails
New quay emails
2016-08-09 09:12:54 -07:00
Ben Spoon
2b92fded68 emails: address review feedback 2016-08-08 13:29:47 -07:00
Ben Spoon
004b834c72 emails: only show quay footer if coming from hosted 2016-08-04 11:55:55 -07:00
Ben Spoon
46a720285a emails: update payment failure admin link
addresses issue #1623
2016-08-04 11:55:50 -07:00
Ben Spoon
5019ef0b6b emails: change the app_link_handler to return just a uri
There is no need for an anchor tag any longer.
2016-08-04 11:55:48 -07:00
Joseph Schorr
770ac0016e Change validate method to work for all storages 2016-08-02 15:01:37 -04:00
Joseph Schorr
0fe3e6510a Prevent invalid tags on builds
Fixes #1632
2016-07-25 17:50:35 -07:00
Joseph Schorr
541764d87b Fix get_priority_for_index method for non-int values
Fixes #1607
2016-07-11 15:04:50 -04:00
Joseph Schorr
a1009af61c Move aggregator into its own repo and add it to the image 2016-07-05 15:39:51 -04:00
Joseph Schorr
713ba3abaf Further updates to the Prometheus client code 2016-07-01 14:16:51 -04:00
Jake Moshenko
668a8edc50 Refactor prometheus integration
Move prometheus to SaaS and make it a plugin
Move static callers to use metrics_queue plugin
Change local-docker to support different quay clone dirnames
Change prom_aggregator to use logrus
2016-07-01 14:16:50 -04:00
Matt Jibson
3d9acf2fff Use prometheus as a metric backend
This entails writing a metric aggregation program since each worker has its
own memory, and thus own metrics because of python gunicorn. The python
client is a simple wrapper that makes web requests to it.
2016-07-01 14:16:50 -04:00
Joseph Schorr
9558c0e937 Fix handling of Github API paths and add tests 2016-06-30 14:10:22 -04:00
Joseph Schorr
ab1756306b Switch to using the leeway parameter on JWT validation 2016-06-27 14:42:44 -04:00
Joseph Schorr
2983195a4a Fix OAuth key not found error for Dex
Fixes #1582
2016-06-27 13:38:11 -04:00
Joseph Schorr
2653d213c9 Add an allowed amount of clock skew to registry JWTs 2016-06-24 15:08:26 -04:00
Joseph Schorr
30ede029d5 Fix GeneratorFile for working with BufferedReader
The user files system uses a BufferedReader along with the magic library to determine the mime type of the user file being served. Currently, BufferedReader fails with an exception on Swift storage, because Swift storage returns a GeneratorFile, which is missing the `readable()` method.
2016-06-23 13:40:57 -04:00
josephschorr
7173d53030 Merge pull request #1549 from coreos-inc/certs
Switch to install custom LDAP cert by name
2016-06-21 15:13:44 -04:00
Joseph Schorr
66ec1d81ce Switch to install custom LDAP cert by name 2016-06-21 15:10:26 -04:00
josephschorr
9e6a264f5f Merge pull request #1523 from coreos-inc/verb-tag-cache-fix
Add a uniqueness hash to derived image storage to break caching over …
2016-06-20 16:38:25 -04:00
Joseph Schorr
a43b741f1b Add a uniqueness hash to derived image storage to break caching over tags
This allows converted ACIs and squashed images to be unique based on the specified tag.

Fixes #92
2016-06-20 16:34:52 -04:00
Jake Moshenko
22562b0156 Merge pull request #1559 from jakedt/finishthejob
Finish removing the AJAX indexing support.
2016-06-20 13:42:05 -04:00
Joseph Schorr
986d20bcad Switch to generic RedisError
Fixes #1558
2016-06-20 11:20:17 -04:00
Jake Moshenko
4130054ef3 Finish removing the AJAX indexing support. 2016-06-20 10:15:21 -04:00
Jake Moshenko
746728ba24 Remove escaped_fragment snapshot rendering. 2016-06-14 12:53:10 -04:00
josephschorr
58bef472d9 Merge pull request #1526 from coreos-inc/superuser-grant
Add ability for super users to take ownership of namespaces
2016-06-13 16:23:10 -04:00
Joseph Schorr
20816804e5 Add ability for super users to take ownership of namespaces
Fixes #1395
2016-06-13 16:22:52 -04:00
Jimmy Zelinskie
f15e5483e7 fix identation according to lint 2016-06-08 15:55:47 -04:00
Jimmy Zelinskie
9fb8b585b5 fix broken import 2016-06-08 15:55:29 -04:00
Joseph Schorr
71b2853f40 Make sure to iterate over a copy of the public_keys dictionary 2016-06-07 18:20:42 -04:00
Joseph Schorr
8887f09ba8 Use the instance service key for registry JWT signing 2016-06-07 11:58:10 -04:00
josephschorr
cad8746f9d Merge pull request #1502 from coreos-inc/image-replication
Enable storage replication for V2 and add backfill tool
2016-06-02 15:02:53 -04:00
Joseph Schorr
12924784ce Enable storage replication for V2 and add backfill tool
Fixes #1501
2016-06-02 14:36:08 -04:00
Jimmy Zelinskie
2317938bfa Merge pull request #1496 from jzelinskie/ripRMS
dockerfile: add check for GPL pip packages
2016-06-02 12:28:18 -04:00
Jimmy Zelinskie
8810157586 remove GPL'd timeparse library 2016-06-02 12:27:49 -04:00
Joseph Schorr
c61c3db728 Remove unused safetar file 2016-05-31 16:50:16 -04:00
Joseph Schorr
4ec3a6c231 Make ACI generation consistent across calls
This will ensure that no matter which signature we write for the generated ACI, it is correct for that image.
2016-05-26 17:09:19 -04:00
Joseph Schorr
f02d295dd8 Fix missing argument change 2016-05-23 17:44:22 -04:00
Joseph Schorr
f670c4c7a9 Change Signer to use the config provider and fix tests
Fixes the broken ACI tests
2016-05-23 17:10:03 -04:00
Jimmy Zelinskie
5568cc77b8 remove all default keys (#1485)
This change:
- Generates a new BitTorrent pepper by default
- Generates a new pagination key by default
- Changes the pagination key format to base64
- Removes selfsigned JWT certs
- Moves test keys to test/data
2016-05-23 16:00:48 -04:00
Jake Moshenko
4266ae7ce5 Fix the x5c header in our registry jwts. 2016-05-23 15:05:54 -04:00
Joseph Schorr
64fe11a5f1 Add ACI signing tests 2016-05-13 18:29:57 -04:00
josephschorr
d572a45a57 Merge pull request #1441 from coreos-inc/fastesttests
Make security scan testing much faster
2016-05-05 13:57:05 -04:00
Joseph Schorr
343a080833 Make security scan testing much faster 2016-05-05 13:55:24 -04:00
Jake Moshenko
75f5df6369 Add clair auth header in generalized interface 2016-05-05 13:28:06 -04:00
Joseph Schorr
232fa42897 Add testing of the new secscan-for-local endpoint and fix a bug 2016-05-04 21:47:03 -04:00
Jake Moshenko
9221a515de Use the registry API for security scanning
when the storage engine doesn't support direct download url
2016-05-04 18:04:06 -04:00
Joseph Schorr
73fa593d02 Various small fixes in prep for QE release 2016-05-04 15:20:27 -04:00
josephschorr
f55fd2049f Merge pull request #1433 from coreos-inc/ldapoptions
Add additional options for LDAP
2016-05-04 14:06:29 -04:00
Joseph Schorr
42515ed9ec Add additional options for LDAP
Fixes #1420
2016-05-04 13:59:20 -04:00
Joseph Schorr
2cbdecb043 Implement setup tool support for Clair
Fixes #1387
2016-05-04 13:40:50 -04:00
Jimmy Zelinskie
437ec84c9f torrent: use quay.pem to mint JWT (#1425) 2016-05-02 18:10:16 -04:00
Evan Cordell
af4106e5c0 Fix generatepresharedkey script 2016-04-29 15:21:19 -05:00
Evan Cordell
2242c6773d Add 'Automatic' ServiceKeyApprovalType 2016-04-29 14:10:33 -04:00
Evan Cordell
c766727d1d address review comments
- more inline documentation
 - don't explicitly specify audience
 - approver is optional in `generate_key`
 - ADD -> RUN for better caching of jwtproxy
2016-04-29 14:10:33 -04:00
Evan Cordell
0c2ecec9a9 Don't check for client certs when talking to clair 2016-04-29 14:10:33 -04:00
Evan Cordell
9ffc32f680 Generate preshared key on boot 2016-04-29 14:10:33 -04:00
Evan Cordell
f30a9e56f3 Be really sure about proxy protocol 2016-04-29 14:10:33 -04:00
Evan Cordell
8595140f38 Use signer proxy for all http(s) requests 2016-04-29 14:10:33 -04:00
Evan Cordell
f4d2fae5d8 Separate jwtproxy signer config from secscan config 2016-04-29 14:10:33 -04:00
Evan Cordell
474884acd7 Don't require certs for clair anymore 2016-04-29 14:10:33 -04:00
Evan Cordell
e499c4a8ef Actually go through signer proxy 2016-04-29 14:10:33 -04:00
Evan Cordell
9e7a501dae Authenticate in the other direction with jwtproxy 2016-04-29 14:10:33 -04:00
Joseph Schorr
dc9bcec9ce Add pre shared generation tool 2016-04-29 14:09:37 -04:00
Jimmy Zelinskie
6577ac3e62 mv JWK-canonicalization util.security.fingerprint 2016-04-29 14:05:16 -04:00
Joseph Schorr
11ff3e9b59 keys ui WIP 2016-04-29 14:05:16 -04:00
Jimmy Zelinskie
97ae800e6c canonicalize json 2016-04-29 13:38:25 -04:00
josephschorr
d63ec8c6b0 Merge pull request #1402 from coreos-inc/clairbugfixes
Fix handling of Clair notifications without `New` block
2016-04-22 15:11:51 -04:00
Joseph Schorr
34a8090328 Fix handling of Defcon 1
Fixes #1397
2016-04-22 13:21:35 -04:00
Joseph Schorr
3f8d51ebd7 Fix handling of Clair notifications without New block
Fixes #1398
2016-04-22 13:05:34 -04:00
josephschorr
affb600423 Merge pull request #1328 from coreos-inc/queuefilefix
Fix QueueFile to support read-to-end semantics and add some tests
2016-04-08 18:07:06 -04:00
Jake Moshenko
45e7c94586 Initialize the db for fixsequences 2016-04-01 14:26:19 -04:00
Jake Moshenko
bd5b44cbd2 Move the sequence fixer to a separate tool which can be run 2016-04-01 13:46:13 -04:00
josephschorr
b9f47f6761 Merge pull request #1285 from coreos-inc/configmaildefaults
Fix mail and signing defaults
2016-03-31 12:31:26 -04:00
Joseph Schorr
6251e63e0e Fix QueueFile to support read-to-end semantics and add some tests 2016-03-31 12:06:49 -04:00
Joseph Schorr
0e84a94146 Make analyzer handle images without features or vulnerabilities 2016-03-29 15:16:22 -04:00
Joseph Schorr
dc8f9713f8 Change logs worker to use a global lock in the inner loop and move storage out of the transaction 2016-03-24 14:09:48 -04:00
Joseph Schorr
aa5587c93c Fixes and added tests for the security notification worker
Fixes #1301

- Ensures that the worker uses pagination properly
- Ensures that the worker handles failure as expected
- Moves marking the notification as read to after the worker processes it
- Increases the number of layers requested to 100
2016-03-18 20:28:06 -04:00
Jimmy Zelinskie
8af0b887ef fix broken tests 2016-03-18 15:48:41 -04:00
Jimmy Zelinskie
5094e1f712 move slash_join to prevent local imports 2016-03-18 15:09:25 -04:00
Jimmy Zelinskie
e5d8a431f4 replace use of URL joining with slash_join 2016-03-18 14:56:10 -04:00
Jimmy Zelinskie
bf477b6b9c add slash_join helper and tests 2016-03-18 14:56:10 -04:00
Jimmy Zelinskie
0dcfcebe34 remove unused imports and lint 2016-03-18 14:56:09 -04:00
Jimmy Zelinskie
bcea268fcb use app.gitlab_trigger for config data
This includes defaults and makes the structure of the Gitlab trigger
parallel the GitHub trigger.
2016-03-18 14:56:09 -04:00
Quentin Machu
d093a7bde5 Merge pull request #1290 from Quentin-M/split_clair_clusters
Split clair clusters
2016-03-15 11:09:51 -04:00
Quentin Machu
81fe315171 Add ability to use another Clair stack for batch tasks 2016-03-14 14:28:34 -04:00
Joseph Schorr
821b09daaf Update Quay Sec UI as per feedback from design team
Fixes #1281
2016-03-10 14:49:36 -05:00
Joseph Schorr
8e1727b6d3 Fix mail and signing defaults 2016-03-08 18:08:40 -05:00
Quentin Machu
897df4de32 Merge pull request #1271 from coreos-inc/allocator_bs
Repair allocator (min/max swapped)
2016-03-04 12:06:04 -05:00
Quentin Machu
d36528a77a Increase POST timeout in secscan API 2016-03-04 11:59:00 -05:00
Quentin Machu
4f7a66ab0e Repair secscan's analyze_layer API call 2016-03-02 16:05:11 -05:00
Quentin Machu
c8bf55c2bb Repair allocator (min/max swapped) 2016-03-02 14:51:54 -05:00
Quentin Machu
c29ce8e1a1 Merge pull request #1268 from Quentin-M/secnotif_feature_flag
Use a feature flag to toggle security notifications
2016-03-01 15:54:37 -05:00
Quentin Machu
888f976e8d Use a feature flag to toggle security notifications 2016-03-01 15:54:18 -05:00
Quentin Machu
ea013b8066 make min_index optionnal in allocator's constructor 2016-03-01 14:54:38 -05:00
Quentin Machu
672168ce78 Close Clair API connections
This forces every API calls to be load-balanced properly.
2016-02-29 14:52:38 -05:00
Joseph Schorr
ae9140caae Implement new vulnerabilities and packages tabs.
Fixes https://github.com/coreos-inc/design/issues/268
2016-02-25 17:09:29 -05:00
Joseph Schorr
f498e92d58 Implement against new Clair paginated notification system 2016-02-25 15:58:42 -05:00
Joseph Schorr
c0374d71c9 Refactor the security worker and API calls and add a bunch of tests 2016-02-25 12:29:41 -05:00
Quentin Machu
0183c519f7 Merge pull request #1253 from Quentin-M/clair2
Adapt securityworker, secscan API and Quay UI for Clair 1.0
2016-02-19 18:21:25 -05:00
josephschorr
11af123ba5 Merge pull request #1244 from coreos-inc/enableaci
Add UI to the setup tool for enabling ACI conversion
2016-02-17 12:29:48 -05:00
Joseph Schorr
1940fd9939 Add UI to the setup tool for enabling ACI conversion
Fixes #1211
2016-02-17 12:05:48 -05:00
josephschorr
6f9fc7fc08 Merge pull request #1225 from coreos-inc/setuptooltest
Add tests for superuser config API calls
2016-02-16 17:01:43 -05:00
josephschorr
81a36ee3b8 Merge pull request #1217 from coreos-inc/v2pagination
Fix V2 catalog and tag pagination
2016-02-16 15:34:49 -05:00
Quentin Machu
c8d825c232 expose min_id in allocator.py 2016-02-16 15:16:22 -05:00
Jake Moshenko
88d84aa182 Fixes for content checksum and torrent pieces backfill
Remove null handler from app.py, was silencing other logs
2016-02-11 16:53:18 -05:00
Joseph Schorr
03533db5a3 Add tests for superuser config API calls 2016-02-11 11:04:37 +02:00
Joseph Schorr
db0eab0461 Fix V2 catalog and tag pagination 2016-02-10 00:25:33 +02:00
Jimmy Zelinskie
5828d8e716 private swarms torrents 2016-02-08 13:56:31 -05:00
Joseph Schorr
1536709c02 Small fixes 2016-01-29 20:01:17 +02:00
Jake Moshenko
01a92a66ba Refresh base image and python dependencies 2016-01-27 11:36:40 -05:00
Joseph Schorr
335c8eb3a9 Add 2 day TTL to page tokens 2016-01-26 14:04:03 -05:00
Joseph Schorr
b4bddacedb Switch to Fernet crypto as per gtank's recommendation 2016-01-26 12:50:48 -05:00
Jimmy Zelinskie
85ae1a2a0a Merge pull request #1161 from jzelinskie/torrenthmac
misc torrent changes
2016-01-22 23:02:44 -05:00
Joseph Schorr
e4ffaff869 Fix Docker Auth and our V2 registry paths to support library (i.e. namespace-less) repositories.
This support is placed behind a feature flag.
2016-01-22 15:54:06 -05:00