Commit graph

897 commits

Author SHA1 Message Date
Joseph Schorr
462f47924e More detailed namespace validation
Fixes namespace validation to use the proper regex for checking length, as well as showing the proper messaging if the entered namespace is invalid

[Delivers #137830461]
2017-01-17 17:31:59 -05:00
josephschorr
aafcb592a6 Merge pull request #2257 from coreos-inc/clair-gc-take2
feat(gc): Garbage collection for security scanning
2017-01-17 14:49:36 -05:00
josephschorr
eb2cafacd4 Merge pull request #2249 from coreos-inc/notifier-fixes
Security notification pagination fix
2017-01-17 11:33:25 -05:00
josephschorr
ac8cddc5a9 Merge pull request #2274 from coreos-inc/custom-cert-management
Custom SSL certificates config panel
2017-01-13 16:24:47 -05:00
josephschorr
6539fa3b20 Merge pull request #2259 from coreos-inc/delete-abuse-tool
Add tool for handling abusing users
2017-01-13 16:22:15 -05:00
Joseph Schorr
1cbacbbb63 Add tool for handling abusing users 2017-01-13 14:42:03 -05:00
Joseph Schorr
7e0fbeb625 Custom SSL certificates config panel
Adds a new panel to the superuser config tool, for managing custom SSL certificates in the config bundle

[Delivers #135586525]
2017-01-13 14:34:35 -05:00
Joseph Schorr
3a24871422 Add SSL certificate utility and tests 2017-01-10 17:06:13 -05:00
Joseph Schorr
f1c9965edf Add more volume file operations and cleanup k8s provider code 2017-01-10 17:06:13 -05:00
Joseph Schorr
29d6abddb5 Linter fixes 2017-01-10 17:06:13 -05:00
EvB
a7122db250 fix(cloudwatch): randomize sleep interval 2017-01-05 11:41:12 -05:00
Jake Moshenko
6c84b9330b Merge pull request #2251 from jakedt/fixaci
Fix port mapping for ACI conversion from newer Docker manifests.
2016-12-27 14:13:03 -05:00
Joseph Schorr
d609e6a1c4 Security scanner garbage collection support
Adds support for calling GC in the security scanner for any layers+storage removed by GC on the Quay side
2016-12-22 14:55:26 -05:00
Joseph Schorr
9413e25123 Change georeplication queuing to use new batch system 2016-12-21 17:44:30 -05:00
Jake Moshenko
d58a1ca35a Fix port mapping for ACI conversion from newer Docker manifests. 2016-12-20 14:01:06 -05:00
Joseph Schorr
5b3212ea0e Change security notification code to use the new stream diff reporters
This ensures that even if security scanner pagination sends Old and New layer IDs on different pages, they will properly be handled across the entire notification.

Fixes https://www.pivotaltracker.com/story/show/136133657
2016-12-20 12:50:19 -05:00
Joseph Schorr
ced0149520 Implement helper classes for tracking streaming diffs, both indexed and non-indexed
These classes will be used to handle the Layer ID paginated diffs from Clair.
2016-12-20 12:50:18 -05:00
Joseph Schorr
405eca074c Security scanner flow changes and auto-retry
Changes the security scanner code to raise exceptions now for non-successful operations. One of the new exceptions raised is MissingParentLayerException, which, when raised, will cause the security worker to perform a full rescan of all parent images for the current layer, before trying once more to scan the current layer. This should allow the system to be "self-healing" in the case where the security scanner engine somehow loses or corrupts a parent layer.
2016-12-16 15:38:09 -05:00
josephschorr
9fa16679f8 Merge pull request #2238 from coreos-inc/fake-clair
Add a fake security scanner class for easier testing
2016-12-15 20:51:24 -05:00
Brad Ison
2730c26b2e Merge pull request #2237 from coreos-inc/metrics-labels
Don't record size in chunk upload metrics
2016-12-15 14:20:34 -05:00
Brad Ison
df7366eace Add chunk size metric 2016-12-15 13:20:16 -05:00
Joseph Schorr
15041ac5ed Add a fake security scanner class for easier testing
The FakeSecurityScanner mocks out all calls that Quay is expected to make to the security scanner API, and returns faked data that can be adjusted by the calling test case
2016-12-14 17:11:45 -05:00
Brad Ison
8f59ac1251 Don't record size in chunk upload metrics 2016-12-14 12:16:02 -05:00
Joseph Schorr
6871eb95b1 Send notifications for previously unscannable layers in QSS
Following this change, if an image was previously indexed unsuccessfully, then we will send notifications once successfully indexed
2016-12-14 11:25:45 -05:00
Joseph Schorr
624b2a8385 Have security scanner analyze only send notifications for *new* layers
Following this change, anytime a layer is indexed by the security scanner, we only send notifications out if the layer previously had a security_indexed_engine value of `-1`, thus ensuring it has *never* been indexed previously. This will allow us to change to version of the security scanner upwards, and have all the images be re-indexed, without firing off notifications in a spammy manner.
2016-12-13 23:17:11 -05:00
Evan Cordell
5686c80af1 Revert "Add GC of layers in Clair"
This reverts 49872838ab
2016-12-13 18:40:58 -05:00
Evan Cordell
dd5f7cbe6c Fix the ephemeral build metrics 2016-12-13 18:28:04 -05:00
Joseph Schorr
1e5b97318a Fix loading of public keys for OIDC under Linux
Python's crypto lib under Linux has issues with loading PEM-encoded keys, so we just load it as a DER here and give PyJWT the key *instance* to use directly.
2016-12-09 14:26:56 -05:00
Joseph Schorr
dbdcb802b1 Add end-to-end OAuth login and attach tests 2016-12-08 18:35:42 -05:00
Joseph Schorr
49872838ab Add GC of layers in Clair
Fixes https://www.pivotaltracker.com/story/show/135583207
2016-12-06 19:52:56 -05:00
Jake Moshenko
21e3001446 Add a bulk insert for queue and notifications.
Use it for Clair spawned notifications.
2016-12-06 14:00:16 -05:00
Charlton Austin
edd9dcd7f6 Adding in some metrics around clair sec scan. 2016-12-01 16:50:02 -05:00
Joseph Schorr
236655adb4 Fix config validator for storage and add a test suite
Note that the test suite doesn't fully verify that each validation succeeds; rather, it ensures that the proper system (storage, security scanning, etc) is called with the configuration and returns at all (usually with an expected error). This should prevent us from forgetting to update these code paths when we change config-based systems. Longer term, we might want to have these tests stand up fake/mock versions of the endpoint services as well, for end-to-end testing.
2016-11-30 11:58:41 -05:00
Joseph Schorr
1a61ef4e04 Report the user's name and company to Marketo
Also fixes the API to report the other changes (username and email) as well
2016-11-14 17:34:50 -05:00
josephschorr
74e54bdbbb Merge pull request #1872 from coreos-inc/qe-torrent
Add QE setup tool support for BitTorrent downloads
2016-11-11 13:56:22 -05:00
Jake Moshenko
b5834a8a66 Collapse all migrations prior to 2.0.0 into one. 2016-11-10 17:31:00 -05:00
Joseph Schorr
74c3346562 Add a warning bar when the license will become invalid in a week 2016-11-08 14:24:55 -05:00
Joseph Schorr
4b926ae189 Add new metrics as requested by some customers
Note that the `status` field on the pull and push metrics will eventually be set to False for failed pulls and pushes in a followup PR
2016-11-03 15:28:40 -04:00
Joseph Schorr
681f975df5 Add QE setup tool support for BitTorrent downloads
Fixes #1871
2016-11-02 17:32:12 -04:00
josephschorr
840ea4e768 Merge pull request #2047 from coreos-inc/external-auth-email-optional
Make email addresses optional in external auth if email feature is turned off
2016-10-31 14:16:33 -04:00
Joseph Schorr
3a473cad2a Enable permanent sessions
Fixes #1955
2016-10-31 13:52:09 -04:00
Joseph Schorr
d7f56350a4 Make email addresses optional in external auth if email feature is turned off
Before this change, external auth such as Keystone would fail if a user without an email address tried to login, even if the email feature was disabled.
2016-10-31 13:50:24 -04:00
josephschorr
934cdecbd6 Merge pull request #1905 from coreos-inc/external-auth-search
Add support for entity search against external auth users not yet linked
2016-10-27 16:06:42 -04:00
Joseph Schorr
b3d1d7227c Add support to Keystone Auth for external user linking
Also adds Keystone V3 support
2016-10-27 15:42:03 -04:00
Joseph Schorr
fbb524e34e Add support to ExternalJWT Auth for external user linking 2016-10-27 15:42:03 -04:00
Jake Moshenko
45bacbabaa s/Regions/Deployments 2016-10-24 16:04:04 -04:00
josephschorr
67dde6e154 Merge pull request #1852 from coreos-inc/underscore_orgs
Better handling of namespace validation to fix a number of issues
2016-10-20 13:36:32 -04:00
Joseph Schorr
3a68740ff7 Better handling of namespace validation to fix a number of issues
- Fixes a bug which allows for underscores at the beginning of namespaces: Fixes #1849
- Allows dots and dashes for newer Docker clients: Fixes #1188
- Has the UI display better messaging associated with namespace entry
2016-10-20 13:32:22 -04:00
Joseph Schorr
213cc856e4 Fix UI for real license handling
Following this change, the user gets detailed errors and entitlement information
2016-10-19 17:49:15 -04:00
Joseph Schorr
2eabf1a291 Fix tests and test provider for real license format 2016-10-18 23:44:08 -04:00
Jake Moshenko
9f1c12e413 Refactor our license code to be entitlement centric. 2016-10-18 22:33:28 -04:00
Jake Moshenko
d90398e9ff Change the monthly license grace period to 11 months. 2016-10-18 18:46:40 -04:00
Joseph Schorr
67f828279d Switch the license validator to use config_provider and have a test license
Fixes the broken tests currently which try (and fail) to read the license file
2016-10-18 11:44:13 -04:00
Joseph Schorr
ee96693252 Add superuser config section for updating license 2016-10-17 21:44:25 -04:00
Jimmy Zelinskie
5fee4d6d19 *: misc formatting cleanup 2016-10-17 21:43:45 -04:00
Jimmy Zelinskie
a42eb09a3e util.license: make bp-modification a method 2016-10-17 21:43:45 -04:00
Jimmy Zelinskie
6eb26d7998 configproviders: pass filemode when opening volume 2016-10-17 21:43:45 -04:00
Jimmy Zelinskie
0c5400b7d1 enforce license across registry blueprints 2016-10-17 21:43:45 -04:00
Joseph Schorr
8fe29c5b89 Add license upload step to the setup flow
Fixes #853
2016-10-17 21:43:15 -04:00
Joseph Schorr
5211c407ff Add license checking to Quay
Based off of mjibson's changes

Fixes #499
2016-10-17 21:43:15 -04:00
josephschorr
78f87d96bc Merge pull request #1986 from coreos-inc/external-tls
Add option to properly handle external TLS
2016-10-15 16:05:28 -04:00
Jake Moshenko
f04b018805 Write our users to Marketo as leads. 2016-10-14 16:29:11 -04:00
Jake Moshenko
013e27f7d5 Clean up mixpanel analytics a bit. 2016-10-13 15:03:04 -04:00
Joseph Schorr
5a8200f17a Add option to properly handle external TLS
Fixes #1984
2016-10-13 14:49:29 -04:00
Joseph Schorr
6ea51afa66 Add a configurable prometheus namespace for all metrics
Fixes #1918
2016-10-05 10:33:35 +03:00
josephschorr
684ace3b5a Merge pull request #1761 from coreos-inc/nginx-direct-download
Add feature flag to force all direct download URLs to be proxied
2016-09-29 22:46:57 +02:00
Evan Cordell
832ee89923 Add duration metric collector decorator (#1885)
Track time-to-start for builders
Track time-to-build for builders
Track ec2 builder fallbacks
Track build time
2016-09-29 15:44:06 -04:00
Joseph Schorr
6ae3faf7fc Add explicit config parameter to the JWT auth methods 2016-09-29 11:15:20 +02:00
Joseph Schorr
dd2e086a20 Add feature flag to force all direct download URLs to be proxied
Fixes #1667
2016-09-29 11:13:41 +02:00
Jimmy Zelinskie
fc7301be0d *: fix legacy imports
This change reorganizes imports and renames the legacy flask extensions.
2016-09-28 20:17:14 -04:00
Jimmy Zelinskie
ae16d24fd1 license: validate via key instance rather than PEM 2016-09-28 15:44:28 -04:00
josephschorr
e1771abe58 Merge pull request #739 from coreos-inc/license
Add license checking to Quay
2016-09-27 16:52:08 +02:00
Joseph Schorr
476576bb70 Add license checking to Quay
Based off of mjibson's changes

Fixes #499
2016-09-27 10:31:34 +02:00
Joseph Schorr
3c8b87e086 Fix verbs in manifestlist
All registry_tests now pass
2016-09-26 14:49:58 -04:00
Jimmy Zelinskie
59529569dc reorder imports 2016-09-26 14:48:05 -04:00
josephschorr
ad4efba802 Merge pull request #1830 from coreos-inc/superuser-dashboard
Add prometheus stats to enable better dashboarding
2016-09-26 17:19:22 +02:00
Joseph Schorr
25ed99f9ef Add feature flag to turn off requirement for team invitations
Fixes #1804
2016-09-20 16:45:00 -04:00
Joseph Schorr
c7beea2032 Fix handling of custom LDAP cert
This change moves the LDAP cert installation into a common script and reorganizes the startup scripts for creating and installing these certs

Fixes #1846
2016-09-19 17:55:08 -04:00
Joseph Schorr
1571b2867a Add executor name to the build metric 2016-09-16 16:26:04 -04:00
Joseph Schorr
30af8aef1a Add a worker for reporting global stats to Prometheus
Fixes #1789
2016-09-12 16:19:19 -04:00
Joseph Schorr
818ea38dac Add repo-specific reporting of repository builds 2016-09-09 15:36:54 -04:00
Joseph Schorr
c8a1b8abab Add prom stats for repository push, pull and verb actions 2016-09-09 15:13:58 -04:00
Jake Moshenko
1d8b72235a Add a helper method to Image to parse ancestor string. 2016-09-07 10:48:58 -04:00
josephschorr
480d890442 Merge pull request #1771 from coreos-inc/kubernetes-save-error
Make sure the Quay Enterprise Kubernetes namespace exists
2016-08-30 12:59:00 -04:00
Joseph Schorr
3f9c82462f Make sure the Quay Enterprise Kubernetes namespace exists
Prevents config from failing to save. Also clarifies any other errors that do occur.

Fixes #1449
2016-08-30 12:58:39 -04:00
Joseph Schorr
aa7c87d765 Fix locking via RedLock
Fixes #1777
2016-08-29 16:06:26 -04:00
Joseph Schorr
608ffd9663 Basic labels support
Adds basic labels support to the registry code (V2), and the API. Note that this does not yet add any UI related support.
2016-08-26 15:24:26 -04:00
Joseph Schorr
193040a473 Fix tag links
Fixes #1741
2016-08-17 15:06:10 -04:00
Joseph Schorr
afc2705b1c Have email read the enterprise logo 2016-08-09 12:18:35 -04:00
Ben Spoon
b0e34692cf Merge pull request #1674 from coreos-inc/new-quay-emails
New quay emails
2016-08-09 09:12:54 -07:00
Ben Spoon
2b92fded68 emails: address review feedback 2016-08-08 13:29:47 -07:00
Ben Spoon
004b834c72 emails: only show quay footer if coming from hosted 2016-08-04 11:55:55 -07:00
Ben Spoon
46a720285a emails: update payment failure admin link
addresses issue #1623
2016-08-04 11:55:50 -07:00
Ben Spoon
5019ef0b6b emails: change the app_link_handler to return just a uri
There is no need for an anchor tag any longer.
2016-08-04 11:55:48 -07:00
Joseph Schorr
770ac0016e Change validate method to work for all storages 2016-08-02 15:01:37 -04:00
Joseph Schorr
0fe3e6510a Prevent invalid tags on builds
Fixes #1632
2016-07-25 17:50:35 -07:00
Joseph Schorr
541764d87b Fix get_priority_for_index method for non-int values
Fixes #1607
2016-07-11 15:04:50 -04:00
Joseph Schorr
a1009af61c Move aggregator into its own repo and add it to the image 2016-07-05 15:39:51 -04:00
Joseph Schorr
713ba3abaf Further updates to the Prometheus client code 2016-07-01 14:16:51 -04:00
Jake Moshenko
668a8edc50 Refactor prometheus integration
Move prometheus to SaaS and make it a plugin
Move static callers to use metrics_queue plugin
Change local-docker to support different quay clone dirnames
Change prom_aggregator to use logrus
2016-07-01 14:16:50 -04:00
Matt Jibson
3d9acf2fff Use prometheus as a metric backend
This entails writing a metric aggregation program since each worker has its
own memory, and thus own metrics because of python gunicorn. The python
client is a simple wrapper that makes web requests to it.
2016-07-01 14:16:50 -04:00
Joseph Schorr
9558c0e937 Fix handling of Github API paths and add tests 2016-06-30 14:10:22 -04:00
Joseph Schorr
ab1756306b Switch to using the leeway parameter on JWT validation 2016-06-27 14:42:44 -04:00
Joseph Schorr
2983195a4a Fix OAuth key not found error for Dex
Fixes #1582
2016-06-27 13:38:11 -04:00
Joseph Schorr
2653d213c9 Add an allowed amount of clock skew to registry JWTs 2016-06-24 15:08:26 -04:00
Joseph Schorr
30ede029d5 Fix GeneratorFile for working with BufferedReader
The user files system uses a BufferedReader along with the magic library to determine the mime type of the user file being served. Currently, BufferedReader fails with an exception on Swift storage, because Swift storage returns a GeneratorFile, which is missing the `readable()` method.
2016-06-23 13:40:57 -04:00
josephschorr
7173d53030 Merge pull request #1549 from coreos-inc/certs
Switch to install custom LDAP cert by name
2016-06-21 15:13:44 -04:00
Joseph Schorr
66ec1d81ce Switch to install custom LDAP cert by name 2016-06-21 15:10:26 -04:00
josephschorr
9e6a264f5f Merge pull request #1523 from coreos-inc/verb-tag-cache-fix
Add a uniqueness hash to derived image storage to break caching over …
2016-06-20 16:38:25 -04:00
Joseph Schorr
a43b741f1b Add a uniqueness hash to derived image storage to break caching over tags
This allows converted ACIs and squashed images to be unique based on the specified tag.

Fixes #92
2016-06-20 16:34:52 -04:00
Jake Moshenko
22562b0156 Merge pull request #1559 from jakedt/finishthejob
Finish removing the AJAX indexing support.
2016-06-20 13:42:05 -04:00
Joseph Schorr
986d20bcad Switch to generic RedisError
Fixes #1558
2016-06-20 11:20:17 -04:00
Jake Moshenko
4130054ef3 Finish removing the AJAX indexing support. 2016-06-20 10:15:21 -04:00
Jake Moshenko
746728ba24 Remove escaped_fragment snapshot rendering. 2016-06-14 12:53:10 -04:00
josephschorr
58bef472d9 Merge pull request #1526 from coreos-inc/superuser-grant
Add ability for super users to take ownership of namespaces
2016-06-13 16:23:10 -04:00
Joseph Schorr
20816804e5 Add ability for super users to take ownership of namespaces
Fixes #1395
2016-06-13 16:22:52 -04:00
Jimmy Zelinskie
f15e5483e7 fix identation according to lint 2016-06-08 15:55:47 -04:00
Jimmy Zelinskie
9fb8b585b5 fix broken import 2016-06-08 15:55:29 -04:00
Joseph Schorr
71b2853f40 Make sure to iterate over a copy of the public_keys dictionary 2016-06-07 18:20:42 -04:00
Joseph Schorr
8887f09ba8 Use the instance service key for registry JWT signing 2016-06-07 11:58:10 -04:00
josephschorr
cad8746f9d Merge pull request #1502 from coreos-inc/image-replication
Enable storage replication for V2 and add backfill tool
2016-06-02 15:02:53 -04:00
Joseph Schorr
12924784ce Enable storage replication for V2 and add backfill tool
Fixes #1501
2016-06-02 14:36:08 -04:00
Jimmy Zelinskie
2317938bfa Merge pull request #1496 from jzelinskie/ripRMS
dockerfile: add check for GPL pip packages
2016-06-02 12:28:18 -04:00
Jimmy Zelinskie
8810157586 remove GPL'd timeparse library 2016-06-02 12:27:49 -04:00
Joseph Schorr
c61c3db728 Remove unused safetar file 2016-05-31 16:50:16 -04:00
Joseph Schorr
4ec3a6c231 Make ACI generation consistent across calls
This will ensure that no matter which signature we write for the generated ACI, it is correct for that image.
2016-05-26 17:09:19 -04:00
Joseph Schorr
f02d295dd8 Fix missing argument change 2016-05-23 17:44:22 -04:00
Joseph Schorr
f670c4c7a9 Change Signer to use the config provider and fix tests
Fixes the broken ACI tests
2016-05-23 17:10:03 -04:00
Jimmy Zelinskie
5568cc77b8 remove all default keys (#1485)
This change:
- Generates a new BitTorrent pepper by default
- Generates a new pagination key by default
- Changes the pagination key format to base64
- Removes selfsigned JWT certs
- Moves test keys to test/data
2016-05-23 16:00:48 -04:00
Jake Moshenko
4266ae7ce5 Fix the x5c header in our registry jwts. 2016-05-23 15:05:54 -04:00
Joseph Schorr
64fe11a5f1 Add ACI signing tests 2016-05-13 18:29:57 -04:00
josephschorr
d572a45a57 Merge pull request #1441 from coreos-inc/fastesttests
Make security scan testing much faster
2016-05-05 13:57:05 -04:00
Joseph Schorr
343a080833 Make security scan testing much faster 2016-05-05 13:55:24 -04:00
Jake Moshenko
75f5df6369 Add clair auth header in generalized interface 2016-05-05 13:28:06 -04:00
Joseph Schorr
232fa42897 Add testing of the new secscan-for-local endpoint and fix a bug 2016-05-04 21:47:03 -04:00
Jake Moshenko
9221a515de Use the registry API for security scanning
when the storage engine doesn't support direct download url
2016-05-04 18:04:06 -04:00
Joseph Schorr
73fa593d02 Various small fixes in prep for QE release 2016-05-04 15:20:27 -04:00
josephschorr
f55fd2049f Merge pull request #1433 from coreos-inc/ldapoptions
Add additional options for LDAP
2016-05-04 14:06:29 -04:00
Joseph Schorr
42515ed9ec Add additional options for LDAP
Fixes #1420
2016-05-04 13:59:20 -04:00
Joseph Schorr
2cbdecb043 Implement setup tool support for Clair
Fixes #1387
2016-05-04 13:40:50 -04:00
Jimmy Zelinskie
437ec84c9f torrent: use quay.pem to mint JWT (#1425) 2016-05-02 18:10:16 -04:00
Evan Cordell
af4106e5c0 Fix generatepresharedkey script 2016-04-29 15:21:19 -05:00
Evan Cordell
2242c6773d Add 'Automatic' ServiceKeyApprovalType 2016-04-29 14:10:33 -04:00
Evan Cordell
c766727d1d address review comments
- more inline documentation
 - don't explicitly specify audience
 - approver is optional in `generate_key`
 - ADD -> RUN for better caching of jwtproxy
2016-04-29 14:10:33 -04:00
Evan Cordell
0c2ecec9a9 Don't check for client certs when talking to clair 2016-04-29 14:10:33 -04:00
Evan Cordell
9ffc32f680 Generate preshared key on boot 2016-04-29 14:10:33 -04:00
Evan Cordell
f30a9e56f3 Be really sure about proxy protocol 2016-04-29 14:10:33 -04:00
Evan Cordell
8595140f38 Use signer proxy for all http(s) requests 2016-04-29 14:10:33 -04:00
Evan Cordell
f4d2fae5d8 Separate jwtproxy signer config from secscan config 2016-04-29 14:10:33 -04:00
Evan Cordell
474884acd7 Don't require certs for clair anymore 2016-04-29 14:10:33 -04:00
Evan Cordell
e499c4a8ef Actually go through signer proxy 2016-04-29 14:10:33 -04:00
Evan Cordell
9e7a501dae Authenticate in the other direction with jwtproxy 2016-04-29 14:10:33 -04:00
Joseph Schorr
dc9bcec9ce Add pre shared generation tool 2016-04-29 14:09:37 -04:00
Jimmy Zelinskie
6577ac3e62 mv JWK-canonicalization util.security.fingerprint 2016-04-29 14:05:16 -04:00
Joseph Schorr
11ff3e9b59 keys ui WIP 2016-04-29 14:05:16 -04:00
Jimmy Zelinskie
97ae800e6c canonicalize json 2016-04-29 13:38:25 -04:00
josephschorr
d63ec8c6b0 Merge pull request #1402 from coreos-inc/clairbugfixes
Fix handling of Clair notifications without `New` block
2016-04-22 15:11:51 -04:00
Joseph Schorr
34a8090328 Fix handling of Defcon 1
Fixes #1397
2016-04-22 13:21:35 -04:00
Joseph Schorr
3f8d51ebd7 Fix handling of Clair notifications without New block
Fixes #1398
2016-04-22 13:05:34 -04:00
josephschorr
affb600423 Merge pull request #1328 from coreos-inc/queuefilefix
Fix QueueFile to support read-to-end semantics and add some tests
2016-04-08 18:07:06 -04:00
Jake Moshenko
45e7c94586 Initialize the db for fixsequences 2016-04-01 14:26:19 -04:00
Jake Moshenko
bd5b44cbd2 Move the sequence fixer to a separate tool which can be run 2016-04-01 13:46:13 -04:00
josephschorr
b9f47f6761 Merge pull request #1285 from coreos-inc/configmaildefaults
Fix mail and signing defaults
2016-03-31 12:31:26 -04:00
Joseph Schorr
6251e63e0e Fix QueueFile to support read-to-end semantics and add some tests 2016-03-31 12:06:49 -04:00
Joseph Schorr
0e84a94146 Make analyzer handle images without features or vulnerabilities 2016-03-29 15:16:22 -04:00
Joseph Schorr
dc8f9713f8 Change logs worker to use a global lock in the inner loop and move storage out of the transaction 2016-03-24 14:09:48 -04:00
Joseph Schorr
aa5587c93c Fixes and added tests for the security notification worker
Fixes #1301

- Ensures that the worker uses pagination properly
- Ensures that the worker handles failure as expected
- Moves marking the notification as read to after the worker processes it
- Increases the number of layers requested to 100
2016-03-18 20:28:06 -04:00
Jimmy Zelinskie
8af0b887ef fix broken tests 2016-03-18 15:48:41 -04:00
Jimmy Zelinskie
5094e1f712 move slash_join to prevent local imports 2016-03-18 15:09:25 -04:00
Jimmy Zelinskie
e5d8a431f4 replace use of URL joining with slash_join 2016-03-18 14:56:10 -04:00
Jimmy Zelinskie
bf477b6b9c add slash_join helper and tests 2016-03-18 14:56:10 -04:00
Jimmy Zelinskie
0dcfcebe34 remove unused imports and lint 2016-03-18 14:56:09 -04:00
Jimmy Zelinskie
bcea268fcb use app.gitlab_trigger for config data
This includes defaults and makes the structure of the Gitlab trigger
parallel the GitHub trigger.
2016-03-18 14:56:09 -04:00
Quentin Machu
d093a7bde5 Merge pull request #1290 from Quentin-M/split_clair_clusters
Split clair clusters
2016-03-15 11:09:51 -04:00
Quentin Machu
81fe315171 Add ability to use another Clair stack for batch tasks 2016-03-14 14:28:34 -04:00
Joseph Schorr
821b09daaf Update Quay Sec UI as per feedback from design team
Fixes #1281
2016-03-10 14:49:36 -05:00
Joseph Schorr
8e1727b6d3 Fix mail and signing defaults 2016-03-08 18:08:40 -05:00
Quentin Machu
897df4de32 Merge pull request #1271 from coreos-inc/allocator_bs
Repair allocator (min/max swapped)
2016-03-04 12:06:04 -05:00
Quentin Machu
d36528a77a Increase POST timeout in secscan API 2016-03-04 11:59:00 -05:00
Quentin Machu
4f7a66ab0e Repair secscan's analyze_layer API call 2016-03-02 16:05:11 -05:00
Quentin Machu
c8bf55c2bb Repair allocator (min/max swapped) 2016-03-02 14:51:54 -05:00
Quentin Machu
c29ce8e1a1 Merge pull request #1268 from Quentin-M/secnotif_feature_flag
Use a feature flag to toggle security notifications
2016-03-01 15:54:37 -05:00
Quentin Machu
888f976e8d Use a feature flag to toggle security notifications 2016-03-01 15:54:18 -05:00
Quentin Machu
ea013b8066 make min_index optionnal in allocator's constructor 2016-03-01 14:54:38 -05:00
Quentin Machu
672168ce78 Close Clair API connections
This forces every API calls to be load-balanced properly.
2016-02-29 14:52:38 -05:00
Joseph Schorr
ae9140caae Implement new vulnerabilities and packages tabs.
Fixes https://github.com/coreos-inc/design/issues/268
2016-02-25 17:09:29 -05:00
Joseph Schorr
f498e92d58 Implement against new Clair paginated notification system 2016-02-25 15:58:42 -05:00
Joseph Schorr
c0374d71c9 Refactor the security worker and API calls and add a bunch of tests 2016-02-25 12:29:41 -05:00
Quentin Machu
0183c519f7 Merge pull request #1253 from Quentin-M/clair2
Adapt securityworker, secscan API and Quay UI for Clair 1.0
2016-02-19 18:21:25 -05:00
josephschorr
11af123ba5 Merge pull request #1244 from coreos-inc/enableaci
Add UI to the setup tool for enabling ACI conversion
2016-02-17 12:29:48 -05:00
Joseph Schorr
1940fd9939 Add UI to the setup tool for enabling ACI conversion
Fixes #1211
2016-02-17 12:05:48 -05:00
josephschorr
6f9fc7fc08 Merge pull request #1225 from coreos-inc/setuptooltest
Add tests for superuser config API calls
2016-02-16 17:01:43 -05:00
josephschorr
81a36ee3b8 Merge pull request #1217 from coreos-inc/v2pagination
Fix V2 catalog and tag pagination
2016-02-16 15:34:49 -05:00
Quentin Machu
c8d825c232 expose min_id in allocator.py 2016-02-16 15:16:22 -05:00
Jake Moshenko
88d84aa182 Fixes for content checksum and torrent pieces backfill
Remove null handler from app.py, was silencing other logs
2016-02-11 16:53:18 -05:00
Joseph Schorr
03533db5a3 Add tests for superuser config API calls 2016-02-11 11:04:37 +02:00
Joseph Schorr
db0eab0461 Fix V2 catalog and tag pagination 2016-02-10 00:25:33 +02:00
Jimmy Zelinskie
5828d8e716 private swarms torrents 2016-02-08 13:56:31 -05:00
Joseph Schorr
1536709c02 Small fixes 2016-01-29 20:01:17 +02:00
Jake Moshenko
01a92a66ba Refresh base image and python dependencies 2016-01-27 11:36:40 -05:00
Joseph Schorr
335c8eb3a9 Add 2 day TTL to page tokens 2016-01-26 14:04:03 -05:00
Joseph Schorr
b4bddacedb Switch to Fernet crypto as per gtank's recommendation 2016-01-26 12:50:48 -05:00
Jimmy Zelinskie
85ae1a2a0a Merge pull request #1161 from jzelinskie/torrenthmac
misc torrent changes
2016-01-22 23:02:44 -05:00
Joseph Schorr
e4ffaff869 Fix Docker Auth and our V2 registry paths to support library (i.e. namespace-less) repositories.
This support is placed behind a feature flag.
2016-01-22 15:54:06 -05:00
Jimmy Zelinskie
2650772db3 add delimiters to per-user torrent filenames 2016-01-22 15:53:21 -05:00
Jimmy Zelinskie
e54b86c6eb s/TORRENT/BITTORRENT 2016-01-22 15:52:28 -05:00
Joseph Schorr
7c572fd218 Add support for torrenting verbs
Fixes #1130
2016-01-20 18:15:32 -05:00
Jake Moshenko
aaf462682f Fix the allocator to use id ranges instead of limits 2016-01-12 15:21:13 -05:00
Jake Moshenko
1ae101c917 Address torrent feature review comments. 2016-01-08 16:38:21 -05:00
Jimmy Zelinskie
932d892276 torrent: remove pubkey token header 2016-01-08 14:29:24 -05:00
Joseph Schorr
9d966c2605 Backport V1 metadata fix 2016-01-08 13:53:04 -05:00
Jake Moshenko
073b68cf0d Fix torrent migration and update backfill to compute torrent pieces 2016-01-08 11:15:34 -05:00
Jimmy Zelinskie
087c6828ad add feature.BITTORRENT and jwk set URI 2016-01-07 19:07:23 -05:00
Jimmy Zelinskie
f774442a84 torrent: send jwt in announce url 2016-01-07 14:16:21 -05:00
Jake Moshenko
476ac8cec9 Add piece hashing to verbs generated image storages 2016-01-06 12:01:15 -05:00
Jake Moshenko
8f80d7064b Hash v1 uploads for torrent chunks 2016-01-05 14:43:40 -05:00
Jake Moshenko
8d5f4466d6 Cleanup some indentation and imports 2016-01-05 12:12:57 -05:00
Jimmy Zelinskie
fff016d0f5 "created by" now uses REGISTRY_TITLE 2016-01-04 16:17:51 -05:00
Jimmy Zelinskie
a0e5de8f29 add torrent options to config 2016-01-04 16:17:51 -05:00
Jimmy Zelinskie
c780572e69 add public/private torrent swarms 2016-01-04 16:17:51 -05:00
Jimmy Zelinskie
4cb06525a4 finish implementing torrent verb 2016-01-04 16:17:51 -05:00
Jake Moshenko
ce8fcbeaae Update the pieces to use base64 encoded binary 2016-01-04 16:17:51 -05:00
Jake Moshenko
fe87d3c796 Hash and track layer file chunks for torrenting 2016-01-04 16:17:51 -05:00
josephschorr
28eb31ed36 Merge pull request #1102 from coreos-inc/deleteimagediff
Delete the image diff feature
2015-12-29 14:47:38 -05:00
Joseph Schorr
31a8a0fba4 Better UX when recovering organization emails
Fixes #291
2015-12-28 15:25:31 -05:00
Joseph Schorr
ab166c4448 Delete the image diff feature
Fixes #1077
2015-12-23 13:08:01 -05:00
Joseph Schorr
63a8b197e4 Break out 5XX errors into their own metric
First part of #983
2015-12-16 13:56:07 -05:00
Jake Moshenko
766d60493f Add the ability to blacklist v2 for specific versions 2015-12-15 18:27:10 -05:00
Joseph Schorr
54095eb5cb Handle the common case of one chunk when calculating the uncompressed size
Reference #992
2015-12-14 15:27:48 -05:00
Jake Moshenko
7205bf5e7f Merge pull request #885 from jakedt/python-registry-v2
Python registry v2 mega merge
2015-11-16 16:15:40 -05:00
Jake Moshenko
0459c3bc54 Merge remote-tracking branch 'upstream/master' into python-registry-v2 2015-11-16 14:22:54 -05:00
Matt Jibson
01fe548abd Use env vars to set k8s endpoint URL
The old DNS method is optionally enabled in k8s, but the env vars are
always there.

partial solution to #864
2015-11-13 17:05:14 -05:00
Matt Jibson
2e1b49b009 Allow None for max_id during migrations
This allows empty databases with no max_id to run.

fixes #869
2015-11-13 15:41:39 -05:00
Joseph Schorr
46745ee30f Remove file added accidentally by merge 2015-11-12 22:07:47 -05:00
Joseph Schorr
7816b0c657 Merge master into vulnerability-tool 2015-11-12 21:52:47 -05:00
Joseph Schorr
25b8b7590f Fix all the things! 2015-11-12 20:55:41 -05:00
Jimmy Zelinskie
37ce84f6af tiny fixes to securityworker 2015-11-12 17:18:04 -05:00
Jimmy Zelinskie
e86a342868 create class for security config validation 2015-11-12 15:47:01 -05:00
Jake Moshenko
ab340e20ea Merge remote-tracking branch 'upstream/master' into python-registry-v2 2015-11-11 16:41:40 -05:00
Jake Moshenko
88bbf34993 Silence a lot of the useless logs for the checksum backfill 2015-11-10 19:49:23 -05:00
Jake Moshenko
83c98882bb Fix the backfill batch message to report the number 2015-11-10 19:49:00 -05:00
Jake Moshenko
941d13ea3e Fix an off by one error in the common backfill code 2015-11-10 16:14:44 -05:00
Joseph Schorr
ca7d736db2 Only send vulnerability events if the minimum priority is gte to that specified
Fixes #770
2015-11-10 16:05:55 -05:00
Jimmy Zelinskie
8e2868737b rename secscan_endpoint and move db close to API 2015-11-10 15:22:31 -05:00
Jimmy Zelinskie
112bef8f8c fix bug where v1 backfill never completed 2015-11-10 14:04:20 -05:00
Jake Moshenko
a33077b978 Optimistically update backfill items, reducing RTs 2015-11-10 11:10:09 -05:00
Jake Moshenko
dc24e8b1a1 Backfill by allocating and selecting ids in random blocks
Fixes #826
2015-11-09 22:29:17 -05:00
Silas Sewell
e826b14ca4 Merge pull request #725 from coreos-inc/setup-tool-georeplication
superuser: add storage replication config
2015-11-09 17:43:38 -05:00
Silas Sewell
5000b1621c superuser: add storage replication config 2015-11-09 17:34:22 -05:00
Joseph Schorr
a69c9e12fd Update quay sec code to fix problems identified in previous review
- Change get_repository_images_recursive to operate over a single docker image and storage uuid
- Move endpoints/sec to endpoints/secscan
- Change notification system to work with new Quay-sec format

Fixes #768
2015-11-09 17:14:35 -05:00
Joseph Schorr
2d2662f53f Fix deleting repos and images under MySQL
MySQL doesn't handle constraints at the end of transactions, so deleting images currently fails. This removes the constraint and just leaves parent_id as an int
2015-11-09 14:42:05 -05:00
Joseph Schorr
fb3d0fa27d Add a SecEndpoint class and move all the cert and config handling in there 2015-11-09 12:49:19 -05:00
Quentin Machu
37118423a5 Add support for Quay's vulnerability tool 2015-11-09 12:49:19 -05:00
Jake Moshenko
c2fcf8bead Merge remote-tracking branch 'upstream/phase4-11-07-2015' into python-registry-v2 2015-11-06 18:18:29 -05:00
Jake Moshenko
79c89ba11d Re-enable parent id backfill, use new backfill style 2015-11-06 15:45:39 -05:00
Jake Moshenko
88b9e80cbb Backfill the v1 checksums from imagestorage 2015-11-06 15:28:44 -05:00
Jimmy Zelinskie
f3c3e684a1 prepare branch to be merged into phase1-11-07-2015
This removes the checksum backfill, removes the migration that runs the
backfills, and defaults the security scan feature off.
2015-11-06 15:22:18 -05:00
Joseph Schorr
cfa03951e1 Add a SecScanEndpoint class and move all the cert and config handling in there 2015-11-06 15:22:18 -05:00
Joseph Schorr
0f3db709ea Add a vulnerability_found event for notice when we detect a vuln
Fixes #637

Note: This PR does *not* actually raise the event; it merely adds support for it
2015-11-06 15:22:18 -05:00
Quentin Machu
f59e35cc81 Add support for Quay's vulnerability tool 2015-11-06 15:22:18 -05:00
Joseph Schorr
bbf4a1fac4 Remove the used_legacy_github column 2015-11-06 15:17:55 -05:00
Joseph Schorr
6bc5c78241 Later migration changed one of the tables, so make local copies 2015-11-03 11:18:42 -05:00
josephschorr
45bfe7dafc Merge pull request #747 from coreos-inc/rebrand
Rebrand Quay
2015-11-02 15:46:59 -05:00
Jimmy Zelinskie
c78c450211 UTF-8 v1_json_metadata, comment, manifest
This will allow us to store unicode JSON blobs in the column on MySQL.
2015-11-02 15:40:19 -05:00
Joseph Schorr
f6a53f7cc5 Change all Quay.io references to Quay, fix tour and change logo
Fixes #741
2015-11-02 14:37:48 -05:00
Jake Moshenko
2c10d28afc Merge remote-tracking branch 'upstream/master' into python-registry-v2 2015-10-26 14:44:16 -04:00
Jake Moshenko
9da64f3aba Stop writing to deprecated columns for image data. 2015-10-24 14:45:15 -04:00
Jimmy Zelinskie
e973289397 Revert "Revert "Merge pull request #682 from jzelinskie/revertrevert""
This reverts commit 278bc736e3.
2015-10-23 15:26:33 -04:00
Joseph Schorr
05262125a0 Make the namespace and secret name configurable via env var for the k8s provider
Fixes #695
2015-10-23 12:18:11 -04:00
Jake Moshenko
e7a6176594 Merge remote-tracking branch 'upstream/v2-phase4' into python-registry-v2 2015-10-22 16:59:28 -04:00
Jimmy Zelinskie
278bc736e3 Revert "Merge pull request #682 from jzelinskie/revertrevert"
This reverts commit 627ad25c9c, reversing
changes made to 31c392fecc.
2015-10-22 16:02:07 -04:00
Jake Moshenko
ce94931540 Stop writing to deprecated columns for image data. 2015-10-22 12:14:39 -04:00
josephschorr
ad53bf5671 Merge pull request #644 from coreos-inc/namechoose
Docker changed their namespace regex, so we need to adjust
2015-10-22 12:07:52 -04:00
Joseph Schorr
a8aa6d1939 Docker changed their namespace regex, so we need to adjust
Fixes #617
2015-10-22 12:07:31 -04:00
Jimmy Zelinskie
67497bb99c write None if we cannot find the json 2015-10-21 16:26:30 -04:00
Jimmy Zelinskie
39cfe77d42 Revert "Merge pull request #557 from coreos-inc/revert-migration"
This reverts commit c4f938898a, reversing
changes made to 7ad2522dbe.
2015-10-21 15:29:57 -04:00
Silas Sewell
dd3d939b31 Update tag validation
Fixes #536
2015-10-05 19:32:10 -04:00
Joseph Schorr
f393236c9f Add repo name check to V2
Fixes #592
2015-10-05 14:19:52 -04:00
Jimmy Zelinskie
ffeb99d4ee BaseStreamFileLike: handle reads that return None
Fixes #555.
2015-09-30 17:46:59 -04:00
Joseph Schorr
a3ebb9028d Add full unit tests for the file-like objects and fix them
Fixes #568
2015-09-30 14:19:25 -04:00
josephschorr
41bfe2ffde Merge pull request #551 from coreos-inc/python-registry-v2-swift
Add V2 storage methods to Swift storage engine
2015-09-28 17:09:34 -04:00
Joseph Schorr
6c59161527 Add V2 storage methods to Swift storage engine
Fixes #508
2015-09-28 16:46:19 -04:00
Silas Sewell
9000169b53 Revert "Merge pull request #491 from jakedt/migratebackp2"
This reverts commit 7ad2522dbe, reversing
changes made to a0b191ffa1.
2015-09-28 16:09:22 -04:00
Jimmy Zelinskie
c5aa3ca4f0 make registry v2 tests pass for GCS
Fixes #509.
2015-09-28 15:42:48 -04:00
josephschorr
7ad2522dbe Merge pull request #491 from jakedt/migratebackp2
Migrate image data back phase 2
2015-09-26 15:11:46 -04:00
Joseph Schorr
a283c8d8ec Add a check to ensure repository names are valid according to an extended set of rules.
Fixes #534
2015-09-24 11:55:08 -04:00
Joseph Schorr
40f3b7137d Fix dict wrapper access to not raise an exception 2015-09-22 14:18:37 -04:00
Joseph Schorr
bf578420f0 Fix import of Github migration 2015-09-21 16:52:56 -04:00
Joseph Schorr
49b575afb6 Start refactoring of the trigger system:
- Move each trigger handler into its own file
- Add dictionary helper classes for easier reading and writing of dict-based data
- Extract the web hook payload -> internal representation building for each trigger system
- Add tests for this transformation
- Remove support for Github archived-based building
2015-09-21 16:36:48 -04:00
Joseph Schorr
1c6933a28d Fix Github build trigger migration 2015-09-19 14:34:46 -04:00
Jake Moshenko
26cea9a07c Merge remote-tracking branch 'upstream/master' into python-registry-v2 2015-09-17 16:16:27 -04:00
Jake Moshenko
a887125c3f Fixes for backfill_aggregate_size script. 2015-09-17 15:47:18 -04:00
Jake Moshenko
8baacd2741 Migrate old data to new locations, read only new. 2015-09-17 15:47:13 -04:00
Joseph Schorr
eff9ff7a66 Migrate all GitHub build triggers to use deploy keys 2015-09-16 17:55:51 -04:00
Joseph Schorr
6f2271d0ae Add support for direct download in Swift storage engine
Fixes #483
2015-09-14 18:00:03 -04:00
josephschorr
57329b6c78 Merge pull request #475 from coreos-inc/seofix
Use a proper HTML parser with BS and catch exceptions
2015-09-14 15:56:03 -04:00
Joseph Schorr
6ca33ca108 Use a proper HTML parser with BS and catch exceptions
Fixes #473
2015-09-10 16:14:29 -04:00
Jake Moshenko
9c3ddf846f Some fixes and tests for v2 auth
Fixes #395
2015-09-10 15:38:57 -04:00
Joseph Schorr
fd3a21fba9 Add Kubernetes configuration provider which writes config to a secret
Fixes #145
2015-09-10 12:19:59 -04:00
Joseph Schorr
88a04441de Extract the config provider into its own sub-module 2015-09-10 12:19:59 -04:00
Joseph Schorr
c2fe751d15 Despite being disabled, OAuth config is still read, so switch to .get 2015-09-10 12:09:01 -04:00
Joseph Schorr
c0286d1ac3 Add support for Dex to Quay
Fixes #306

- Adds support for Dex as an OAuth external login provider
- Adds support for OIDC in general
- Extract out external logins on the JS side into a service
- Add a feature flag for disabling direct login
- Add support for directing to the single external login service
- Does *not* yet support the config in the superuser tool
2015-09-04 17:05:06 -04:00
Jake Moshenko
210ed7cf02 Merge remote-tracking branch 'upstream/master' into python-registry-v2 2015-09-04 16:32:01 -04:00
Jake Moshenko
82efc746b3 Make our JWT checking more strict. 2015-09-04 15:18:57 -04:00
Jake Moshenko
8269d4ac90 Checkpoint implementing PATCH according to Docker 2015-09-03 16:26:02 -04:00
Joseph Schorr
b7f487da42 Build the OAuth redirect URL ourselves, rather than relying on undocumented Flask behavior 2015-09-02 13:32:11 -04:00
josephschorr
c693afca6a Merge pull request #426 from coreos-inc/unicodefix
Fix Dockerfile parsing for unicode and add testing
2015-08-31 15:03:01 -04:00
Joseph Schorr
fb86b4bf2c Fix Dockerfile parsing for unicode and add testing
Fixes #423
2015-08-31 14:32:26 -04:00
josephschorr
adc66a2894 Merge pull request #422 from coreos-inc/logsgzipfix
Change build logs load to using streaming Gzip
2015-08-31 12:15:30 -04:00
Joseph Schorr
c0c1da3232 Change build logs load to using streaming Gzip 2015-08-28 14:08:13 -04:00
Joseph Schorr
43e77a7a14 Add missing tell() method to GeneratorFile and add tests 2015-08-28 12:10:03 -04:00
Matt Jibson
4aa5ab88dd Use real cloudwatch limit
Although cloudwatch allows 40KB of data, it may be from no more than 20
different metrics. Until we can do that, limit the total points to 20.
2015-08-26 16:48:48 -04:00
Joseph Schorr
84458811d5 Rename wrap_with_hash to a more generic wrap_with_handler 2015-08-25 15:53:13 -04:00
Joseph Schorr
c07dec4d39 File reader fixes for verbs
- Fix local file reader to always read in chunks
- Have gzip stream raise an exception if the full data is requested
2015-08-25 13:49:21 -04:00
Joseph Schorr
84276ee945 Better notifications UI
Fixes #369
2015-08-17 17:08:58 -04:00
Joseph Schorr
4625ecf273 Fix tests in response to breakage in #351 2015-08-17 16:26:20 -04:00
Matt Jibson
9a7e5bb35e Batch cloudwatch puts 2015-08-17 12:03:49 -04:00
Jake Moshenko
e1b3e9e6ae Another huge batch of registry v2 changes
Add patch support and resumeable sha
Implement all actual registry methods
Add a simple database generation option
2015-08-12 16:41:12 -04:00
Matt Jibson
7c3b555ee9 Code review 2015-08-12 16:31:01 -04:00
Matt Jibson
f043bc1379 Don't enable the metric queue if there's no Cloudwatch 2015-08-12 15:14:09 -04:00
Matt Jibson
b483209862 Wrap API and registry requests with common metric timings
Record response times, codes, and rollup non-2XX responses.
2015-08-12 12:16:00 -04:00
Matt Jibson
b04c190ca0 Prevent the metric queue from growing unbounded 2015-08-12 12:16:00 -04:00
Matt Jibson
cfb6e884f2 Refactor metric collection
This change adds a generic queue onto which metrics can be pushed. A
separate module removes metrics from the queue and adds them to Cloudwatch.
Since these are now separate ideas, we can easily change the consumer from
Cloudwatch to anything else.

This change maintains near feature parity (the only change is there is now
just one queue instead of two - not a big deal).
2015-08-12 12:15:52 -04:00
Jake Moshenko
74d838697f Fix tarfile to support non-unicode pax fields 2015-08-07 11:56:38 -04:00
Jake Moshenko
18100be481 Refactor the util directory to use subpackages. 2015-08-03 16:04:19 -04:00
Joseph Schorr
26ae629189 Prevent local storage setup on non-mounted paths
Fixes #269
2015-07-27 14:32:02 -04:00
Joseph Schorr
52d833b3c6 Fix spacing 2015-07-23 16:00:36 -04:00
Joseph Schorr
c3f269ee23 Add migration for BitBucket web hooks
This needs to added only *after* we roll out #255
2015-07-23 14:45:12 -04:00
Joseph Schorr
38a6b3621c Automatically link the superuser account to federated service for auth
When the user commits the configuration, if they have chosen a non-DB auth system, we now auto-link the superuser account to that auth system, to ensure they can login again after restart.
2015-07-22 13:37:23 -04:00
Joseph Schorr
33b54218cc Refactor the users class into their own files, add a common base class for federated users and add a verify_credentials method which only does the verification, without the linking. We use this in the superuser verification pass 2015-07-20 11:39:59 -04:00
Joseph Schorr
066637f496 Basic Keystone Auth support
Note: This has been verified as working by the end customer
2015-07-20 10:55:21 -04:00
Jake Moshenko
bc29561f8f Fix and templatize the logic for external JWT AuthN and registry v2 Auth.
Make it explicit that the registry-v2 stuff is not ready for prime time.
2015-07-17 11:56:15 -04:00
Jake Moshenko
3efaa255e8 Accidental refactor, split out legacy.py into separate sumodules and update all call sites. 2015-07-17 11:56:15 -04:00
Jake Moshenko
bea8b9ac53 More changes for registry-v2 in python.
Implement the minimal changes to the local filesystem storage driver and feed them through the distributed storage driver.
Create a digest package which contains digest_tools and checksums.
Fix the tests to use the new v1 endpoint locations.
Fix repository.delete_instance to properly filter the generated queries to avoid most subquery deletes, but still generate them when not explicitly filtered.
2015-07-17 11:50:41 -04:00
Joseph Schorr
4726559322 The database SSL name needs to be in its own list
FIxes #243
2015-07-16 00:49:07 +03:00
Joseph Schorr
4333bb9e14 Implement stream_read_file for the Swift storage engine
Note that Swift doesn't seem to have a file-like interface, so we need to wrap the generator we get back from it.

Fixes #210
2015-07-02 17:52:43 +03:00
Jimmy Zelinskie
756d6784ca Merge pull request #192 from coreos-inc/sqlssl
Allow SSL cert for the database to be configured
2015-06-29 13:33:31 -04:00
Joseph Schorr
dc5af7496c Allow superusers to disable user accounts 2015-06-29 18:40:52 +03:00
Joseph Schorr
bb07d0965f Allow SSL cert for the database to be configured
This change adds a field for the SSL cert for the database in the setup tool. Fixes #89
2015-06-29 08:08:10 +03:00
Joseph Schorr
07439328a4 Remove user_exists endpoint from all auth systems 2015-06-23 17:33:51 -04:00
Joseph Schorr
331c300893 Refactor JWT auth to not import app locally 2015-06-17 15:53:21 -04:00
Joseph Schorr
e7fa560787 Add support for custom fields in billing invoices
Customers (especially in Europe) need the ability to add Tax IDs, VAT IDs, and other custom fields to their invoices.

Fixes #106
2015-06-12 16:45:01 -04:00
Joseph Schorr
90b4f0a2ed Fix default log archive location for ER
Before this change, the ER was using the default of 'local_us' from the base config, which is incorrect, and caused no logs to be archived.
2015-06-11 13:43:29 -04:00
Joseph Schorr
457ee7306e Parenthesis fix on the JWT auth error message 2015-06-10 16:00:25 -04:00
Jake Moshenko
2a2414d6af Merge pull request #60 from coreos-inc/jwtauthentication
Add support for an external JWT-based authentication system
2015-06-05 13:37:42 -04:00
Joseph Schorr
8aac3fd86e Add support for an external JWT-based authentication system
This authentication system hits two HTTP endpoints to check and verify the existence of users:

Existance endpoint:
GET http://endpoint/ with Authorization: Basic (username:) =>
    Returns 200 if the username/email exists, 4** otherwise

Verification endpoint:
GET http://endpoint/ with Authorization: Basic (username:password) =>
    Returns 200 and a signed JWT with the user's username and email address if the username+password validates, 4** otherwise with the body containing an optional error message

The JWT produced by the endpoint must be issued with an issuer matching that configured in the config.yaml, and the audience must be "quay.io/jwtauthn". The JWT is signed using a private key and then validated on the Quay.io side with the associated public key, found as "jwt-authn.cert" in the conf/stack directory.
2015-06-05 13:20:10 -04:00
Joseph Schorr
c0e995c1d4 Merge branch 'master' into nolurk 2015-06-02 13:55:16 -04:00
Joseph Schorr
dd28a845db Fix NPE in cache control decorator 2015-05-28 13:22:42 -04:00
Joseph Schorr
ac239ec4ee Make sure to only split into two parts max 2015-05-20 14:54:41 -04:00
Joseph Schorr
54992c23b7 Add a feature flag for disabling unauthenticated access to the registry in its entirety. 2015-05-19 17:52:44 -04:00