vbatts/quay
Archived
1
0
Fork 0
Code Issues Pull requests Releases Wiki Activity
8783 commits 2 branches 62 tags 205 MiB
Commit graph

877 commits

Author SHA1 Message Date
josephschorr
7cd2c00d4d
Merge pull request #2967 from coreos-inc/joseph.schorr/QS-111/auth-refactor 
Refactor auth code to be cleaner and more extensible
 2018-02-15 16:02:22 -05:00
Joseph Schorr
e220b50543 Refactor auth code to be cleaner and more extensible 
We move all the auth handling, serialization and deserialization into a new AuthContext interface, and then standardize a registration model for handling of specific auth context types (user, robot, token, etc).
 2018-02-14 15:35:27 -05:00
Brad Ison
87e55870b7 Add script for fixing missing admin permissions 
Adds a util script to find and fix repositories in user namespaces
that are missing admin permissions for the owning user.  These admin
permissions are required, but were missing in some cases.  See:

  https://github.com/coreos-inc/quay/pull/2998
 2018-02-13 16:23:17 -05:00
Joseph Schorr
d45161b120 Add a worker to automatically GC expired app specific tokens 
Fixes https://jira.coreos.com/browse/QUAY-822
 2018-02-12 14:56:01 -05:00
josephschorr
846deb75fe
Merge pull request #2902 from coreos-inc/joseph.schorr/QS-51/azure-blob-store 
Add support for Azure Blob Storage
 2018-02-07 11:34:29 -05:00
Brad Ison
5965929187 Include location in user analytics 2018-02-06 16:06:17 -05:00
Joseph Schorr
5490e64669 Fill out schema and schema whitelist 2018-02-06 15:27:01 -05:00
Joseph Schorr
7893ef6acc Add test to ensure that all config.py properties are defined in the config schema 2018-02-06 15:26:31 -05:00
Joseph Schorr
d488517b36 Implement storage driver for Azure Blob Storage 2018-02-06 13:48:40 -05:00
josephschorr
9f7b08d0ff
Merge pull request #2993 from coreos-inc/joseph.schorr/QUAY-797/pagination-size 
Allow size of pages in V2 api to be configurable
 2018-02-02 15:21:15 -05:00
Joseph Schorr
eae9175950 Allow size of pages in V2 api to be configurable 2018-02-02 13:54:41 -05:00
Joseph Schorr
bbdf9e074c Add metrics for tracking when instance key renewal succeeds and fails, as well as when instance key *lookup* fails 2018-02-02 11:14:42 -05:00
josephschorr
6514bf229f
Merge pull request #2973 from coreos-inc/joseph.schorr/QS-116/cloudfront-storage 
Add support for configuring cloudfront storage
 2018-02-02 10:14:28 -05:00
Joseph Schorr
b0f656731c Add support for configuring CloudFront storage engine 
Fixes https://jira.coreos.com/browse/QS-116
 2018-01-31 11:22:14 -05:00
Joseph Schorr
462500a389 Temp revert dot fix because it applies to repo names as well on pull 2018-01-26 14:43:37 -05:00
IvanCherepov
c228734978
Generates HTML documentation explaining all of configuration fields (#2952) 
* create HTML documentation explaining all of schema's configuration fields
 2018-01-24 14:09:29 -05:00
Joseph Schorr
ede3a81c68 Disallow dots in repository names to fix reflected text "attack" 
Fixes https://jira.coreos.com/browse/QS-125
 2018-01-18 13:19:37 -05:00
Joseph Schorr
524d77f527 Add an AppSpecificAuthToken data model for app-specific auth tokens. These will be used for the Docker CLI in place of username+password 2018-01-04 15:27:41 -05:00
Joseph Schorr
72bfebdb60 Add license validation to the config validation check 
Should prevent a customer from accidentally saving a config that violates their license

Fixes https://jira.coreos.com/browse/QS-97
 2017-12-19 13:44:08 -05:00
IvanCherepov
c383ac1f9d
Add config validation on startup (#2903) 
* WIP

* Finish schema

Add three sections: security scanning, bittorrent support and feature flags.
 2017-12-01 10:46:39 -05:00
Joseph Schorr
c168413a8e Fix bug when running ipresolver under Gitlab CI 
Since the container does contain IP data, this would fail
 2017-11-30 10:23:58 -05:00
Joseph Schorr
400a5db719 Add additional metrics on executor start and failure 
This will allow us to register a pager if one of the executors starts failing consistently
 2017-11-27 11:52:37 +02:00
Ivan Cherapau
a0adc1b0ec Fix typo in metrics 2017-11-14 23:16:25 -05:00
Joseph Schorr
2677720577 Fix exception raised for certain non-JSON strings given to is_json 
This is breaking pushes in production for certain manifests

Fixes https://jira.prod.coreos.systems/browse/QS-60
 2017-11-14 13:46:06 -05:00
Joseph Schorr
74f99ba94a Ensure encrypted passwords are not enabled with OIDC auth 
Fixes https://jira.prod.coreos.systems/browse/QS-49
 2017-10-31 16:03:28 -04:00
Joseph Schorr
8194f5cf72 Switch ipresolver to always be defined in the storage context 
We now use a no-op IP resolver instead of an IF check

Fixes https://jira.prod.coreos.systems/browse/QS-38
 2017-10-17 14:29:40 -04:00
josephschorr
3bef21253d Merge pull request #2695 from coreos-inc/oidc-internal-auth 
OIDC internal auth support
 2017-10-02 16:51:17 -04:00
Joseph Schorr
f51a863158 Remove access_token from user_info 2017-10-02 16:51:09 -04:00
Joseph Schorr
05b4a7d457 Add worker to update ipresolver data files every few hours 2017-09-28 14:40:59 -04:00
Joseph Schorr
52927de7f6 Add resolved IP information to track_and_log 2017-09-28 14:40:58 -04:00
Joseph Schorr
010dda2c52 Add CloudFrontedS3Storage, which redirects to CloudFront for non-S3 ips 2017-09-28 14:40:58 -04:00
Joseph Schorr
2d522764f7 Add IP resolver utility that returns whether an IP is under AWS 2017-09-26 16:11:16 -04:00
josephschorr
c44cc072fa Merge pull request #2864 from coreos-inc/partial-autocomplete 
Partial autocomplete
 2017-09-13 11:26:11 -04:00
Joseph Schorr
54a4476cbb Make missing log more descriptive 2017-09-12 16:19:55 -04:00
Joseph Schorr
c105123ad4 Add superuser config for prefix autocomplete setting 2017-09-12 15:57:57 -04:00
Joseph Schorr
bc82edb2d1 Add ability to configure OIDC internal auth engine via superuser panel 2017-09-12 12:23:52 -04:00
Joseph Schorr
783799c227 Make team sync timeout config actually configurable 2017-09-06 14:08:30 -04:00
Joseph Schorr
751598056e Enable support in OIDC for endpoints without user info support 
The user info endpoint is apparently optional.
 2017-08-01 13:24:27 -04:00
Antoine Legrand
2d60ad71b6 Print only first line of s3 error message 2017-07-27 18:05:06 +02:00
Joseph Schorr
e7dbc4ee91 Move notification helper code into the root module 2017-07-25 17:00:07 -04:00
Joseph Schorr
ce56031846 Move notifications into its own package 2017-07-25 17:00:06 -04:00
Jake Moshenko
3b79955c8c Fix the relative path problem when running quay from non-root 2017-07-13 15:30:50 -04:00
Joseph Schorr
e00437c227 Add support for disabling an entire namespace, including its team members 2017-07-13 12:25:19 +03:00
Joseph Schorr
7910dc4b2a Fix reference error 2017-07-13 12:25:19 +03:00
Joseph Schorr
2814d2d5eb Add support for organizations to disableabuser 2017-07-13 12:25:19 +03:00
josephschorr
96d1fd128d Merge pull request #2757 from coreos-inc/joseph.schorr/QUAY-606/logarchive-georep 
Add support for QE customers to enable log rotation
 2017-07-12 00:30:04 +03:00
Evan Cordell
ac54dd6f5d fix(secscan): don't use slash_join, it discards the root 2017-07-11 14:12:57 -04:00
Evan Cordell
b9581e0baf fix(secscan): fix mitm cert path calculation 2017-07-11 13:26:19 -04:00
Joseph Schorr
a13235c032 Fix typo 2017-07-10 18:35:51 +03:00
Evan Cordell
939ddfd1d7 Merge v2.4.0-release into cherrypick-2.4.0 2017-07-10 10:25:18 -04:00
Joseph Schorr
176c26e3f7 Add config validation for action log archiving 2017-07-10 13:09:33 +03:00
EvB
ccca0c9655 refactor(util/tufmetadata/test): move app test to gc suite 2017-07-07 15:14:14 -04:00
Antoine Legrand
cdb3722c17 Use $QUAYPATH and $QUAYDIR in conf and init files 2017-07-05 16:23:54 +02:00
Evan Cordell
d64b8b1fcf Revert to old secret handling, fix license loading 2017-06-28 23:15:14 -04:00
Jimmy Zelinskie
1d2640e012 util.secscan.fake: add test for unexpected status 2017-06-28 13:40:04 -04:00
Evan Cordell
ef459a2d18 Update the expected response layout for kubernetes config 2017-06-28 07:28:57 -04:00
Jimmy Zelinskie
46087d5e64 util.secscan.api: more robust API failures cases 
Addresses QUAY-672 by handling all status codes that are not 404 and 5xx
and moving response decoding inside the try/except block to ensure that
the response object is in scope.
 2017-06-26 17:13:51 -04:00
Jimmy Zelinskie
e028e159c0 add app registry config to setup tool: default off 2017-06-16 15:44:00 -04:00
Jimmy Zelinskie
9df04a09d6 Merge pull request #2694 from jzelinskie/fix-torrent-config-validation 
Fix torrent config validation
 2017-06-09 13:39:01 -04:00
Jimmy Zelinskie
a16b469d9b util.registry.torrent: stash kid in JWT headers 
Upstream, chihaya reads this header in order to find the kid in the list
of maintained keys. A long time ago, it used to just iterate, but now it
needs to know the kid.
 2017-06-09 13:31:38 -04:00
Jimmy Zelinskie
7d07c2ed07 util.config.validators: fix torrent validation 
This code was mistaken the info dict with the params passed in an
announce request. Rather, now we expose a function for creating a jwt
from infohashes directly.
 2017-06-09 13:31:38 -04:00
Antoine Legrand
f0dd2e348b Merge pull request #2551 from coreos-inc/structured-logs 
Add log formatter class
 2017-06-07 08:22:18 -07:00
Antoine Legrand
3c99928a27 Add log JSON formatter 2017-06-07 00:02:52 +02:00
Kenny Lee Sin Cheong
1f76e9dc3b Merge pull request #2661 from kleesc/securityworker_cpu 
Raise an APIRequestFailure exception when security scanner is unavail…
 2017-06-03 12:15:45 -04:00
Joseph Schorr
0ba54ed4fc Simplify the caching of service keys to hopefully avoid the not found issue 
Makes accesses simpler and reduces the number of dictionaries to one, in an effort to remove race conditions
 2017-05-26 13:51:48 -04:00
josephschorr
2ec43483a8 Merge pull request #2662 from coreos-inc/direct-login 
Enable toggling of the direct login feature in the superuser panel
 2017-05-24 16:51:43 -04:00
Joseph Schorr
2b9873483a Enable toggling of the direct login feature in the superuser panel 
Allows superusers to disable login to the UI via credentials if at least one OIDC provider is configured
 2017-05-24 12:57:55 -04:00
Evan Cordell
c55c233f1f Merge pull request #2646 from ecordell/kubernetes-ca-fix 
ConfigProviders abstract over path construction
 2017-05-24 11:37:17 -04:00
josephschorr
8e8470890a Merge pull request #2653 from coreos-inc/new-signing-ui 
Implement updated UI for displaying the signing status of a tag, now …
 2017-05-24 11:31:52 -04:00
Kenny Lee Sin Cheong
203c0b76e0 Raise an APIRequestFailure exception when security scanner is unavailable 
Put worker to sleep for the duration of the default indexing interval
when an APIRequestFailure occurs, when the API request fails due to a
connection error, timeout, or other ambiguous errors, from
analyze_layer or get_layer_data .
 2017-05-24 11:04:44 -04:00
Evan Cordell
20da91d879 Add tests for providers and update install script 2017-05-23 15:43:21 -04:00
Evan Cordell
b3a5f0db1b Merge coreos/new-signing-ui into new-signing-ui 2017-05-23 13:07:18 -04:00
Evan Cordell
897da1df67 Fix tuf api calls 2017-05-23 12:36:49 -04:00
Evan Cordell
f877865e82 Fix tuf api calls 2017-05-23 10:47:59 -04:00
Evan Cordell
01b59e8d66 ConfigProviders abstract over path construction 
Fixes issue where certs can't be uploaded in UI in k8s
 2017-05-17 08:12:09 -04:00
Jimmy Zelinskie
702cdf59ff Merge pull request #2637 from jzelinskie/audit-apps 
Audit Logs for Apps
 2017-05-16 17:06:25 -04:00
Jimmy Zelinskie
4db789b656 add audit logging to app registry endpoints 2017-05-16 15:54:02 -04:00
Evan Cordell
e2be8481b0 Merge pull request #2643 from ecordell/all-delegations-tuf 
Return all tags in all delegations in tuf api
 2017-05-15 17:23:05 -04:00
Evan Cordell
3e3ed11634 Add api for getting all signed tags, separated by delegation 2017-05-15 16:18:30 -04:00
Jake Moshenko
21cb9f1aa1 Handle null executor cancellations separately from other exceptions 2017-05-15 13:45:44 -04:00
josephschorr
19f67bfa1b Merge pull request #2607 from coreos-inc/faster-security-notify 
Batch the tag lookups in the security notification worker in an attempt to significant reduce load
 2017-05-03 13:49:13 -04:00
Joseph Schorr
977bbc20a2 Add filtering onto the images query in get_matching_tags_for_images 
Should make the query even faster in the security notification case
 2017-05-02 18:29:14 -04:00
Joseph Schorr
4e09fff181 Remove test that breaks MySQL full DB tests 2017-05-02 16:04:46 -04:00
Joseph Schorr
98fcae753b Change the security notification system to use get_matching_tags_for_images 
This should vastly reduce the number of database calls we make, as instead of making 2-3 calls per image, we'll make two calls per ~100 images
 2017-05-02 15:39:27 -04:00
Evan Cordell
738f53f61a Merge pull request #2597 from ecordell/sni 
TUF metadata api SNI support
 2017-05-02 13:01:16 -04:00
Evan Cordell
b2569ffbb2 Support SNI in python requests, and only delete tuf metadata if it 
exists
 2017-05-02 09:32:12 -04:00
Joseph Schorr
ae0d1e831b Add prometheus metric for queued builds 2017-05-01 15:16:55 -04:00
josephschorr
8b148bf1d4 Merge pull request #2576 from coreos-inc/full-db-tests-tox 
Reenable full database testing locally and in concourse
 2017-04-27 18:09:15 -04:00
Joseph Schorr
4ea4ee3aa4 Fix time machine config validator on old-style config 
Existing config won't have the keys defined, so make sure we skip in that case (and just use the defaults)
 2017-04-27 14:24:47 -04:00
Joseph Schorr
cb3695a629 Change config validator tests to use the shared fixtures 2017-04-24 16:45:14 -04:00
Joseph Schorr
f296599162 Add additional logging around secscan analyze 2017-04-21 16:52:47 -04:00
Jake Moshenko
3b26e819d3 Merge pull request #2558 from jakedt/betternooper 
Make the nooper impl even smaller!
 2017-04-21 14:29:52 -04:00
Joseph Schorr
3dcbe3c631 If enabled, allow users and orgs to set their time machine expiration 
Fixes https://www.pivotaltracker.com/story/show/142881203
 2017-04-21 11:32:45 -04:00
Jimmy Zelinskie
6bef1d1ff3 Merge pull request #2322 from jzelinskie/acifix 
image/appc: fix volume conversion and add tests
 2017-04-21 10:15:03 -04:00
Jake Moshenko
e97ef09bd3 Make the nooper impl even smaller! 2017-04-20 13:42:49 -04:00
josephschorr
b03771669b Merge pull request #2554 from coreos-inc/no-secscan-delete 
Fix deleting repos when sec scan or signing is disabled
 2017-04-19 17:09:59 -04:00
Joseph Schorr
c5bb9abf11 Fix deleting repos when sec scan or signing is disabled 
Make sure we don't invoke the APIs to non-existent endpoints
 2017-04-19 16:57:36 -04:00
Joseph Schorr
08b9c4b0d4 Fill backfill script for recent changes 
We forgot that we need to lookup by user *object* and we need to lookup locations on their own
 2017-04-19 16:50:51 -04:00
Jake Moshenko
ba07270bb2 Turn off in-app sentry logging, only log 500s at the WSGI layer 2017-04-18 16:38:22 -04:00
Jake Moshenko
22f5934f34 Add error logging to Marketo calls 2017-04-17 10:19:52 -04:00
Evan Cordell
2661db7485 Add flag to enable trust per repo (#2541) 
* Add flag to enable trust per repo

* Add api for enabling/disabling trust

* Add new LogEntryKind for changing repo trust settings
Also add tests for repo trust api

* Add `set_trust` method to repository

* Expose new logkind to UI

* Fix registry tests

* Rebase migrations and regen test.db

* Raise downstreamissue if trust metadata can't be removed

* Refactor change_repo_trust

* Add show_if to change_repo_trust endpoint
 2017-04-15 08:26:33 -04:00
Evan Cordell
ec63e495fc Add repo purge callbacks and register TUF metadata deletion as one 2017-04-12 17:33:51 -04:00
Evan Cordell
883692345b Add unit tests for gun calculation 2017-04-12 17:33:51 -04:00
Evan Cordell
70ae34357f urljoin GUN together instead of manually concatenating 2017-04-12 17:33:51 -04:00
Evan Cordell
68128b938b Add tests for tuf metadata delete 2017-04-12 17:33:51 -04:00
Evan Cordell
abe6f40bc5 Add support for deleting TUF metadata when repo is deleted 2017-04-12 17:33:51 -04:00
josephschorr
2bc619137a Merge pull request #2512 from ecordell/tufmetadata 
Add tufmetadata endpoint
 2017-04-07 17:16:11 -04:00
Evan Cordell
217b4a5ab2 Return hashes and expiration when fetching signed tags 2017-04-07 16:12:28 -04:00
Joseph Schorr
ed3da4697f Add client ID and client secret to OIDC config validator 2017-04-07 11:33:02 -04:00
Jake Moshenko
c7241911a5 Fix old-style flask imports to silence deprecation warnings. 2017-04-06 13:15:48 -04:00
Jake Moshenko
a0817bfd59 Refresh dependencies and fix tests. 2017-04-06 13:15:48 -04:00
Evan Cordell
9515f18fb6 Add tufmetadata endpoint 2017-04-05 10:03:27 -04:00
Joseph Schorr
0b6c062e32 Add superuser panel config for team syncing 2017-04-03 11:31:30 -04:00
Joseph Schorr
a6486b7823 Gitlab validation must allow unspecified endpoint 
Gitlab config validator currently requires the gitlab endpoint to be specified, even though we support leaving it unspecified for non-enterprise installs. Fix the validator to allow this case.
 2017-03-30 12:57:41 -04:00
Joseph Schorr
45179216af Have sec scan retries actually work 
Until this change, if `ping` raised an exception, we wouldn't retry properly
 2017-03-29 16:19:46 -04:00
Jimmy Zelinskie
65a17dc155 Merge pull request #2473 from coreos-inc/certs-fixes 
Fixes and improvements around custom certificate handling
 2017-03-27 15:08:36 -04:00
Evan Cordell
1016641f8d refactor jwt context building 2017-03-27 11:37:17 -04:00
Evan Cordell
abd78bce56 Use constants for TUF roots 2017-03-27 11:37:17 -04:00
Evan Cordell
6ad107709c Change build_context_and_subject to take kwargs 2017-03-27 11:37:17 -04:00
Evan Cordell
43dd974dca Determine which TUF root to show based on actual access, not requested 
access
 2017-03-27 11:37:17 -04:00
Joseph Schorr
b017133cc6 Make QSS validation errors more descriptive 2017-03-24 17:28:16 -04:00
Jimmy Zelinskie
23759a1592 util.config.db: ensure blob locations sync on boot 2017-03-22 22:57:21 -04:00
Joseph Schorr
6ab5b8be45 Have storage replication backfill tool only backfill missing storages 
Prevents overload of the queue
 2017-03-22 11:30:49 -04:00
Joseph Schorr
6476488221 Skip bitbucket pushes without any commits 
Fixes https://sentry.io/coreos/backend-production/issues/178220183/
 2017-03-20 18:23:21 -04:00
josephschorr
432b2d3fe8 Merge pull request #2392 from coreos-inc/search-optimization 
Optimize repository search by changing our lookup strategy
 2017-03-10 15:44:26 -05:00
josephschorr
6d6be63ca6 Merge pull request #2393 from coreos-inc/oidc-ui 
OIDC configuration support in superuser config panel
 2017-03-10 12:13:48 -05:00
Joseph Schorr
b5bb76cdea Optimize repository search by changing our lookup strategy 
Previous to this change, repositories were looked up unfiltered in six different queries, and then filtered using the permissions model, which issued a query per repository found, making search incredibly slow. Instead, we now lookup a chunk of repositories unfiltered and then filter them via a single query to the database. By layering the filtering on top of the lookup, each as queries, we can minimize the number of queries necessary, without (at the same time) using a super expensive join.

Other changes:
- Remove the 5 page pre-lookup on V1 search and simply return that there is one more page available, until there isn't. While technically not correct, it is much more efficient, and no one should be using pagination with V1 search anyway.
- Remove the lookup for repos without entries in the RAC table. Instead, we now add a new RAC entry when the repository is created for *the day before*, with count 0, so that it is immediately searchable
- Remove lookup of results with a matching namespace; these aren't very relevant anyway, and it overly complicates sorting
 2017-03-09 19:47:55 -05:00
Joseph Schorr
eff1827d9d Batch QSS notifications after initial scan 2017-03-01 15:42:49 -05:00
Jimmy Zelinskie
cbb2fff0e2 util.secscan.api: raise exception for !200 status 2017-03-01 00:40:47 -05:00
Jimmy Zelinskie
cba7816caf util.failover: re-raise exceptions on failure 2017-03-01 00:40:47 -05:00
Joseph Schorr
157640e696 Add config validator for OIDC logins 2017-02-28 16:18:19 -05:00
Joseph Schorr
88b808f468 Fix typo 2017-02-24 12:23:18 -05:00
Joseph Schorr
d4eb4f7f3c Pull out github trigger and login validation into validator class 2017-02-24 12:23:18 -05:00
Joseph Schorr
a31f2267e8 Pull out gitlab trigger validation into validator class 2017-02-24 12:23:18 -05:00
Joseph Schorr
7a260d81d3 Pull out bitbucket trigger validation into validator class 2017-02-24 12:23:17 -05:00
Joseph Schorr
49638b081b Pull out google login validation into validator class 2017-02-24 12:23:17 -05:00
Joseph Schorr
620e377faf Pull out ssl validation into validator class 2017-02-24 12:23:17 -05:00
Joseph Schorr
e76b95f0e6 Add S3 storage test to validator tests 2017-02-24 12:23:17 -05:00
Joseph Schorr
09b3cfd549 Pull out torrent validation into validator class 2017-02-24 12:23:17 -05:00
Joseph Schorr
2944a4e13d Pull out signing validation into validator class 2017-02-24 12:23:17 -05:00
Joseph Schorr
8844ecbb7c Fix imports 2017-02-24 12:23:16 -05:00
Joseph Schorr
dcabb36ac7 Add TODO 2017-02-24 12:23:16 -05:00
Joseph Schorr
3db4c15459 Pull out security scanner validation into validator class 2017-02-24 12:23:16 -05:00
Joseph Schorr
c0f7530b29 Pull out JWT auth validation into validator class 
Also fixes a small bug in validation (yay tests!)
 2017-02-24 12:23:16 -05:00
Joseph Schorr
678f868bc4 Pull out keystone validation into validator class 2017-02-24 12:23:15 -05:00
Joseph Schorr
c55ddf7341 Pull out ldap validation into validator class 2017-02-24 12:23:15 -05:00
Joseph Schorr
2d64cf3000 Rename config validation source files 2017-02-24 12:23:15 -05:00
Joseph Schorr
00eceb7ed5 Pull out email validation into validator class 2017-02-24 12:23:15 -05:00
Joseph Schorr
ee4f5ed5d6 Move registry storage validator to new location 2017-02-24 12:23:15 -05:00
Joseph Schorr
b2afe68632 Pull out redis validation into validator class 2017-02-24 12:23:15 -05:00