vbatts/quay
Archived
1
0
Fork 0
Code Issues Pull requests Releases Wiki Activity
8844 commits 2 branches 62 tags 205 MiB
Commit graph

890 commits

Author SHA1 Message Date
Joseph Schorr
f9e46b414c Move filelike tests to pytest 2018-07-12 15:45:30 +03:00
Joseph Schorr
0a23a51fba Move dockerfile parse tests to pytest 2018-07-12 15:43:03 +03:00
Joseph Schorr
3c65447299 Move backfill allocator tests to pytest 2018-07-12 15:32:57 +03:00
Sam Chow
9024419896 Modify ldap validator to just check user existence 
Remove auth user check from updating config app config

remove duplicate certs install script
 2018-07-11 16:49:13 -04:00
Sam Chow
bd54eacbad Add app var for init scripts location to access certs install 2018-07-10 11:43:34 -04:00
Joseph Schorr
4f152fd7c7 Make API errors more informative 
Fixes https://jira.coreos.com/browse/QUAY-999
 2018-07-08 11:45:33 +03:00
Sam Chow
84f604739f Add log rotation threshold configuration 2018-06-29 17:16:44 -04:00
Sam Chow
31e4c6d380
Merge pull request #3127 from quay/project/download-tar 
Q.E. Config User can update a config tarball pt 2
 2018-06-29 16:53:28 -04:00
Sam Chow
14fefea38f Revert local validation context, extract another util 2018-06-29 15:09:33 -04:00
Sam Chow
d7ffb54333 Move tar filter to file, add tests for it 2018-06-28 17:02:33 -04:00
Sam Chow
aa93d698b2 Tarball the config and give it to the front end 
Download file as blob to avoid binary string encoding
 2018-06-28 13:53:17 -04:00
Brad Ison
73cb7f3228
endpoints/api: Allow null fields in user metadata 
The user metadata fields are nullable in the database, but were not in
the json sechema.  This prevented users from updating some of their
information on the site if they hadn't set the metadata fields.
 2018-06-27 15:34:55 -04:00
Sam Chow
7619ab44e5 Revert inmemoryprov, skip local storage validation 2018-06-25 15:23:30 -04:00
Joseph Schorr
d4882f0077 Fix the IP data lookup to take in an API key 
Fixes https://jira.coreos.com/browse/QUAY-989
 2018-06-20 15:44:40 -04:00
Joseph Schorr
a3554a73d8
Merge pull request #3069 from quay/joseph.schorr/QUAY-913/db-test-data 
Add support for populating test data during migration testing
 2018-06-19 10:40:28 -04:00
Joseph Schorr
f6ff0d6ca0 Add support for populating test data during migration testing 
This change ensures that the tables in the database during migration have at least one row of "real" data, which should help catch issues in the future where we forget to set column defaults and other such schema oversights that can only be truly tested with non-empty tables

Fixes https://jira.coreos.com/browse/QUAY-913
 2018-06-18 14:14:32 -04:00
Joseph Schorr
0fdefd78e9 Extract app from torrent handling code 
Fixes https://jira.coreos.com/browse/QUAY-969
 2018-06-14 17:29:39 -04:00
Sam Chow
27721a8632 Add docker shell mode, and remove some config 2018-06-12 14:45:04 -04:00
Sam Chow
13293ecdea Fix dockerfile being able to pass in params 
Change config directory to local config_app one
 2018-06-12 14:45:00 -04:00
Sam Chow
ce26ded74b Constructor for ValidatorContext returns instance 2018-06-05 11:49:42 -04:00
Sam Chow
7f21d0da58
Merge pull request #3096 from quay/refactor/secscan-simple 
Refactor validators to decouple app
 2018-06-01 15:44:18 -04:00
Sam Chow
301cc6992a Remove jwt validation for jschorr to fix later 
Refactor oauth validate method to take config over entire appconfig
 2018-06-01 15:07:06 -04:00
Sam Chow
7df8ed4a60 Add a security scanner api config object for params 
Change SecScanAPI to use a uri creation func instead of test context

Pass config provider through validator context

Remove app config dependency for validators
 2018-06-01 15:06:50 -04:00
Sam Chow
554d4f47a8 Change validators to use the validator_context 
Change InstanceKeys to take a namedtuple for context
 2018-06-01 14:59:49 -04:00
Joseph Schorr
b2262eaf46 Add feature flag to disable username confirmation 
Fixes https://jira.coreos.com/browse/QUAY-914
 2018-06-01 13:30:50 -04:00
Sam Chow
e967fde3ae Decouple oauth methods from app with a namedtuple 2018-05-31 14:53:27 -04:00
Sam Chow
d45b925155 Move config provider to _init to decouple from app 
remove app references from validators
 2018-05-25 11:15:06 -04:00
Joseph Schorr
6ffafe44d3
Merge pull request #3059 from quay/joseph.schorr/QUAY-906/reg-tests 
Move registry integration tests to py.test
 2018-05-22 17:09:11 -04:00
Joseph Schorr
ad1da86986 Add some additional caching and logs to the robot cleanup migration 2018-05-15 15:34:54 -04:00
josephschorr
7345ff855c
Merge pull request #3071 from quay/joseph.schorr/QUAY-932/namespace-robot-deletion 
Fix issue where we didn't delete robots immediately under a namespace
 2018-05-11 00:27:17 +03:00
Joseph Schorr
2ae69dc651 Further fixes to the Kubernetes config provider, and a new set of proper unit tests 2018-05-10 16:44:18 +03:00
Joseph Schorr
f06eec8a35 Fix issue where we didn't delete robots immediately under a namespace 
This could result in "hanging" robot accounts, although that would only leak the names of said accounts. Now we delete them immediately AND we proactively delete them before replacing the namespace (just to be sure)
 2018-05-09 17:53:30 +03:00
josephschorr
7722721396
Merge pull request #3064 from quay/joseph.schorr/QUAY-928/fix-worker-count 
Fix worker count to  use CPU affinity correctly and be properly bounded
 2018-05-07 20:45:26 +03:00
josephschorr
de36b36f9a
Merge pull request #3051 from quay/joseph.schorr/QUAY-911/fix-kub-provider 
Fix Kubernetes config provider for recent changes in Kub API
 2018-05-07 20:45:09 +03:00
Joseph Schorr
942f526016 Missing parens on IP resolver lookup 
Also adds a generic catch in case this happens again; we should *never* fail
 2018-05-04 02:14:26 +03:00
Joseph Schorr
b26a131085 Fix worker count to use CPU affinity correctly and be properly bounded 
We were using the `cpu_count`, which doesn't respect container affinity. Now, we use `cpu_affinity` and also bound to make sure we don't start a million workers

Fixes https://jira.coreos.com/browse/QUAY-928
 2018-05-03 11:57:20 +03:00
Joseph Schorr
77adf9dd77 Fix bug which allowed for implicit library namespace access via the V1 registry protocol when the feature flag was off 
Now we raise a 400 as expected
 2018-05-01 13:28:24 +03:00
Joseph Schorr
178c8e7cb0 Fix bug in in-memory data cache 
Previously, if we didn't find a key, we'd empty the entire cache, making it essentially a single-key cache. We skip clearing now, although this does mean we won't GC expired entries (not a problem for tests, though)
 2018-05-01 13:28:24 +03:00
Sam Chow
f89ad30320
Merge pull request #3060 from quay/max-results-help-text 
Configurable options for search, disable next page & add help text when at max results
 2018-04-25 08:17:35 -07:00
Sam Chow
1afedafcbb Configurable options for search, info when at max 
includes the options for  maximum search results per page, and the
maximum number of pages available before help text is shown, and
the next page button is disabled
 2018-04-25 11:12:09 -04:00
Joseph Schorr
e20295f573 Fix Kubernetes config provider for recent changes in Kub API 
Kubernetes secret volumes are now mounted as read-only, so we have to write the files *only* via the Kub API

Fixes https://jira.coreos.com/browse/QUAY-911
 2018-04-22 17:22:28 +03:00
Joseph Schorr
3309daa32e Add support for reduced initial build count for new possible abusing users 
If configured, we now check the IP address of the user signing up and, if they are a possible threat, we further reduce their number of allowed maximum builds to the configured value.
 2018-04-20 18:46:32 +03:00
Joseph Schorr
a59c951aa3 Add support for multiple scope parameters on V2 auth requests 
Fixes https://jira.coreos.com/browse/QUAY-892
 2018-04-18 20:16:49 +03:00
Brad Ison
c2ad6c5060
Check for null model objects 
As of v2.8.2, peewee will not create model objects with all null
fields when an FK reference is null.  We have to check the model
instances for None.  See:

  https://github.com/coleifer/peewee/issues/1012
 2018-04-04 14:19:45 -04:00
Jimmy Zelinskie
58072f8673 util/config/validators: ensure endpoint isn't prom 2018-04-02 17:59:48 -04:00
josephschorr
323eb63747
Merge pull request #3032 from coreos-inc/joseph.schorr/QUAY-885/squashed-sym 
Retarget broken hard links in squashed images
 2018-03-26 17:59:52 -04:00
Joseph Schorr
747819fbd2 Fix error with unicode URLs in torrent creation 
Fixes https://jira.coreos.com/browse/QUAY-896
 2018-03-26 17:07:00 -04:00
Joseph Schorr
dd470bdc9d Add a reporter for verbs to track number of storage streams are needed 2018-03-26 17:02:39 -04:00
Joseph Schorr
110366f656 Retarget hard links pointing to deleted files by emitting the deleted file contents under the first hard link instance. This fixes a breakage in the squashed TAR where we were pointing hard links to missing data. 
Fixes https://jira.coreos.com/browse/QUAY-885
 2018-03-23 14:00:46 -04:00
josephschorr
6c43b7ff0d
Merge pull request #3024 from coreos-inc/manageable-robots 
Manageable robots epic
 2018-03-21 18:50:17 -04:00
Joseph Schorr
2ea13e86a0 Add last_accessed information to User and expose for robot accounts 
Fixes https://jira.coreos.com/browse/QUAY-848
 2018-03-21 15:28:34 -04:00
Joseph Schorr
3586955669 Remove license code in Quay 
No longer needed under Red Hat rules \o/

Fixes https://jira.coreos.com/browse/QUAY-883
 2018-03-20 17:03:35 -04:00
Joseph Schorr
8e6ede4ac7 Small fixes for config schema validator in response to customer logs 2018-03-20 13:35:26 -04:00
Joseph Schorr
3438c1bfad Add new config fields to the schema 2018-03-01 16:49:51 -05:00
Joseph Schorr
ab0172d2fd Switch Quay to using an in-container memcached for data model caching 2018-02-27 16:55:22 -05:00
Joseph Schorr
8bc55a5676 Make namespace deletion asynchronous 
Instead of deleting a namespace synchronously as before, we now mark the namespace for deletion, disable it, and rename it. A worker then comes along and deletes the namespace in the background. This results in a *significantly* better user experience, as the namespace deletion operation now "completes" in under a second, where before it could take 10s of minutes at the worse.

Fixes https://jira.coreos.com/browse/QUAY-838
 2018-02-27 13:12:51 -05:00
Brad Ison
5da8744ddf Reject JWTs with future issued-at times 
PyJWT stopped doing this in 1.5.0 because it's not part of the spec,
and there are legitimate reasons to issue future tokens.  We still
want to reject these though as we don't have that need.
 2018-02-26 12:55:32 -05:00
josephschorr
d77aa9228f
Merge pull request #3002 from coreos-inc/joseph.schorr/QUAY-822/gc-app-tokens 
Add a worker to automatically GC expired app specific tokens
 2018-02-20 17:21:48 -05:00
Joseph Schorr
4d0ad0074d Fix config schema for bitbucket trigger 2018-02-20 16:59:34 -05:00
Joseph Schorr
9a452ace11 Add configurable limits for number of builds allowed under a namespace 
We also support that limit being increased automatically once a successful billing charge has gone through
 2018-02-20 16:54:22 -05:00
Brad Ison
62971b7f20
Merge pull request #2999 from bison/user-location 
Add user location metadata filed
 2018-02-20 16:48:37 -05:00
Joseph Schorr
188ea98441 Add new decorator to prevent reflected text attacks 
Instead of disabling repo names with periods in them, we simply disallow calls to the API when they are GET requests, whose path ends in a dot, and that do not have a referrer from the frontend.
 2018-02-20 11:33:45 -05:00
Evan Cordell
b545cad380 log response data on non-200 responses in TUF API 2018-02-20 09:46:53 -05:00
josephschorr
7cd2c00d4d
Merge pull request #2967 from coreos-inc/joseph.schorr/QS-111/auth-refactor 
Refactor auth code to be cleaner and more extensible
 2018-02-15 16:02:22 -05:00
Joseph Schorr
e220b50543 Refactor auth code to be cleaner and more extensible 
We move all the auth handling, serialization and deserialization into a new AuthContext interface, and then standardize a registration model for handling of specific auth context types (user, robot, token, etc).
 2018-02-14 15:35:27 -05:00
Brad Ison
87e55870b7 Add script for fixing missing admin permissions 
Adds a util script to find and fix repositories in user namespaces
that are missing admin permissions for the owning user.  These admin
permissions are required, but were missing in some cases.  See:

  https://github.com/coreos-inc/quay/pull/2998
 2018-02-13 16:23:17 -05:00
Joseph Schorr
d45161b120 Add a worker to automatically GC expired app specific tokens 
Fixes https://jira.coreos.com/browse/QUAY-822
 2018-02-12 14:56:01 -05:00
josephschorr
846deb75fe
Merge pull request #2902 from coreos-inc/joseph.schorr/QS-51/azure-blob-store 
Add support for Azure Blob Storage
 2018-02-07 11:34:29 -05:00
Brad Ison
5965929187 Include location in user analytics 2018-02-06 16:06:17 -05:00
Joseph Schorr
5490e64669 Fill out schema and schema whitelist 2018-02-06 15:27:01 -05:00
Joseph Schorr
7893ef6acc Add test to ensure that all config.py properties are defined in the config schema 2018-02-06 15:26:31 -05:00
Joseph Schorr
d488517b36 Implement storage driver for Azure Blob Storage 2018-02-06 13:48:40 -05:00
josephschorr
9f7b08d0ff
Merge pull request #2993 from coreos-inc/joseph.schorr/QUAY-797/pagination-size 
Allow size of pages in V2 api to be configurable
 2018-02-02 15:21:15 -05:00
Joseph Schorr
eae9175950 Allow size of pages in V2 api to be configurable 2018-02-02 13:54:41 -05:00
Joseph Schorr
bbdf9e074c Add metrics for tracking when instance key renewal succeeds and fails, as well as when instance key *lookup* fails 2018-02-02 11:14:42 -05:00
josephschorr
6514bf229f
Merge pull request #2973 from coreos-inc/joseph.schorr/QS-116/cloudfront-storage 
Add support for configuring cloudfront storage
 2018-02-02 10:14:28 -05:00
Joseph Schorr
b0f656731c Add support for configuring CloudFront storage engine 
Fixes https://jira.coreos.com/browse/QS-116
 2018-01-31 11:22:14 -05:00
Joseph Schorr
462500a389 Temp revert dot fix because it applies to repo names as well on pull 2018-01-26 14:43:37 -05:00
IvanCherepov
c228734978
Generates HTML documentation explaining all of configuration fields (#2952) 
* create HTML documentation explaining all of schema's configuration fields
 2018-01-24 14:09:29 -05:00
Joseph Schorr
ede3a81c68 Disallow dots in repository names to fix reflected text "attack" 
Fixes https://jira.coreos.com/browse/QS-125
 2018-01-18 13:19:37 -05:00
Joseph Schorr
524d77f527 Add an AppSpecificAuthToken data model for app-specific auth tokens. These will be used for the Docker CLI in place of username+password 2018-01-04 15:27:41 -05:00
Joseph Schorr
72bfebdb60 Add license validation to the config validation check 
Should prevent a customer from accidentally saving a config that violates their license

Fixes https://jira.coreos.com/browse/QS-97
 2017-12-19 13:44:08 -05:00
IvanCherepov
c383ac1f9d
Add config validation on startup (#2903) 
* WIP

* Finish schema

Add three sections: security scanning, bittorrent support and feature flags.
 2017-12-01 10:46:39 -05:00
Joseph Schorr
c168413a8e Fix bug when running ipresolver under Gitlab CI 
Since the container does contain IP data, this would fail
 2017-11-30 10:23:58 -05:00
Joseph Schorr
400a5db719 Add additional metrics on executor start and failure 
This will allow us to register a pager if one of the executors starts failing consistently
 2017-11-27 11:52:37 +02:00
Ivan Cherapau
a0adc1b0ec Fix typo in metrics 2017-11-14 23:16:25 -05:00
Joseph Schorr
2677720577 Fix exception raised for certain non-JSON strings given to is_json 
This is breaking pushes in production for certain manifests

Fixes https://jira.prod.coreos.systems/browse/QS-60
 2017-11-14 13:46:06 -05:00
Joseph Schorr
74f99ba94a Ensure encrypted passwords are not enabled with OIDC auth 
Fixes https://jira.prod.coreos.systems/browse/QS-49
 2017-10-31 16:03:28 -04:00
Joseph Schorr
8194f5cf72 Switch ipresolver to always be defined in the storage context 
We now use a no-op IP resolver instead of an IF check

Fixes https://jira.prod.coreos.systems/browse/QS-38
 2017-10-17 14:29:40 -04:00
josephschorr
3bef21253d Merge pull request #2695 from coreos-inc/oidc-internal-auth 
OIDC internal auth support
 2017-10-02 16:51:17 -04:00
Joseph Schorr
f51a863158 Remove access_token from user_info 2017-10-02 16:51:09 -04:00
Joseph Schorr
05b4a7d457 Add worker to update ipresolver data files every few hours 2017-09-28 14:40:59 -04:00
Joseph Schorr
52927de7f6 Add resolved IP information to track_and_log 2017-09-28 14:40:58 -04:00
Joseph Schorr
010dda2c52 Add CloudFrontedS3Storage, which redirects to CloudFront for non-S3 ips 2017-09-28 14:40:58 -04:00
Joseph Schorr
2d522764f7 Add IP resolver utility that returns whether an IP is under AWS 2017-09-26 16:11:16 -04:00
josephschorr
c44cc072fa Merge pull request #2864 from coreos-inc/partial-autocomplete 
Partial autocomplete
 2017-09-13 11:26:11 -04:00
Joseph Schorr
54a4476cbb Make missing log more descriptive 2017-09-12 16:19:55 -04:00
Joseph Schorr
c105123ad4 Add superuser config for prefix autocomplete setting 2017-09-12 15:57:57 -04:00
Joseph Schorr
bc82edb2d1 Add ability to configure OIDC internal auth engine via superuser panel 2017-09-12 12:23:52 -04:00
Joseph Schorr
783799c227 Make team sync timeout config actually configurable 2017-09-06 14:08:30 -04:00