Commit graph

827 commits

Author SHA1 Message Date
Joseph Schorr
d4882f0077 Fix the IP data lookup to take in an API key
Fixes https://jira.coreos.com/browse/QUAY-989
2018-06-20 15:44:40 -04:00
Joseph Schorr
a3554a73d8
Merge pull request #3069 from quay/joseph.schorr/QUAY-913/db-test-data
Add support for populating test data during migration testing
2018-06-19 10:40:28 -04:00
Joseph Schorr
f6ff0d6ca0 Add support for populating test data during migration testing
This change ensures that the tables in the database during migration have at least one row of "real" data, which should help catch issues in the future where we forget to set column defaults and other such schema oversights that can only be truly tested with non-empty tables

Fixes https://jira.coreos.com/browse/QUAY-913
2018-06-18 14:14:32 -04:00
Joseph Schorr
0fdefd78e9 Extract app from torrent handling code
Fixes https://jira.coreos.com/browse/QUAY-969
2018-06-14 17:29:39 -04:00
Sam Chow
27721a8632 Add docker shell mode, and remove some config 2018-06-12 14:45:04 -04:00
Sam Chow
13293ecdea Fix dockerfile being able to pass in params
Change config directory to local config_app one
2018-06-12 14:45:00 -04:00
Sam Chow
ce26ded74b Constructor for ValidatorContext returns instance 2018-06-05 11:49:42 -04:00
Sam Chow
7f21d0da58
Merge pull request #3096 from quay/refactor/secscan-simple
Refactor validators to decouple app
2018-06-01 15:44:18 -04:00
Sam Chow
301cc6992a Remove jwt validation for jschorr to fix later
Refactor oauth validate method to take config over entire appconfig
2018-06-01 15:07:06 -04:00
Sam Chow
7df8ed4a60 Add a security scanner api config object for params
Change SecScanAPI to use a uri creation func instead of test context

Pass config provider through validator context

Remove app config dependency for validators
2018-06-01 15:06:50 -04:00
Sam Chow
554d4f47a8 Change validators to use the validator_context
Change InstanceKeys to take a namedtuple for context
2018-06-01 14:59:49 -04:00
Joseph Schorr
b2262eaf46 Add feature flag to disable username confirmation
Fixes https://jira.coreos.com/browse/QUAY-914
2018-06-01 13:30:50 -04:00
Sam Chow
e967fde3ae Decouple oauth methods from app with a namedtuple 2018-05-31 14:53:27 -04:00
Sam Chow
d45b925155 Move config provider to _init to decouple from app
remove app references from validators
2018-05-25 11:15:06 -04:00
Joseph Schorr
6ffafe44d3
Merge pull request #3059 from quay/joseph.schorr/QUAY-906/reg-tests
Move registry integration tests to py.test
2018-05-22 17:09:11 -04:00
Joseph Schorr
ad1da86986 Add some additional caching and logs to the robot cleanup migration 2018-05-15 15:34:54 -04:00
josephschorr
7345ff855c
Merge pull request #3071 from quay/joseph.schorr/QUAY-932/namespace-robot-deletion
Fix issue where we didn't delete robots immediately under a namespace
2018-05-11 00:27:17 +03:00
Joseph Schorr
2ae69dc651 Further fixes to the Kubernetes config provider, and a new set of proper unit tests 2018-05-10 16:44:18 +03:00
Joseph Schorr
f06eec8a35 Fix issue where we didn't delete robots immediately under a namespace
This could result in "hanging" robot accounts, although that would only leak the names of said accounts. Now we delete them immediately AND we proactively delete them before replacing the namespace (just to be sure)
2018-05-09 17:53:30 +03:00
josephschorr
7722721396
Merge pull request #3064 from quay/joseph.schorr/QUAY-928/fix-worker-count
Fix worker count to  use CPU affinity correctly and be properly bounded
2018-05-07 20:45:26 +03:00
josephschorr
de36b36f9a
Merge pull request #3051 from quay/joseph.schorr/QUAY-911/fix-kub-provider
Fix Kubernetes config provider for recent changes in Kub API
2018-05-07 20:45:09 +03:00
Joseph Schorr
942f526016 Missing parens on IP resolver lookup
Also adds a generic catch in case this happens again; we should *never* fail
2018-05-04 02:14:26 +03:00
Joseph Schorr
b26a131085 Fix worker count to use CPU affinity correctly and be properly bounded
We were using the `cpu_count`, which doesn't respect container affinity. Now, we use `cpu_affinity` and also bound to make sure we don't start a million workers

Fixes https://jira.coreos.com/browse/QUAY-928
2018-05-03 11:57:20 +03:00
Joseph Schorr
77adf9dd77 Fix bug which allowed for implicit library namespace access via the V1 registry protocol when the feature flag was off
Now we raise a 400 as expected
2018-05-01 13:28:24 +03:00
Joseph Schorr
178c8e7cb0 Fix bug in in-memory data cache
Previously, if we didn't find a key, we'd empty the entire cache, making it essentially a single-key cache. We skip clearing now, although this does mean we won't GC expired entries (not a problem for tests, though)
2018-05-01 13:28:24 +03:00
Sam Chow
f89ad30320
Merge pull request #3060 from quay/max-results-help-text
Configurable options for search, disable next page & add help text when at max results
2018-04-25 08:17:35 -07:00
Sam Chow
1afedafcbb Configurable options for search, info when at max
includes the options for  maximum search results per page, and the
maximum number of pages available before help text is shown, and
the next page button is disabled
2018-04-25 11:12:09 -04:00
Joseph Schorr
e20295f573 Fix Kubernetes config provider for recent changes in Kub API
Kubernetes secret volumes are now mounted as read-only, so we have to write the files *only* via the Kub API

Fixes https://jira.coreos.com/browse/QUAY-911
2018-04-22 17:22:28 +03:00
Joseph Schorr
3309daa32e Add support for reduced initial build count for new possible abusing users
If configured, we now check the IP address of the user signing up and, if they are a possible threat, we further reduce their number of allowed maximum builds to the configured value.
2018-04-20 18:46:32 +03:00
Joseph Schorr
a59c951aa3 Add support for multiple scope parameters on V2 auth requests
Fixes https://jira.coreos.com/browse/QUAY-892
2018-04-18 20:16:49 +03:00
Brad Ison
c2ad6c5060
Check for null model objects
As of v2.8.2, peewee will not create model objects with all null
fields when an FK reference is null.  We have to check the model
instances for None.  See:

  https://github.com/coleifer/peewee/issues/1012
2018-04-04 14:19:45 -04:00
Jimmy Zelinskie
58072f8673 util/config/validators: ensure endpoint isn't prom 2018-04-02 17:59:48 -04:00
josephschorr
323eb63747
Merge pull request #3032 from coreos-inc/joseph.schorr/QUAY-885/squashed-sym
Retarget broken hard links in squashed images
2018-03-26 17:59:52 -04:00
Joseph Schorr
747819fbd2 Fix error with unicode URLs in torrent creation
Fixes https://jira.coreos.com/browse/QUAY-896
2018-03-26 17:07:00 -04:00
Joseph Schorr
dd470bdc9d Add a reporter for verbs to track number of storage streams are needed 2018-03-26 17:02:39 -04:00
Joseph Schorr
110366f656 Retarget hard links pointing to deleted files by emitting the deleted file contents under the first hard link instance. This fixes a breakage in the squashed TAR where we were pointing hard links to missing data.
Fixes https://jira.coreos.com/browse/QUAY-885
2018-03-23 14:00:46 -04:00
josephschorr
6c43b7ff0d
Merge pull request #3024 from coreos-inc/manageable-robots
Manageable robots epic
2018-03-21 18:50:17 -04:00
Joseph Schorr
2ea13e86a0 Add last_accessed information to User and expose for robot accounts
Fixes https://jira.coreos.com/browse/QUAY-848
2018-03-21 15:28:34 -04:00
Joseph Schorr
3586955669 Remove license code in Quay
No longer needed under Red Hat rules \o/

Fixes https://jira.coreos.com/browse/QUAY-883
2018-03-20 17:03:35 -04:00
Joseph Schorr
8e6ede4ac7 Small fixes for config schema validator in response to customer logs 2018-03-20 13:35:26 -04:00
Joseph Schorr
3438c1bfad Add new config fields to the schema 2018-03-01 16:49:51 -05:00
Joseph Schorr
ab0172d2fd Switch Quay to using an in-container memcached for data model caching 2018-02-27 16:55:22 -05:00
Joseph Schorr
8bc55a5676 Make namespace deletion asynchronous
Instead of deleting a namespace synchronously as before, we now mark the namespace for deletion, disable it, and rename it. A worker then comes along and deletes the namespace in the background. This results in a *significantly* better user experience, as the namespace deletion operation now "completes" in under a second, where before it could take 10s of minutes at the worse.

Fixes https://jira.coreos.com/browse/QUAY-838
2018-02-27 13:12:51 -05:00
Brad Ison
5da8744ddf Reject JWTs with future issued-at times
PyJWT stopped doing this in 1.5.0 because it's not part of the spec,
and there are legitimate reasons to issue future tokens.  We still
want to reject these though as we don't have that need.
2018-02-26 12:55:32 -05:00
josephschorr
d77aa9228f
Merge pull request #3002 from coreos-inc/joseph.schorr/QUAY-822/gc-app-tokens
Add a worker to automatically GC expired app specific tokens
2018-02-20 17:21:48 -05:00
Joseph Schorr
4d0ad0074d Fix config schema for bitbucket trigger 2018-02-20 16:59:34 -05:00
Joseph Schorr
9a452ace11 Add configurable limits for number of builds allowed under a namespace
We also support that limit being increased automatically once a successful billing charge has gone through
2018-02-20 16:54:22 -05:00
Brad Ison
62971b7f20
Merge pull request #2999 from bison/user-location
Add user location metadata filed
2018-02-20 16:48:37 -05:00
Joseph Schorr
188ea98441 Add new decorator to prevent reflected text attacks
Instead of disabling repo names with periods in them, we simply disallow calls to the API when they are GET requests, whose path ends in a dot, and that do not have a referrer from the frontend.
2018-02-20 11:33:45 -05:00
Evan Cordell
b545cad380 log response data on non-200 responses in TUF API 2018-02-20 09:46:53 -05:00