Commit graph

802 commits

Author SHA1 Message Date
Joseph Schorr
b26a131085 Fix worker count to use CPU affinity correctly and be properly bounded
We were using the `cpu_count`, which doesn't respect container affinity. Now, we use `cpu_affinity` and also bound to make sure we don't start a million workers

Fixes https://jira.coreos.com/browse/QUAY-928
2018-05-03 11:57:20 +03:00
Sam Chow
f89ad30320
Merge pull request #3060 from quay/max-results-help-text
Configurable options for search, disable next page & add help text when at max results
2018-04-25 08:17:35 -07:00
Sam Chow
1afedafcbb Configurable options for search, info when at max
includes the options for  maximum search results per page, and the
maximum number of pages available before help text is shown, and
the next page button is disabled
2018-04-25 11:12:09 -04:00
Joseph Schorr
3309daa32e Add support for reduced initial build count for new possible abusing users
If configured, we now check the IP address of the user signing up and, if they are a possible threat, we further reduce their number of allowed maximum builds to the configured value.
2018-04-20 18:46:32 +03:00
Joseph Schorr
a59c951aa3 Add support for multiple scope parameters on V2 auth requests
Fixes https://jira.coreos.com/browse/QUAY-892
2018-04-18 20:16:49 +03:00
Brad Ison
c2ad6c5060
Check for null model objects
As of v2.8.2, peewee will not create model objects with all null
fields when an FK reference is null.  We have to check the model
instances for None.  See:

  https://github.com/coleifer/peewee/issues/1012
2018-04-04 14:19:45 -04:00
Jimmy Zelinskie
58072f8673 util/config/validators: ensure endpoint isn't prom 2018-04-02 17:59:48 -04:00
josephschorr
323eb63747
Merge pull request #3032 from coreos-inc/joseph.schorr/QUAY-885/squashed-sym
Retarget broken hard links in squashed images
2018-03-26 17:59:52 -04:00
Joseph Schorr
747819fbd2 Fix error with unicode URLs in torrent creation
Fixes https://jira.coreos.com/browse/QUAY-896
2018-03-26 17:07:00 -04:00
Joseph Schorr
dd470bdc9d Add a reporter for verbs to track number of storage streams are needed 2018-03-26 17:02:39 -04:00
Joseph Schorr
110366f656 Retarget hard links pointing to deleted files by emitting the deleted file contents under the first hard link instance. This fixes a breakage in the squashed TAR where we were pointing hard links to missing data.
Fixes https://jira.coreos.com/browse/QUAY-885
2018-03-23 14:00:46 -04:00
josephschorr
6c43b7ff0d
Merge pull request #3024 from coreos-inc/manageable-robots
Manageable robots epic
2018-03-21 18:50:17 -04:00
Joseph Schorr
2ea13e86a0 Add last_accessed information to User and expose for robot accounts
Fixes https://jira.coreos.com/browse/QUAY-848
2018-03-21 15:28:34 -04:00
Joseph Schorr
3586955669 Remove license code in Quay
No longer needed under Red Hat rules \o/

Fixes https://jira.coreos.com/browse/QUAY-883
2018-03-20 17:03:35 -04:00
Joseph Schorr
8e6ede4ac7 Small fixes for config schema validator in response to customer logs 2018-03-20 13:35:26 -04:00
Joseph Schorr
3438c1bfad Add new config fields to the schema 2018-03-01 16:49:51 -05:00
Joseph Schorr
ab0172d2fd Switch Quay to using an in-container memcached for data model caching 2018-02-27 16:55:22 -05:00
Joseph Schorr
8bc55a5676 Make namespace deletion asynchronous
Instead of deleting a namespace synchronously as before, we now mark the namespace for deletion, disable it, and rename it. A worker then comes along and deletes the namespace in the background. This results in a *significantly* better user experience, as the namespace deletion operation now "completes" in under a second, where before it could take 10s of minutes at the worse.

Fixes https://jira.coreos.com/browse/QUAY-838
2018-02-27 13:12:51 -05:00
Brad Ison
5da8744ddf Reject JWTs with future issued-at times
PyJWT stopped doing this in 1.5.0 because it's not part of the spec,
and there are legitimate reasons to issue future tokens.  We still
want to reject these though as we don't have that need.
2018-02-26 12:55:32 -05:00
josephschorr
d77aa9228f
Merge pull request #3002 from coreos-inc/joseph.schorr/QUAY-822/gc-app-tokens
Add a worker to automatically GC expired app specific tokens
2018-02-20 17:21:48 -05:00
Joseph Schorr
4d0ad0074d Fix config schema for bitbucket trigger 2018-02-20 16:59:34 -05:00
Joseph Schorr
9a452ace11 Add configurable limits for number of builds allowed under a namespace
We also support that limit being increased automatically once a successful billing charge has gone through
2018-02-20 16:54:22 -05:00
Brad Ison
62971b7f20
Merge pull request #2999 from bison/user-location
Add user location metadata filed
2018-02-20 16:48:37 -05:00
Joseph Schorr
188ea98441 Add new decorator to prevent reflected text attacks
Instead of disabling repo names with periods in them, we simply disallow calls to the API when they are GET requests, whose path ends in a dot, and that do not have a referrer from the frontend.
2018-02-20 11:33:45 -05:00
Evan Cordell
b545cad380 log response data on non-200 responses in TUF API 2018-02-20 09:46:53 -05:00
josephschorr
7cd2c00d4d
Merge pull request #2967 from coreos-inc/joseph.schorr/QS-111/auth-refactor
Refactor auth code to be cleaner and more extensible
2018-02-15 16:02:22 -05:00
Joseph Schorr
e220b50543 Refactor auth code to be cleaner and more extensible
We move all the auth handling, serialization and deserialization into a new AuthContext interface, and then standardize a registration model for handling of specific auth context types (user, robot, token, etc).
2018-02-14 15:35:27 -05:00
Brad Ison
87e55870b7 Add script for fixing missing admin permissions
Adds a util script to find and fix repositories in user namespaces
that are missing admin permissions for the owning user.  These admin
permissions are required, but were missing in some cases.  See:

  https://github.com/coreos-inc/quay/pull/2998
2018-02-13 16:23:17 -05:00
Joseph Schorr
d45161b120 Add a worker to automatically GC expired app specific tokens
Fixes https://jira.coreos.com/browse/QUAY-822
2018-02-12 14:56:01 -05:00
josephschorr
846deb75fe
Merge pull request #2902 from coreos-inc/joseph.schorr/QS-51/azure-blob-store
Add support for Azure Blob Storage
2018-02-07 11:34:29 -05:00
Brad Ison
5965929187 Include location in user analytics 2018-02-06 16:06:17 -05:00
Joseph Schorr
5490e64669 Fill out schema and schema whitelist 2018-02-06 15:27:01 -05:00
Joseph Schorr
7893ef6acc Add test to ensure that all config.py properties are defined in the config schema 2018-02-06 15:26:31 -05:00
Joseph Schorr
d488517b36 Implement storage driver for Azure Blob Storage 2018-02-06 13:48:40 -05:00
josephschorr
9f7b08d0ff
Merge pull request #2993 from coreos-inc/joseph.schorr/QUAY-797/pagination-size
Allow size of pages in V2 api to be configurable
2018-02-02 15:21:15 -05:00
Joseph Schorr
eae9175950 Allow size of pages in V2 api to be configurable 2018-02-02 13:54:41 -05:00
Joseph Schorr
bbdf9e074c Add metrics for tracking when instance key renewal succeeds and fails, as well as when instance key *lookup* fails 2018-02-02 11:14:42 -05:00
josephschorr
6514bf229f
Merge pull request #2973 from coreos-inc/joseph.schorr/QS-116/cloudfront-storage
Add support for configuring cloudfront storage
2018-02-02 10:14:28 -05:00
Joseph Schorr
b0f656731c Add support for configuring CloudFront storage engine
Fixes https://jira.coreos.com/browse/QS-116
2018-01-31 11:22:14 -05:00
Joseph Schorr
462500a389 Temp revert dot fix because it applies to repo names as well on pull 2018-01-26 14:43:37 -05:00
IvanCherepov
c228734978
Generates HTML documentation explaining all of configuration fields (#2952)
* create HTML documentation explaining all of schema's configuration fields
2018-01-24 14:09:29 -05:00
Joseph Schorr
ede3a81c68 Disallow dots in repository names to fix reflected text "attack"
Fixes https://jira.coreos.com/browse/QS-125
2018-01-18 13:19:37 -05:00
Joseph Schorr
524d77f527 Add an AppSpecificAuthToken data model for app-specific auth tokens. These will be used for the Docker CLI in place of username+password 2018-01-04 15:27:41 -05:00
Joseph Schorr
72bfebdb60 Add license validation to the config validation check
Should prevent a customer from accidentally saving a config that violates their license

Fixes https://jira.coreos.com/browse/QS-97
2017-12-19 13:44:08 -05:00
IvanCherepov
c383ac1f9d
Add config validation on startup (#2903)
* WIP

* Finish schema

Add three sections: security scanning, bittorrent support and feature flags.
2017-12-01 10:46:39 -05:00
Joseph Schorr
c168413a8e Fix bug when running ipresolver under Gitlab CI
Since the container does contain IP data, this would fail
2017-11-30 10:23:58 -05:00
Joseph Schorr
400a5db719 Add additional metrics on executor start and failure
This will allow us to register a pager if one of the executors starts failing consistently
2017-11-27 11:52:37 +02:00
Ivan Cherapau
a0adc1b0ec Fix typo in metrics 2017-11-14 23:16:25 -05:00
Joseph Schorr
2677720577 Fix exception raised for certain non-JSON strings given to is_json
This is breaking pushes in production for certain manifests

Fixes https://jira.prod.coreos.systems/browse/QS-60
2017-11-14 13:46:06 -05:00
Joseph Schorr
74f99ba94a Ensure encrypted passwords are not enabled with OIDC auth
Fixes https://jira.prod.coreos.systems/browse/QS-49
2017-10-31 16:03:28 -04:00