Commit graph

1314 commits

Author SHA1 Message Date
Jake Moshenko
c2fcf8bead Merge remote-tracking branch 'upstream/phase4-11-07-2015' into python-registry-v2 2015-11-06 18:18:29 -05:00
Jake Moshenko
4314882fa0 Reverse the order of get_parent_images 2015-11-06 17:47:08 -05:00
Jake Moshenko
ad93425ead Stop writing to v1 checksum on ImageStorage 2015-11-06 16:40:04 -05:00
Jake Moshenko
75f917f592 Stop reading the v1 checksums from storage 2015-11-06 16:17:12 -05:00
Joseph Schorr
cfa03951e1 Add a SecScanEndpoint class and move all the cert and config handling in there 2015-11-06 15:22:18 -05:00
Joseph Schorr
e4508fc0d0 Add vulnerabilities and packages API to Quay
Fixes #564
2015-11-06 15:22:18 -05:00
Joseph Schorr
0f3db709ea Add a vulnerability_found event for notice when we detect a vuln
Fixes #637

Note: This PR does *not* actually raise the event; it merely adds support for it
2015-11-06 15:22:18 -05:00
Jake Moshenko
3d0bcbaaeb Move v1 checksums to image and track v2 separately 2015-11-06 15:17:55 -05:00
Matt Jibson
f4b57eff96 Set and use ETag headers
Also set no-cache exactly as github recommends. The removed @no_cache
decorater used "Cache-Control:no-cache, no-store, must-revalidate", but
just no-cache should be sufficient, and should certainly work correctly
with github.

See: https://github.com/github/markup/issues/224#issuecomment-48532178

fixes #712
2015-11-06 12:15:15 -05:00
Quentin Machu
da1fe7d48b Merge pull request #790 from Quentin-M/set4O4
Define nginx v2 vhost & properly set 404 status code
2015-11-04 16:32:11 -05:00
josephschorr
11be448d75 Merge pull request #773 from coreos-inc/imageload
Never load the full repo image list
2015-11-04 16:29:20 -05:00
Matt Jibson
4d81567a0c Remove error notification when user deletes repos
Also prevent duplicate notifications of that type.

fixes #493
2015-11-04 16:11:15 -05:00
Joseph Schorr
4f41f79fa8 Never load the full repo image list
Always make smaller queries per tag to ensure we scale better

Fixes #754
2015-11-04 15:53:00 -05:00
Quentin Machu
c1fa22d9b0 Define nginx v2 vhost & properly set 404 status code
Fixes #777
2015-11-04 14:56:18 -05:00
josephschorr
c3a4c36df7 Merge pull request #761 from coreos-inc/fixtoomanylogin
Move decorator for TooManyLoginAttempts into general decorated module
2015-11-04 12:29:01 -05:00
Joseph Schorr
d4646e459e Disable 404, as it is breaking V2 API checks 2015-11-04 02:47:33 -05:00
Joseph Schorr
95c47fe250 Fix layer ordering in verbs 2015-11-03 14:43:47 -05:00
Joseph Schorr
5e1cd2b2ad Move decorator for TooManyLoginAttempts into general decorated module
Currently, this is missing in gunicorn_registry which causes 500s when user logins become throttled
2015-11-03 12:16:01 -05:00
Joseph Schorr
f6a53f7cc5 Change all Quay.io references to Quay, fix tour and change logo
Fixes #741
2015-11-02 14:37:48 -05:00
josephschorr
4ae940aede Merge pull request #660 from coreos-inc/superuser
Superuser Panel Improvements
2015-10-30 14:32:16 -04:00
Quentin Machu
3f35265858 Merge pull request #683 from Quentin-M/whoops-404
Add 404 page
2015-10-30 14:30:20 -04:00
Jake Moshenko
1666ac50fe Filter down the signing key to only public portion 2015-10-26 16:40:19 -04:00
Jake Moshenko
2c10d28afc Merge remote-tracking branch 'upstream/master' into python-registry-v2 2015-10-26 14:44:16 -04:00
Jake Moshenko
fc55730db8 Add a feature flag to advertise v2 endpoints 2015-10-26 14:20:51 -04:00
Jake Moshenko
9da64f3aba Stop writing to deprecated columns for image data. 2015-10-24 14:45:15 -04:00
Joseph Schorr
7bac042954 Fix verbs for merged changes to image and image storage
Fixes #698
2015-10-23 15:49:31 -04:00
Jimmy Zelinskie
e973289397 Revert "Revert "Merge pull request #682 from jzelinskie/revertrevert""
This reverts commit 278bc736e3.
2015-10-23 15:26:33 -04:00
Jake Moshenko
e7a6176594 Merge remote-tracking branch 'upstream/v2-phase4' into python-registry-v2 2015-10-22 16:59:28 -04:00
Jimmy Zelinskie
278bc736e3 Revert "Merge pull request #682 from jzelinskie/revertrevert"
This reverts commit 627ad25c9c, reversing
changes made to 31c392fecc.
2015-10-22 16:02:07 -04:00
Jake Moshenko
ce94931540 Stop writing to deprecated columns for image data. 2015-10-22 12:14:39 -04:00
Quentin Machu
adb744089e Add 404 page
Fixes coreos-inc/quay#677
2015-10-21 18:40:15 -04:00
josephschorr
5dae970787 Merge pull request #681 from coreos-inc/userorg
Return user orgs when making a call via OAuth
2015-10-21 16:41:43 -04:00
Joseph Schorr
5d8121e060 Return user orgs when making a call via OAuth
Fixes #673
2015-10-21 16:40:31 -04:00
Jimmy Zelinskie
39cfe77d42 Revert "Merge pull request #557 from coreos-inc/revert-migration"
This reverts commit c4f938898a, reversing
changes made to 7ad2522dbe.
2015-10-21 15:29:57 -04:00
Joseph Schorr
147b7b26b4 Fix is_public in repo list
Fixes #678
2015-10-21 14:13:39 -04:00
Matt Jibson
b4554f4d14 Verify signed manifests
fixes #394
2015-10-20 02:08:45 -04:00
Joseph Schorr
5941f3937c Enable async GC for all
Fixes #569
2015-10-19 14:22:41 -04:00
Joseph Schorr
d464af4cce Add ability to update superusers via the UI
Fixes #634
2015-10-16 15:41:18 -04:00
Joseph Schorr
a37b9394d9 Add org email address to orgs list 2015-10-16 15:17:51 -04:00
Joseph Schorr
ad5beab3ef Disable superuser functions around users when not using DB auth 2015-10-16 15:14:49 -04:00
josephschorr
d3857e509f Merge pull request #643 from coreos-inc/nullimage
Check and handle NULL image_size
2015-10-15 13:26:13 -04:00
Joseph Schorr
fe79d5fb66 Check and handle NULL image_size
Fixes #613
2015-10-15 13:25:54 -04:00
josephschorr
24b54f1e34 Merge pull request #615 from coreos-inc/queriesunite
Unionize the mega query - It needed more performance-based benefits
2015-10-15 13:17:01 -04:00
Joseph Schorr
c9daf7d8a9 Add additional tests for repo visibility and further simplify the query for perf 2015-10-15 12:12:57 -04:00
Jimmy Zelinskie
7c1547221d raise a 520 for any GitLab timeouts 2015-10-13 17:34:08 -04:00
Jimmy Zelinskie
9818481b08 limit logs to a maximum number of pages 2015-10-06 14:13:23 -04:00
josephschorr
3e7a95407b Merge pull request #598 from coreos-inc/limitbadquery
Prevent unlimited insane query from running and fix tests
2015-10-05 21:29:35 -04:00
Silas Sewell
c6da322ec1 Merge pull request #597 from coreos-inc/tag-validation
Update tag validation
2015-10-05 21:10:55 -04:00
Silas Sewell
dd3d939b31 Update tag validation
Fixes #536
2015-10-05 19:32:10 -04:00
Joseph Schorr
dd804816ba Prevent unlimited insane query from running and fix tests
Fixes #591
2015-10-05 17:11:49 -04:00
Joseph Schorr
8ca92d6828 Remove old search API and switch V1 search to use the new search system 2015-10-05 14:36:43 -04:00
Joseph Schorr
f393236c9f Add repo name check to V2
Fixes #592
2015-10-05 14:19:52 -04:00
Joseph Schorr
d0dc8fe45d Add endpoint security tests for the V2 endpoints
Fixes #581
2015-10-02 14:01:12 -04:00
Joseph Schorr
35c35d9913 Load images and storage references in bulk during V1 synthesize
Currently, we perform multiple queries for each layer, making it much slower (especially cross-region)

Fixes #413
2015-09-29 17:53:39 -04:00
Joseph Schorr
f44ca79391 Add _catalog endpoint as specified by V2 API
Fixes #391
2015-09-29 16:02:19 -04:00
Joseph Schorr
decdaa4c79 New tests and small fixes while comparing against the V2 spec
Fixes #391
2015-09-29 15:18:48 -04:00
Silas Sewell
9000169b53 Revert "Merge pull request #491 from jakedt/migratebackp2"
This reverts commit 7ad2522dbe, reversing
changes made to a0b191ffa1.
2015-09-28 16:09:22 -04:00
Joseph Schorr
2e694dd3f0 Move Docker V2 key to be loaded from file or generated on server load
Fixes #394
2015-09-28 15:43:51 -04:00
Joseph Schorr
09f8ad695b Fix resumable upload support and add another test 2015-09-28 12:17:17 -04:00
josephschorr
7ad2522dbe Merge pull request #491 from jakedt/migratebackp2
Migrate image data back phase 2
2015-09-26 15:11:46 -04:00
Joseph Schorr
4dc30d6321 Remove yaml and switch to JSON because yaml is so slow 2015-09-24 16:17:42 -04:00
Joseph Schorr
a283c8d8ec Add a check to ensure repository names are valid according to an extended set of rules.
Fixes #534
2015-09-24 11:55:08 -04:00
Joseph Schorr
49b575afb6 Start refactoring of the trigger system:
- Move each trigger handler into its own file
- Add dictionary helper classes for easier reading and writing of dict-based data
- Extract the web hook payload -> internal representation building for each trigger system
- Add tests for this transformation
- Remove support for Github archived-based building
2015-09-21 16:36:48 -04:00
Jake Moshenko
26cea9a07c Merge remote-tracking branch 'upstream/master' into python-registry-v2 2015-09-17 16:16:27 -04:00
Jake Moshenko
8baacd2741 Migrate old data to new locations, read only new. 2015-09-17 15:47:13 -04:00
josephschorr
c801965626 Merge pull request #492 from coreos-inc/nofreelunch
UI and API fixes for disallowing private repo count abuse
2015-09-16 17:53:11 -04:00
Silas Sewell
a8183ed87b Sample pull_repo events 2015-09-15 18:41:48 -04:00
Joseph Schorr
fbfe7fdb54 Make change repo visibility and create repo raise a 402 when applicable
We now check the user or org's subscription plan and raise a 402 if the user attempts to create/make a repo private over their limit
2015-09-15 14:33:35 -04:00
Jake Moshenko
b56de3355c Migrate data back to Image in preparation for v2 2015-09-15 11:53:31 -04:00
Jake Moshenko
9c3ddf846f Some fixes and tests for v2 auth
Fixes #395
2015-09-10 15:38:57 -04:00
Joseph Schorr
fd3a21fba9 Add Kubernetes configuration provider which writes config to a secret
Fixes #145
2015-09-10 12:19:59 -04:00
Joseph Schorr
88a04441de Extract the config provider into its own sub-module 2015-09-10 12:19:59 -04:00
josephschorr
edef283697 Merge pull request #447 from coreos-inc/ronon
Add support for Dex to Quay
2015-09-10 11:42:01 -04:00
Jimmy Zelinskie
d55ab78fbe fix pagination of tags in API
Fixes #461.
2015-09-09 15:52:21 -04:00
Joseph Schorr
c0286d1ac3 Add support for Dex to Quay
Fixes #306

- Adds support for Dex as an OAuth external login provider
- Adds support for OIDC in general
- Extract out external logins on the JS side into a service
- Add a feature flag for disabling direct login
- Add support for directing to the single external login service
- Does *not* yet support the config in the superuser tool
2015-09-04 17:05:06 -04:00
Joseph Schorr
48cf33a8c1 Add missing superuser aggregate logs endpoint
Reference: d47d28ea4e/Screen-Shot-2015-09-04-at-11-04-41.png
2015-09-04 16:48:32 -04:00
Jake Moshenko
210ed7cf02 Merge remote-tracking branch 'upstream/master' into python-registry-v2 2015-09-04 16:32:01 -04:00
Jake Moshenko
8269d4ac90 Checkpoint implementing PATCH according to Docker 2015-09-03 16:26:02 -04:00
Joseph Schorr
42dba8655c Fix auth and add V2 tests! 2015-09-03 12:21:21 -04:00
Matt Jibson
a821ad2b01 Return an error on failed S3 uploads
The previous change to this file didn't raise the error up to stream_write,
and so the complete_upload function still ran because the loop was only
broken. It errored because the data was already canceled. This is better
than what we had before, which was to silently fail but report success
(even internally to ourselves!) on bad image upload.

This means we discovered a bug where a user could have failed during image
upload, but quay would write that image to the repository, potentially
writing broken images to S3.
2015-09-01 15:53:32 -04:00
josephschorr
62ea4a6cf4 Merge pull request #191 from coreos-inc/carmen
Add automatic storage replication
2015-09-01 15:04:36 -04:00
Joseph Schorr
724b1607d7 Add automatic storage replication
Adds a worker to automatically replicate data between storages and update the database accordingly
2015-09-01 14:53:32 -04:00
Joseph Schorr
51c507d02d Add back the ability to retrieve information for an org member directly
Fixes #427
2015-08-31 16:45:24 -04:00
Jake Moshenko
398202e6fc Implement some new methods on the storage engines. 2015-08-27 11:29:19 -04:00
josephschorr
4fa37a46d1 Merge pull request #408 from coreos-inc/python-registry-v2-notfun
Change to return a proper manifest URL
2015-08-26 16:17:25 -04:00
josephschorr
9d1296da38 Merge pull request #407 from coreos-inc/python-registry-v2-pushevent
Spawn the repo_push event in V2
2015-08-26 16:13:02 -04:00
Joseph Schorr
9842857b89 Change to return a proper manifest URL 2015-08-26 14:14:59 -04:00
Joseph Schorr
aa90caa97e Spawn the repo_push event in V2
Fixes #403
2015-08-25 16:02:21 -04:00
Joseph Schorr
84458811d5 Rename wrap_with_hash to a more generic wrap_with_handler 2015-08-25 15:53:13 -04:00
Joseph Schorr
767be0798f Fix V1 tests due to a typo
Fixes #397
2015-08-25 14:31:25 -04:00
Joseph Schorr
31fdb94436 Enable rate limiting of V2 requests 2015-08-25 14:18:34 -04:00
Joseph Schorr
f2252f0726 Add a temporary multiplier for the Docker size -> uncompressed size 2015-08-25 13:49:36 -04:00
Joseph Schorr
1450b7e84c Fix verbs support in V2 2015-08-24 12:05:09 -04:00
Joseph Schorr
cf030e2a98 Save the compressed image size on blob upload completion 2015-08-24 12:05:09 -04:00
Joseph Schorr
d246e68e68 Move shared V1/V2 code into common methods and fix verbs 2015-08-24 12:05:09 -04:00
Joseph Schorr
0e93dbb832 Fix image API for V2 images 2015-08-24 12:05:09 -04:00
Jake Moshenko
b998eca8e5 Fix the tests for registry v2 changes. 2015-08-24 11:59:12 -04:00
josephschorr
b8bfed915d Merge pull request #382 from coreos-inc/manytags
Limit the number of branches and tags loaded to 30
2015-08-21 14:36:43 -04:00
Joseph Schorr
b6502d9302 Limit the number of branches and tags loaded to 30
Fixes #380
2015-08-21 14:07:20 -04:00
Matt Jibson
2ea784cd6d Send SSL client certs on webhook notifications
This allows clients that provide a HTTPS webook endpoint a way to verify
that the source of the notification came from quay.io. Needed for the
kubernetes auto deployer so it can verify the request. And apparently
others have also wanted this.
2015-08-19 13:53:34 -04:00
Joseph Schorr
dead054b43 Fix image API for V2 images 2015-08-19 13:35:47 -04:00
Jimmy Zelinskie
523dc912f7 Merge pull request #372 from coreos-inc/notifyui
Better notifications UI
2015-08-17 17:13:24 -04:00
Jimmy Zelinskie
239f76d39f Merge pull request #368 from coreos-inc/buildarchive
Allow builds to be started with an external archive URL
2015-08-17 17:09:14 -04:00
Joseph Schorr
84276ee945 Better notifications UI
Fixes #369
2015-08-17 17:08:58 -04:00
Joseph Schorr
f092c00621 Allow builds to be started with an external archive URL
Fixes #114
2015-08-17 17:01:49 -04:00
Jake Moshenko
2fd1d5969e Merge pull request #351 from mjibson/more-metrics
More metrics
2015-08-17 13:09:08 -04:00
Joseph Schorr
bc4e07343e Fix PATCH semantics due to recent API changes 2015-08-13 15:35:37 -04:00
Joseph Schorr
f50ea3329a Fix URL scheme on the authenticate header 2015-08-13 14:44:42 -04:00
Joseph Schorr
17d3bed835 Fix small NPE in BB trigger code 2015-08-13 12:28:48 -04:00
Jake Moshenko
e1b3e9e6ae Another huge batch of registry v2 changes
Add patch support and resumeable sha
Implement all actual registry methods
Add a simple database generation option
2015-08-12 16:41:12 -04:00
Matt Jibson
b483209862 Wrap API and registry requests with common metric timings
Record response times, codes, and rollup non-2XX responses.
2015-08-12 12:16:00 -04:00
Joseph Schorr
09bb98f161 Really fix the build trigger schema and add a test for it 2015-08-11 17:17:18 -04:00
Jimmy Zelinskie
7d6c6ba8e8 Merge pull request #316 from jzelinskie/read-tags
tag history requires READ instead of WRITE
2015-08-07 13:26:04 -04:00
Joseph Schorr
7ea4c7d17e Add missing JSON schema for 'refs' and 'branch_name'
Fixes #325
2015-08-07 13:01:49 -04:00
Joseph Schorr
aab8866345 Fix accidental rename of key 2015-08-06 13:21:52 -04:00
Joseph Schorr
ea25538646 MySQL and Postgres complain about the group by, so calculate dates ourselves 2015-08-06 12:52:55 -04:00
Joseph Schorr
d34afde954 Fix logs view and API
- We needed to use an engine-agnostic way to extract the days
- Joining with the LogEntryKind table has *horrible* performance in MySQL, so do it ourselves
- Limit to 50 logs per page
2015-08-05 17:47:03 -04:00
Jimmy Zelinskie
411d4e7abb tag history requires READ instead of WRITE
Fixes #315.
2015-08-05 17:09:34 -04:00
Jake Moshenko
65f982577d Merge pull request #289 from coreos-inc/swaggerfix
Fix swagger errors
2015-08-04 10:23:05 -04:00
Jimmy Zelinskie
b5b6076b4a Merge pull request #303 from coreos-inc/docsearch
Add documentation search to the main search bar
2015-08-03 17:19:48 -04:00
Joseph Schorr
8a8955d234 Add documentation search to the main search bar 2015-08-03 17:15:53 -04:00
josephschorr
f772bd0c9e Merge pull request #300 from coreos-inc/toomanyutils
Refactor the util directory to use subpackages.
2015-08-03 16:18:55 -04:00
Jake Moshenko
18100be481 Refactor the util directory to use subpackages. 2015-08-03 16:04:19 -04:00
Jimmy Zelinskie
8e6a0fbbee Merge pull request #294 from coreos-inc/logsload
Switch to using an aggregated logs query and infinite scrolling
2015-08-03 14:52:04 -04:00
Joseph Schorr
5c1d195a19 Fix swagger errors
Fixes #287
2015-08-03 14:10:15 -04:00
Joseph Schorr
3d6c92901c Switch to using an aggregated logs query and infinite scrolling
This should allow users to work with large logs set.

Fixes #294
2015-07-31 16:38:02 -04:00
Joseph Schorr
4160b720f9 UI and API improvements for working with large repositories
- Change the tag check bar to only select the current page (by default), but allow for selecting ALL tags
- Limit the number of tags compared in the visualization view to 10
- Fix the multiselect dropdown to limit itself to 10 items selected
- Remove saving the selected tags in the URL, as it is too slow and overloads the URLs in Chrome when there are 1000+ tags selected
- Change the images API to not return locations: By skipping the extra join and looping, it made the /images API call 10x faster (in hand tests)

Fixes #292
Fixes #293
2015-07-31 16:31:29 -04:00
Joseph Schorr
a916177c16 Fix broken imports 2015-07-29 18:28:58 -04:00
Joseph Schorr
572d6ba53c Fix broken tests 2015-07-29 14:21:29 -04:00
Joseph Schorr
11c7994398 Fix 500 on logout 2015-07-28 15:47:04 -04:00
Joseph Schorr
80e2739b41 Manually load GHE org repos if none returned by default
Fixes #276
2015-07-28 14:01:22 -04:00
Joseph Schorr
5d243bb45f Fix potential NPE 2015-07-24 12:12:30 -04:00
Joseph Schorr
c3f269ee23 Add migration for BitBucket web hooks
This needs to added only *after* we roll out #255
2015-07-23 14:45:12 -04:00
Joseph Schorr
f6311b09fe Fix NPE in BitBucket V2 handling code 2015-07-23 14:06:35 -04:00
Jimmy Zelinskie
fee8bf8607 Merge pull request #255 from coreos-inc/betterbb
Change to use the new BitBucket webhooks
2015-07-23 13:36:07 -04:00
Joseph Schorr
3c6f13da56 Change to use the new BitBucket webhooks
BitBucket has deprecated services (which will be removed in approx 6 months), and they don't even really work all that well anyway.

Fixes #251
2015-07-23 12:09:17 -04:00
Jimmy Zelinskie
2c29dc048d Merge pull request #260 from coreos-inc/githubfix
GitHub api now returns ALL the visible repositories for user
2015-07-22 13:53:58 -04:00
Joseph Schorr
69ca34161c GitHub api now returns ALL the visible repositories for user
Change the code to simply filter the single list returned, rather than reloading the org repos again
2015-07-22 13:50:46 -04:00
Joseph Schorr
687bab1c05 Support invite codes for verification of email
Also changes the system so we don't apply the invite until it is called explicitly from the frontend

Fixes #241
2015-07-22 13:41:27 -04:00
Jake Moshenko
5d86fa80e7 Merge pull request #197 from coreos-inc/keystone
Add Keystone Auth
2015-07-22 13:38:47 -04:00
Joseph Schorr
38a6b3621c Automatically link the superuser account to federated service for auth
When the user commits the configuration, if they have chosen a non-DB auth system, we now auto-link the superuser account to that auth system, to ensure they can login again after restart.
2015-07-22 13:37:23 -04:00
Joseph Schorr
a0c4e72f13 Clean up the repository list API and loads stars with it
We load stars with the same list API now so that we get the extra metadata needed in the repo list (popularity and last modified)
2015-07-22 13:05:02 -04:00
Joseph Schorr
a397bb7af6 Update to new-style format 2015-07-21 15:39:23 -04:00
Joseph Schorr
60b05a0493 Fix string concats
Issue found here: https://app.getsentry.com/quayio/production/group/72831901/
2015-07-21 15:04:50 -04:00
Joseph Schorr
5f2729f41f Fix logic bug in param check 2015-07-20 17:04:06 -04:00
Jake Moshenko
679044574a Merge pull request #231 from coreos-inc/smallfix
Small API fixes
2015-07-20 13:45:24 -04:00
Joseph Schorr
33b54218cc Refactor the users class into their own files, add a common base class for federated users and add a verify_credentials method which only does the verification, without the linking. We use this in the superuser verification pass 2015-07-20 11:39:59 -04:00
Jake Moshenko
f2508fb48a Have the auth realm url computed based on the request url. 2015-07-17 11:56:15 -04:00
Jake Moshenko
bc29561f8f Fix and templatize the logic for external JWT AuthN and registry v2 Auth.
Make it explicit that the registry-v2 stuff is not ready for prime time.
2015-07-17 11:56:15 -04:00
Jake Moshenko
3efaa255e8 Accidental refactor, split out legacy.py into separate sumodules and update all call sites. 2015-07-17 11:56:15 -04:00