vbatts/quay
Archived
1
0
Fork 0
Code Issues Pull requests Releases Wiki Activity
6168 commits 2 branches 62 tags 205 MiB
Commit graph

1102 commits

Author SHA1 Message Date
Jimmy Zelinskie
609f4fccd8 data.queue: simplify put method 2016-07-14 15:34:22 -04:00
Joseph Schorr
c1e4bf79b7 Fix delete team error message for admin teams 2016-07-11 15:47:05 -04:00
Joseph Schorr
241ebaa084 Fix typo 2016-07-07 15:06:29 -04:00
Joseph Schorr
adaeeba5d0 Allow for multiple user RDNs in LDAP 
Fixes #1600
 2016-07-07 14:46:38 -04:00
Joseph Schorr
e252ee07cb Fix popularity metrics on list repos API 2016-07-06 16:15:54 -04:00
Joseph Schorr
713ba3abaf Further updates to the Prometheus client code 2016-07-01 14:16:51 -04:00
Jake Moshenko
668a8edc50 Refactor prometheus integration 
Move prometheus to SaaS and make it a plugin
Move static callers to use metrics_queue plugin
Change local-docker to support different quay clone dirnames
Change prom_aggregator to use logrus
 2016-07-01 14:16:50 -04:00
Matt Jibson
3d9acf2fff Use prometheus as a metric backend 
This entails writing a metric aggregation program since each worker has its
own memory, and thus own metrics because of python gunicorn. The python
client is a simple wrapper that makes web requests to it.
 2016-07-01 14:16:50 -04:00
Joseph Schorr
117ccda1cf Fix postgres error in SQL query 2016-07-01 13:04:20 -04:00
Joseph Schorr
1eec6f53b2 Fix SQL error with pagination around Repositories 
Fixes #1591
 2016-06-30 17:31:35 -04:00
Joseph Schorr
310ecd11cc Handle user events Redis not working in tutorial 
Also does some basic restyling

Fixes #1586
 2016-06-28 17:04:31 -04:00
Joseph Schorr
853cca35f3 Change repo stats to use the RAC table and a nice UI 2016-06-22 15:06:53 -04:00
josephschorr
7173d53030 Merge pull request #1549 from coreos-inc/certs 
Switch to install custom LDAP cert by name
 2016-06-21 15:13:44 -04:00
Joseph Schorr
66ec1d81ce Switch to install custom LDAP cert by name 2016-06-21 15:10:26 -04:00
josephschorr
9e6a264f5f Merge pull request #1523 from coreos-inc/verb-tag-cache-fix 
Add a uniqueness hash to derived image storage to break caching over …
 2016-06-20 16:38:25 -04:00
Joseph Schorr
a43b741f1b Add a uniqueness hash to derived image storage to break caching over tags 
This allows converted ACIs and squashed images to be unique based on the specified tag.

Fixes #92
 2016-06-20 16:34:52 -04:00
Joseph Schorr
3b994431eb Auto expire the build status and logs in redis 2016-06-20 13:53:13 -04:00
Joseph Schorr
986d20bcad Switch to generic RedisError 
Fixes #1558
 2016-06-20 11:20:17 -04:00
Jake Moshenko
a1cf12e460 Add a sitemap.txt for popular public repos 
and reference it from the robots.txt
 2016-06-17 14:34:20 -04:00
josephschorr
614b9124ae Merge pull request #1512 from coreos-inc/optimize-queries 
Optimize various queries
 2016-06-16 14:22:59 -04:00
josephschorr
58bef472d9 Merge pull request #1526 from coreos-inc/superuser-grant 
Add ability for super users to take ownership of namespaces
 2016-06-13 16:23:10 -04:00
Joseph Schorr
20816804e5 Add ability for super users to take ownership of namespaces 
Fixes #1395
 2016-06-13 16:22:52 -04:00
josephschorr
bda5d7ae29 Merge pull request #1511 from coreos-inc/location-cache 
Use a cache for ImageStorageLocation
 2016-06-09 14:03:07 -04:00
Joseph Schorr
7aa6b812e2 Use a cache for ImageStorageLocation 
No need to reload it from the DB or join as it is a static set only changed during migration
 2016-06-09 14:02:42 -04:00
Joseph Schorr
8887f09ba8 Use the instance service key for registry JWT signing 2016-06-07 11:58:10 -04:00
Joseph Schorr
894b5fed6f Remove TODO since we always need storage 2016-06-03 13:45:13 -04:00
Joseph Schorr
03fd2ea15a Remove Image from _load_tag_manifests query 
Doesn't appear used or necessary
 2016-06-03 13:44:01 -04:00
Joseph Schorr
9a747ca6a0 Have get_parent_images not join on placements 
The only case that needs the placements is in verbs, for which we use a new method
 2016-06-03 13:33:15 -04:00
Joseph Schorr
8064419715 Remove Image join from get_active_tag 
It isn't used anywhere in the query and appears to be completely unnecessary
 2016-06-03 13:06:57 -04:00
Joseph Schorr
53538f9001 Optimize get_tag_image query 
No caller uses the image placements or locations, so no need to load them.
 2016-06-02 16:36:38 -04:00
josephschorr
cad8746f9d Merge pull request #1502 from coreos-inc/image-replication 
Enable storage replication for V2 and add backfill tool
 2016-06-02 15:02:53 -04:00
Joseph Schorr
12924784ce Enable storage replication for V2 and add backfill tool 
Fixes #1501
 2016-06-02 14:36:08 -04:00
josephschorr
a85c3ebff7 Merge pull request #1457 from coreos-inc/xauth 
Add support for direct granting of OAuth tokens and add tests
 2016-06-01 12:07:12 -04:00
josephschorr
1ddc73416c Merge pull request #1500 from coreos-inc/better-errors 
Better errors
 2016-05-31 15:54:41 -04:00
Jimmy Zelinskie
1f488acf12 data.queue: move name matching clause 2016-05-31 15:44:11 -04:00
Joseph Schorr
04df2410ec Add better errors if Redis is down 
Fixes #1497
 2016-05-31 15:24:36 -04:00
Jimmy Zelinskie
26300d3c8e data.queue: lint 2016-05-27 14:51:19 -04:00
Jimmy Zelinskie
8a5aa65d74 data.queue: limiting before order by rand 2016-05-27 14:44:30 -04:00
Jimmy Zelinskie
44b56ae2cf queue: explicitly declare ordering requirement 
This change defaults the ordering requirement of queue items to be off
and only enables it for the build manager. This should make the queries
for getting queueitems significantly faster for every other use case.
 2016-05-27 14:44:30 -04:00
Joseph Schorr
7933aecf25 Add support for direct granting of OAuth tokens and add tests 
This allows a client (when authorized in a whitelist) to send direct credentials via a Basic auth header and therefore bypass the OAuth approval UI for that user.
 2016-05-23 17:17:06 -04:00
Joseph Schorr
60bbca2185 Fix setup tool when binding to external auth 
We now query the external auth provider for the external service's identifier before adding the linking row into the database. This fixes the case where the external service resolves a different identifier for the same username.

Fixes #1477
 2016-05-23 17:11:36 -04:00
Joseph Schorr
043699cfb3 Always use log entry kind cache 
Fixes #1445
 2016-05-13 15:20:55 -04:00
Jimmy Zelinskie
972e4be811 log: cutoff at the max id past the cutoff_date 
Previously we were using the min, which is always going to be equivalant
to the min id in the table.
 2016-05-10 20:13:10 -07:00
Joseph Schorr
a736407611 Fix user:admin scope handling and add test 2016-05-09 11:16:01 +02:00
josephschorr
f55fd2049f Merge pull request #1433 from coreos-inc/ldapoptions 
Add additional options for LDAP
 2016-05-04 14:06:29 -04:00
Joseph Schorr
42515ed9ec Add additional options for LDAP 
Fixes #1420
 2016-05-04 13:59:20 -04:00
Joseph Schorr
6e2df3b339 Fix key server to not list expired keys 
Fixes the key server to not list expire keys and by default not return expired or unapproved keys unless explicitly requested.

Fixes #1430
 2016-05-03 17:58:47 -04:00
Jimmy Zelinskie
2aa88dcb80 only send notifications when superusers enabled 2016-04-29 15:42:25 -04:00
Jimmy Zelinskie
29e2d7c9d4 data.model.log: remove unused method 2016-04-29 14:22:53 -04:00
Jimmy Zelinskie
e47b29a974 migration: add missing delete from down migration 
This also reorganizes the file a bit.
 2016-04-29 14:10:33 -04:00
Jimmy Zelinskie
4a521f5844 database: revert logentry foreign key proxy 2016-04-29 14:10:33 -04:00
Evan Cordell
489752a0b7 Only refresh current instance service key 2016-04-29 14:10:33 -04:00
Evan Cordell
a6f6a114c2 service key worker to refresh automatic keys 2016-04-29 14:10:33 -04:00
Evan Cordell
2242c6773d Add 'Automatic' ServiceKeyApprovalType 2016-04-29 14:10:33 -04:00
Joseph Schorr
6091db983b Hide expired keys outside of their staleness window 2016-04-29 14:10:33 -04:00
Joseph Schorr
4f63a50a17 Change account-less logs to use a user and not null 
This allows us to skip the migration
 2016-04-29 14:09:37 -04:00
Jimmy Zelinskie
5cb6ba4d12 keyserver migration: fix constraint name 2016-04-29 14:09:37 -04:00
Joseph Schorr
28a80ef6a9 Make sure to verify service names on key creation 2016-04-29 14:09:37 -04:00
Joseph Schorr
522cf68c5d Lots of smaller fixes: 
- Add the rotation_duration to the keys API
- Have the key service UI use the new rotation_duration field
- Fix notification deletion lookup path
- Add proper support for the new notification in the UI
- Only delete expired keys after 7 days (configurable)
- Fix angular digest loop
- Fix unit tests
- Regenerate initdb
 2016-04-29 14:05:16 -04:00
Jimmy Zelinskie
370ac3ecd0 service keys: add rotation_duration field 2016-04-29 14:05:16 -04:00
Jimmy Zelinskie
6577ac3e62 mv JWK-canonicalization util.security.fingerprint 2016-04-29 14:05:16 -04:00
Jimmy Zelinskie
4020ab9f55 service keys: delete notifications by prefix 2016-04-29 14:05:16 -04:00
Joseph Schorr
08017c5111 Further UI updates 2016-04-29 14:05:16 -04:00
Joseph Schorr
a4a01e76c0 Fix up the migration to include the additional changes needed 2016-04-29 14:05:16 -04:00
Jimmy Zelinskie
d19eb16b45 keyserver: add generate key function 
The superuser API, initdb, and tests will all need this functionality.
 2016-04-29 14:05:16 -04:00
Joseph Schorr
fb1dca4e94 Add API usage tests 2016-04-29 14:05:16 -04:00
Joseph Schorr
11ff3e9b59 keys ui WIP 2016-04-29 14:05:16 -04:00
Jimmy Zelinskie
885a41e6f5 key server: misc fixes to make jwtproxy work 2016-04-29 13:38:25 -04:00
Jimmy Zelinskie
5cdc7812dc migration.sh: update to reflect timing 2016-04-29 13:38:25 -04:00
Jimmy Zelinskie
50ad1bb6b1 key server: misc cleanup to get it working 2016-04-29 13:38:25 -04:00
Jimmy Zelinskie
c79bb14049 service keys: fix stale query 2016-04-29 13:38:25 -04:00
Jimmy Zelinskie
86df0124c1 service keys: join with approvals 
Also fixes a bug where we weren't reassigning the query after adding a
WHERE.
 2016-04-29 13:38:25 -04:00
Jimmy Zelinskie
dff59b4a39 service key migration 2016-04-29 13:38:25 -04:00
Jimmy Zelinskie
c6b8b3ce8c service_keys: s/get_keys/list_keys 2016-04-29 13:38:25 -04:00
Jimmy Zelinskie
0ec54fc70e clear notifications on delete/replace service_key 2016-04-29 13:38:25 -04:00
Jimmy Zelinskie
42b5196b21 add notification path and use for service keys 2016-04-29 13:38:25 -04:00
Jimmy Zelinskie
f406942984 converging on proper rotation 2016-04-29 13:38:25 -04:00
Jimmy Zelinskie
aaf9e83278 basically finish superuser key api 2016-04-29 13:38:25 -04:00
Jimmy Zelinskie
35ed73e195 rework superuser api 2016-04-29 13:38:25 -04:00
Jimmy Zelinskie
4079dba167 service keys: do all the right stuff 2016-04-29 13:38:25 -04:00
Jimmy Zelinskie
6ecff950ab service keys: add txs and select4update 2016-04-29 13:38:25 -04:00
Jimmy Zelinskie
499bb16306 service key server wip 2016-04-29 13:38:25 -04:00
josephschorr
9e88b1413d Merge pull request #1325 from coreos-inc/blobuncompressedsize 
Fix uncompressed size for blob store and add test
 2016-04-28 13:15:33 -04:00
Jimmy Zelinskie
7239c465bf improve stale cutoff id perf (#1392) 2016-04-20 15:03:06 -04:00
josephschorr
b0cc55276f Merge pull request #1373 from coreos-inc/orgconvert 
Org conversion improvements
 2016-04-19 16:16:35 -04:00
Jimmy Zelinskie
5585e16c90 Merge pull request #1356 from jzelinskie/actionlogarchive 
logrotateworker: save to storage via userfiles
 2016-04-15 13:57:11 -04:00
Jimmy Zelinskie
3d190b786f userfiles: make handler optional 2016-04-15 13:56:07 -04:00
Joseph Schorr
c604dbd0f6 Fix permissions when converting a user to an org 
Fixes #1366
 2016-04-14 17:39:45 -04:00
Joseph Schorr
1009362d26 Have recovery auto-verify the user 
Fixes #1355
 2016-04-08 13:41:16 -04:00
Jake Moshenko
bd5b44cbd2 Move the sequence fixer to a separate tool which can be run 2016-04-01 13:46:13 -04:00
josephschorr
edb157c5cb Merge pull request #1294 from coreos-inc/partialperms 
Change permissions to only load required by default
 2016-03-30 16:40:40 -04:00
Joseph Schorr
42e934d84f Make notification lookup faster and fix repo pagination on Postgres 2016-03-30 14:46:31 -04:00
Joseph Schorr
0dffdb87c9 Fix uncompressed size for blob store and add test 2016-03-29 14:16:56 -04:00
Joseph Schorr
a3aa4592cf Change permissions to only load required by default 
Permissions now load just the namespace and/or repository permissions requested, with a fallback to a full permissions load if necessary.
 2016-03-28 16:33:32 -04:00
Jake Moshenko
eed07722cb Add even larger plans for enterprises on SaaS 2016-03-21 16:38:34 -04:00
Jake Moshenko
fe2cd240bc Revert "Remove old search API which is no longer in use" 2016-03-07 10:07:41 -05:00
Jimmy Zelinskie
b5d904f373 Merge pull request #1218 from jzelinskie/logrotate5ever 
vastly simplify log rotation
 2016-03-04 13:48:21 -05:00
josephschorr
57430a18b4 Merge pull request #1224 from coreos-inc/removeoldsearch 
Remove old search API which is no longer in use
 2016-03-04 12:05:07 -05:00
Joseph Schorr
f498e92d58 Implement against new Clair paginated notification system 2016-02-25 15:58:42 -05:00
Joseph Schorr
c0374d71c9 Refactor the security worker and API calls and add a bunch of tests 2016-02-25 12:29:41 -05:00
Quentin Machu
0183c519f7 Merge pull request #1253 from Quentin-M/clair2 
Adapt securityworker, secscan API and Quay UI for Clair 1.0
 2016-02-19 18:21:25 -05:00
Quentin Machu
e5da33578c Adapt security worker for Clair v1.0 (except notifications) 2016-02-19 17:44:14 -05:00
Joseph Schorr
abd2e3c234 V1 Docker ID <-> V2 layer SHA mismatch fix 
Fix handling of V1 Docker ID <-> V2 layer SHA mismatch by dynamically rewriting the manifest to use new synthesized IDs for all layers above the mismatch. Also adds a bunch of tests for this and other use cases, fixes a bug around manifest digest uniqueness and fixes the 5.5 migration for MySQL.
 2016-02-12 17:39:27 +02:00
josephschorr
a9c64545fa Merge pull request #1228 from coreos-inc/v2storagevalidation 
Add a check that will fail if we try to mislink V1 layers
 2016-02-11 22:49:33 +02:00
Joseph Schorr
27f1cc0a13 Add a check that will fail if we try to mislink V1 layers 
Also logs some useful information
 2016-02-11 22:40:00 +02:00
Jake Moshenko
59a6f5bc77 Replace incompatible MySQL 5.5 server_default 2016-02-11 15:07:16 -05:00
Joseph Schorr
1887dc879c Remove old search API which is no longer in use 2016-02-10 15:02:27 +02:00
Jimmy Zelinskie
ee705fe7a9 vastly simplify log rotation 2016-02-09 18:20:14 -05:00
Joseph Schorr
4e771e667f Change sec scan candidate query to match parents to the expected version only 2016-02-09 22:23:48 +02:00
Joseph Schorr
534ec9cb2b Add pagination to the repository list API to make it better for public 
Fixes #1166
 2016-02-01 22:42:44 +02:00
Joseph Schorr
1536709c02 Small fixes 2016-01-29 20:01:17 +02:00
Joseph Schorr
a80ac8eabb Fix import for alembic 2016-01-29 17:59:23 +02:00
Joseph Schorr
bd0a098282 Add ID-based pagination to logs using new decorators and an encrypted token 
Fixes #599
 2016-01-26 12:50:48 -05:00
Jimmy Zelinskie
e54b86c6eb s/TORRENT/BITTORRENT 2016-01-22 15:52:28 -05:00
Jake Moshenko
fe2bdeb6cb Require some data from all models in initdb 2016-01-19 15:30:27 -05:00
Jake Moshenko
1b392dcb9a Remove dependent signatures before removing image storages 2016-01-19 14:56:02 -05:00
Jake Moshenko
7d0be20842 Formatting and syntax improvements 2016-01-19 14:56:02 -05:00
Joseph Schorr
e4da61a05d Fix piece hash calculation 2016-01-12 17:44:19 -05:00
josephschorr
047c2c2c0f Merge pull request #1129 from coreos-inc/backfill 
Add checksum and torrent info backfill
 2016-01-12 14:20:58 -05:00
Jake Moshenko
96c72e73df Clean up torrents before removing referenced storages 2016-01-12 11:43:07 -05:00
Jake Moshenko
8ab6c8a22d Fix torrent hash generation to work in mixed stacks 2016-01-11 16:43:46 -05:00
Joseph Schorr
c36a7c21c8 Order sadly matters with this check in peewee 2016-01-11 15:10:46 -05:00
Joseph Schorr
bd715c0c71 Add checksum and torrent info backfill 2016-01-08 17:32:30 -05:00
Jake Moshenko
1ae101c917 Address torrent feature review comments. 2016-01-08 16:38:21 -05:00
Jake Moshenko
073b68cf0d Fix torrent migration and update backfill to compute torrent pieces 2016-01-08 11:15:34 -05:00
Jake Moshenko
77aa58996a Fix the db definition for torrentinfo and add migration 2016-01-06 14:04:03 -05:00
Jake Moshenko
fd1e5f2407 Remove an unnecessary outer join 2016-01-05 14:43:40 -05:00
Jake Moshenko
8f80d7064b Hash v1 uploads for torrent chunks 2016-01-05 14:43:40 -05:00
Jake Moshenko
8d5f4466d6 Cleanup some indentation and imports 2016-01-05 12:12:57 -05:00
Jimmy Zelinskie
9b0a84c02f implement get_torrent_info 2016-01-04 16:17:51 -05:00
Jake Moshenko
a9b7ac6b48 Rotate robot user uuid when the credentials change 2016-01-04 16:17:51 -05:00
Jake Moshenko
ce8fcbeaae Update the pieces to use base64 encoded binary 2016-01-04 16:17:51 -05:00
Jake Moshenko
5c6e033d21 Fix indentation 2016-01-04 16:17:51 -05:00
Jake Moshenko
fe87d3c796 Hash and track layer file chunks for torrenting 2016-01-04 16:17:51 -05:00
josephschorr
f748d4348d Merge pull request #1106 from coreos-inc/billingemail 
Add support for custom billing invoice email address
 2016-01-04 14:34:30 -05:00
Joseph Schorr
31a8a0fba4 Better UX when recovering organization emails 
Fixes #291
 2015-12-28 15:25:31 -05:00
Joseph Schorr
10efa96009 Add support for custom billing invoice email address 
Fixes #782
 2015-12-28 13:59:50 -05:00
Joseph Schorr
01723d5546 Catch other cases where the queue item has been removed 
Fixes #1096
 2015-12-22 15:58:51 -05:00
Jake Moshenko
9c1a2e7e1b Improve performance by removing unnecessary group by fields 2015-12-22 11:35:49 -05:00
josephschorr
5ac7369bf5 Merge pull request #1068 from coreos-inc/slowqueryfix 
Remove check for derived image storages on image storage
 2015-12-18 16:32:22 -05:00
Joseph Schorr
94ece129d4 Remove remaining recursive queries on repo delete and add test 2015-12-18 16:04:03 -05:00
josephschorr
16f814b7d9 Merge pull request #1075 from coreos-inc/userdeletefix 
Fix user deletion under MySQL
 2015-12-17 15:09:18 -05:00
Joseph Schorr
2e7835c372 Fix user deletion under MySQL 
Fixes #973
 2015-12-17 15:05:15 -05:00
Jimmy Zelinskie
e1f955a3f6 add a log rotation worker 
Fixes #609.
 2015-12-16 17:22:28 -05:00
Joseph Schorr
f59f4e51e8 Remove check for derived image storages on image storage 
Derived image storages are now 1-to-1 with image storages, so we know they have already been removed at this point

Fixes #1067
 2015-12-16 13:41:25 -05:00
Joseph Schorr
73531d08b5 Add server default for the chunk_count column 2015-12-15 15:44:33 -05:00
Joseph Schorr
141f664bf7 Fix subquery delete which messes up MySQL 
Fixes #1061
 2015-12-15 13:15:10 -05:00
Joseph Schorr
9698d6f6a0 Add created column to blob upload 
Fixes first half of #1054
 2015-12-14 15:27:48 -05:00
Joseph Schorr
54095eb5cb Handle the common case of one chunk when calculating the uncompressed size 
Reference #992
 2015-12-14 15:27:48 -05:00
josephschorr
94effb5aaa Merge pull request #1023 from coreos-inc/getblobopt 
Optimize blob lookup
 2015-12-04 16:11:28 -05:00