1545 commits

Author	SHA1	Message	Date
Matt Jibson	3d9acf2fff	Use prometheus as a metric backend ... This entails writing a metric aggregation program since each worker has its own memory, and thus own metrics because of python gunicorn. The python client is a simple wrapper that makes web requests to it.	2016-07-01 14:16:50 -04:00
Joseph Schorr	117ccda1cf	Fix postgres error in SQL query	2016-07-01 13:04:20 -04:00
Joseph Schorr	1eec6f53b2	Fix SQL error with pagination around Repositories ... Fixes #1591	2016-06-30 17:31:35 -04:00
Joseph Schorr	310ecd11cc	Handle user events Redis not working in tutorial ... Also does some basic restyling Fixes #1586	2016-06-28 17:04:31 -04:00
Joseph Schorr	853cca35f3	Change repo stats to use the RAC table and a nice UI	2016-06-22 15:06:53 -04:00
josephschorr	7173d53030	Merge pull request #1549 from coreos-inc/certs ... Switch to install custom LDAP cert by name	2016-06-21 15:13:44 -04:00
Joseph Schorr	66ec1d81ce	Switch to install custom LDAP cert by name	2016-06-21 15:10:26 -04:00
josephschorr	9e6a264f5f	Merge pull request #1523 from coreos-inc/verb-tag-cache-fix ... Add a uniqueness hash to derived image storage to break caching over …	2016-06-20 16:38:25 -04:00
Joseph Schorr	a43b741f1b	Add a uniqueness hash to derived image storage to break caching over tags ... This allows converted ACIs and squashed images to be unique based on the specified tag. Fixes #92	2016-06-20 16:34:52 -04:00
Joseph Schorr	3b994431eb	Auto expire the build status and logs in redis	2016-06-20 13:53:13 -04:00
Joseph Schorr	986d20bcad	Switch to generic RedisError ... Fixes #1558	2016-06-20 11:20:17 -04:00
Jake Moshenko	a1cf12e460	Add a sitemap.txt for popular public repos ... and reference it from the robots.txt	2016-06-17 14:34:20 -04:00
josephschorr	614b9124ae	Merge pull request #1512 from coreos-inc/optimize-queries ... Optimize various queries	2016-06-16 14:22:59 -04:00
josephschorr	58bef472d9	Merge pull request #1526 from coreos-inc/superuser-grant ... Add ability for super users to take ownership of namespaces	2016-06-13 16:23:10 -04:00
Joseph Schorr	20816804e5	Add ability for super users to take ownership of namespaces ... Fixes #1395	2016-06-13 16:22:52 -04:00
josephschorr	bda5d7ae29	Merge pull request #1511 from coreos-inc/location-cache ... Use a cache for ImageStorageLocation	2016-06-09 14:03:07 -04:00
Joseph Schorr	7aa6b812e2	Use a cache for ImageStorageLocation ... No need to reload it from the DB or join as it is a static set only changed during migration	2016-06-09 14:02:42 -04:00
Joseph Schorr	8887f09ba8	Use the instance service key for registry JWT signing	2016-06-07 11:58:10 -04:00
Joseph Schorr	894b5fed6f	Remove TODO since we always need storage	2016-06-03 13:45:13 -04:00
Joseph Schorr	03fd2ea15a	Remove Image from _load_tag_manifests query ... Doesn't appear used or necessary	2016-06-03 13:44:01 -04:00
Joseph Schorr	9a747ca6a0	Have get_parent_images not join on placements ... The only case that needs the placements is in verbs, for which we use a new method	2016-06-03 13:33:15 -04:00
Joseph Schorr	8064419715	Remove Image join from get_active_tag ... It isn't used anywhere in the query and appears to be completely unnecessary	2016-06-03 13:06:57 -04:00
Joseph Schorr	53538f9001	Optimize get_tag_image query ... No caller uses the image placements or locations, so no need to load them.	2016-06-02 16:36:38 -04:00
josephschorr	cad8746f9d	Merge pull request #1502 from coreos-inc/image-replication ... Enable storage replication for V2 and add backfill tool	2016-06-02 15:02:53 -04:00
Joseph Schorr	12924784ce	Enable storage replication for V2 and add backfill tool ... Fixes #1501	2016-06-02 14:36:08 -04:00
josephschorr	a85c3ebff7	Merge pull request #1457 from coreos-inc/xauth ... Add support for direct granting of OAuth tokens and add tests	2016-06-01 12:07:12 -04:00
josephschorr	1ddc73416c	Merge pull request #1500 from coreos-inc/better-errors ... Better errors	2016-05-31 15:54:41 -04:00
Jimmy Zelinskie	1f488acf12	data.queue: move name matching clause	2016-05-31 15:44:11 -04:00
Joseph Schorr	04df2410ec	Add better errors if Redis is down ... Fixes #1497	2016-05-31 15:24:36 -04:00
Jimmy Zelinskie	26300d3c8e	data.queue: lint	2016-05-27 14:51:19 -04:00
Jimmy Zelinskie	8a5aa65d74	data.queue: limiting before order by rand	2016-05-27 14:44:30 -04:00
Jimmy Zelinskie	44b56ae2cf	queue: explicitly declare ordering requirement ... This change defaults the ordering requirement of queue items to be off and only enables it for the build manager. This should make the queries for getting queueitems significantly faster for every other use case.	2016-05-27 14:44:30 -04:00
Joseph Schorr	7933aecf25	Add support for direct granting of OAuth tokens and add tests ... This allows a client (when authorized in a whitelist) to send direct credentials via a Basic auth header and therefore bypass the OAuth approval UI for that user.	2016-05-23 17:17:06 -04:00
Joseph Schorr	60bbca2185	Fix setup tool when binding to external auth ... We now query the external auth provider for the external service's identifier before adding the linking row into the database. This fixes the case where the external service resolves a different identifier for the same username. Fixes #1477	2016-05-23 17:11:36 -04:00
Joseph Schorr	043699cfb3	Always use log entry kind cache ... Fixes #1445	2016-05-13 15:20:55 -04:00
Jimmy Zelinskie	972e4be811	log: cutoff at the max id past the cutoff_date ... Previously we were using the min, which is always going to be equivalant to the min id in the table.	2016-05-10 20:13:10 -07:00
Joseph Schorr	a736407611	Fix user:admin scope handling and add test	2016-05-09 11:16:01 +02:00
josephschorr	f55fd2049f	Merge pull request #1433 from coreos-inc/ldapoptions ... Add additional options for LDAP	2016-05-04 14:06:29 -04:00
Joseph Schorr	42515ed9ec	Add additional options for LDAP ... Fixes #1420	2016-05-04 13:59:20 -04:00
Joseph Schorr	6e2df3b339	Fix key server to not list expired keys ... Fixes the key server to not list expire keys and by default not return expired or unapproved keys unless explicitly requested. Fixes #1430	2016-05-03 17:58:47 -04:00
Jimmy Zelinskie	2aa88dcb80	only send notifications when superusers enabled	2016-04-29 15:42:25 -04:00
Jimmy Zelinskie	29e2d7c9d4	data.model.log: remove unused method	2016-04-29 14:22:53 -04:00
Jimmy Zelinskie	e47b29a974	migration: add missing delete from down migration ... This also reorganizes the file a bit.	2016-04-29 14:10:33 -04:00
Jimmy Zelinskie	4a521f5844	database: revert logentry foreign key proxy	2016-04-29 14:10:33 -04:00
Evan Cordell	489752a0b7	Only refresh current instance service key	2016-04-29 14:10:33 -04:00
Evan Cordell	a6f6a114c2	service key worker to refresh automatic keys	2016-04-29 14:10:33 -04:00
Evan Cordell	2242c6773d	Add 'Automatic' ServiceKeyApprovalType	2016-04-29 14:10:33 -04:00
Joseph Schorr	6091db983b	Hide expired keys outside of their staleness window	2016-04-29 14:10:33 -04:00
Joseph Schorr	4f63a50a17	Change account-less logs to use a user and not null ... This allows us to skip the migration	2016-04-29 14:09:37 -04:00
Jimmy Zelinskie	5cb6ba4d12	keyserver migration: fix constraint name	2016-04-29 14:09:37 -04:00
Joseph Schorr	28a80ef6a9	Make sure to verify service names on key creation	2016-04-29 14:09:37 -04:00
Joseph Schorr	522cf68c5d	Lots of smaller fixes: ... - Add the rotation_duration to the keys API - Have the key service UI use the new rotation_duration field - Fix notification deletion lookup path - Add proper support for the new notification in the UI - Only delete expired keys after 7 days (configurable) - Fix angular digest loop - Fix unit tests - Regenerate initdb	2016-04-29 14:05:16 -04:00
Jimmy Zelinskie	370ac3ecd0	service keys: add rotation_duration field	2016-04-29 14:05:16 -04:00
Jimmy Zelinskie	6577ac3e62	mv JWK-canonicalization util.security.fingerprint	2016-04-29 14:05:16 -04:00
Jimmy Zelinskie	4020ab9f55	service keys: delete notifications by prefix	2016-04-29 14:05:16 -04:00
Joseph Schorr	08017c5111	Further UI updates	2016-04-29 14:05:16 -04:00
Joseph Schorr	a4a01e76c0	Fix up the migration to include the additional changes needed	2016-04-29 14:05:16 -04:00
Jimmy Zelinskie	d19eb16b45	keyserver: add generate key function ... The superuser API, initdb, and tests will all need this functionality.	2016-04-29 14:05:16 -04:00
Joseph Schorr	fb1dca4e94	Add API usage tests	2016-04-29 14:05:16 -04:00
Joseph Schorr	11ff3e9b59	keys ui WIP	2016-04-29 14:05:16 -04:00
Jimmy Zelinskie	885a41e6f5	key server: misc fixes to make jwtproxy work	2016-04-29 13:38:25 -04:00
Jimmy Zelinskie	5cdc7812dc	migration.sh: update to reflect timing	2016-04-29 13:38:25 -04:00
Jimmy Zelinskie	50ad1bb6b1	key server: misc cleanup to get it working	2016-04-29 13:38:25 -04:00
Jimmy Zelinskie	c79bb14049	service keys: fix stale query	2016-04-29 13:38:25 -04:00
Jimmy Zelinskie	86df0124c1	service keys: join with approvals ... Also fixes a bug where we weren't reassigning the query after adding a WHERE.	2016-04-29 13:38:25 -04:00
Jimmy Zelinskie	dff59b4a39	service key migration	2016-04-29 13:38:25 -04:00
Jimmy Zelinskie	c6b8b3ce8c	service_keys: s/get_keys/list_keys	2016-04-29 13:38:25 -04:00
Jimmy Zelinskie	0ec54fc70e	clear notifications on delete/replace service_key	2016-04-29 13:38:25 -04:00
Jimmy Zelinskie	42b5196b21	add notification path and use for service keys	2016-04-29 13:38:25 -04:00
Jimmy Zelinskie	f406942984	converging on proper rotation	2016-04-29 13:38:25 -04:00
Jimmy Zelinskie	aaf9e83278	basically finish superuser key api	2016-04-29 13:38:25 -04:00
Jimmy Zelinskie	35ed73e195	rework superuser api	2016-04-29 13:38:25 -04:00
Jimmy Zelinskie	4079dba167	service keys: do all the right stuff	2016-04-29 13:38:25 -04:00
Jimmy Zelinskie	6ecff950ab	service keys: add txs and select4update	2016-04-29 13:38:25 -04:00
Jimmy Zelinskie	499bb16306	service key server wip	2016-04-29 13:38:25 -04:00
josephschorr	9e88b1413d	Merge pull request #1325 from coreos-inc/blobuncompressedsize ... Fix uncompressed size for blob store and add test	2016-04-28 13:15:33 -04:00
Jimmy Zelinskie	7239c465bf	improve stale cutoff id perf (#1392)	2016-04-20 15:03:06 -04:00
josephschorr	b0cc55276f	Merge pull request #1373 from coreos-inc/orgconvert ... Org conversion improvements	2016-04-19 16:16:35 -04:00
Jimmy Zelinskie	5585e16c90	Merge pull request #1356 from jzelinskie/actionlogarchive ... logrotateworker: save to storage via userfiles	2016-04-15 13:57:11 -04:00
Jimmy Zelinskie	3d190b786f	userfiles: make handler optional	2016-04-15 13:56:07 -04:00
Joseph Schorr	c604dbd0f6	Fix permissions when converting a user to an org ... Fixes #1366	2016-04-14 17:39:45 -04:00
Joseph Schorr	1009362d26	Have recovery auto-verify the user ... Fixes #1355	2016-04-08 13:41:16 -04:00
Jake Moshenko	bd5b44cbd2	Move the sequence fixer to a separate tool which can be run	2016-04-01 13:46:13 -04:00
josephschorr	edb157c5cb	Merge pull request #1294 from coreos-inc/partialperms ... Change permissions to only load required by default	2016-03-30 16:40:40 -04:00
Joseph Schorr	42e934d84f	Make notification lookup faster and fix repo pagination on Postgres	2016-03-30 14:46:31 -04:00
Joseph Schorr	0dffdb87c9	Fix uncompressed size for blob store and add test	2016-03-29 14:16:56 -04:00
Joseph Schorr	a3aa4592cf	Change permissions to only load required by default ... Permissions now load just the namespace and/or repository permissions requested, with a fallback to a full permissions load if necessary.	2016-03-28 16:33:32 -04:00
Jake Moshenko	eed07722cb	Add even larger plans for enterprises on SaaS	2016-03-21 16:38:34 -04:00
Jake Moshenko	fe2cd240bc	Revert "Remove old search API which is no longer in use"	2016-03-07 10:07:41 -05:00
Jimmy Zelinskie	b5d904f373	Merge pull request #1218 from jzelinskie/logrotate5ever ... vastly simplify log rotation	2016-03-04 13:48:21 -05:00
josephschorr	57430a18b4	Merge pull request #1224 from coreos-inc/removeoldsearch ... Remove old search API which is no longer in use	2016-03-04 12:05:07 -05:00
Joseph Schorr	f498e92d58	Implement against new Clair paginated notification system	2016-02-25 15:58:42 -05:00
Joseph Schorr	c0374d71c9	Refactor the security worker and API calls and add a bunch of tests	2016-02-25 12:29:41 -05:00
Quentin Machu	0183c519f7	Merge pull request #1253 from Quentin-M/clair2 ... Adapt securityworker, secscan API and Quay UI for Clair 1.0	2016-02-19 18:21:25 -05:00
Quentin Machu	e5da33578c	Adapt security worker for Clair v1.0 (except notifications)	2016-02-19 17:44:14 -05:00
Joseph Schorr	abd2e3c234	V1 Docker ID <-> V2 layer SHA mismatch fix ... Fix handling of V1 Docker ID <-> V2 layer SHA mismatch by dynamically rewriting the manifest to use new synthesized IDs for all layers above the mismatch. Also adds a bunch of tests for this and other use cases, fixes a bug around manifest digest uniqueness and fixes the 5.5 migration for MySQL.	2016-02-12 17:39:27 +02:00
josephschorr	a9c64545fa	Merge pull request #1228 from coreos-inc/v2storagevalidation ... Add a check that will fail if we try to mislink V1 layers	2016-02-11 22:49:33 +02:00
Joseph Schorr	27f1cc0a13	Add a check that will fail if we try to mislink V1 layers ... Also logs some useful information	2016-02-11 22:40:00 +02:00
Jake Moshenko	59a6f5bc77	Replace incompatible MySQL 5.5 server_default	2016-02-11 15:07:16 -05:00
Joseph Schorr	1887dc879c	Remove old search API which is no longer in use	2016-02-10 15:02:27 +02:00
Jimmy Zelinskie	ee705fe7a9	vastly simplify log rotation	2016-02-09 18:20:14 -05:00
Joseph Schorr	4e771e667f	Change sec scan candidate query to match parents to the expected version only	2016-02-09 22:23:48 +02:00
Joseph Schorr	534ec9cb2b	Add pagination to the repository list API to make it better for public ... Fixes #1166	2016-02-01 22:42:44 +02:00
Joseph Schorr	1536709c02	Small fixes	2016-01-29 20:01:17 +02:00
Joseph Schorr	a80ac8eabb	Fix import for alembic	2016-01-29 17:59:23 +02:00
Joseph Schorr	bd0a098282	Add ID-based pagination to logs using new decorators and an encrypted token ... Fixes #599	2016-01-26 12:50:48 -05:00
Jimmy Zelinskie	e54b86c6eb	s/TORRENT/BITTORRENT	2016-01-22 15:52:28 -05:00
Jake Moshenko	fe2bdeb6cb	Require some data from all models in initdb	2016-01-19 15:30:27 -05:00
Jake Moshenko	1b392dcb9a	Remove dependent signatures before removing image storages	2016-01-19 14:56:02 -05:00
Jake Moshenko	7d0be20842	Formatting and syntax improvements	2016-01-19 14:56:02 -05:00
Joseph Schorr	e4da61a05d	Fix piece hash calculation	2016-01-12 17:44:19 -05:00
josephschorr	047c2c2c0f	Merge pull request #1129 from coreos-inc/backfill ... Add checksum and torrent info backfill	2016-01-12 14:20:58 -05:00
Jake Moshenko	96c72e73df	Clean up torrents before removing referenced storages	2016-01-12 11:43:07 -05:00
Jake Moshenko	8ab6c8a22d	Fix torrent hash generation to work in mixed stacks	2016-01-11 16:43:46 -05:00
Joseph Schorr	c36a7c21c8	Order sadly matters with this check in peewee	2016-01-11 15:10:46 -05:00
Joseph Schorr	bd715c0c71	Add checksum and torrent info backfill	2016-01-08 17:32:30 -05:00
Jake Moshenko	1ae101c917	Address torrent feature review comments.	2016-01-08 16:38:21 -05:00
Jake Moshenko	073b68cf0d	Fix torrent migration and update backfill to compute torrent pieces	2016-01-08 11:15:34 -05:00
Jake Moshenko	77aa58996a	Fix the db definition for torrentinfo and add migration	2016-01-06 14:04:03 -05:00
Jake Moshenko	fd1e5f2407	Remove an unnecessary outer join	2016-01-05 14:43:40 -05:00
Jake Moshenko	8f80d7064b	Hash v1 uploads for torrent chunks	2016-01-05 14:43:40 -05:00
Jake Moshenko	8d5f4466d6	Cleanup some indentation and imports	2016-01-05 12:12:57 -05:00
Jimmy Zelinskie	9b0a84c02f	implement get_torrent_info	2016-01-04 16:17:51 -05:00
Jake Moshenko	a9b7ac6b48	Rotate robot user uuid when the credentials change	2016-01-04 16:17:51 -05:00
Jake Moshenko	ce8fcbeaae	Update the pieces to use base64 encoded binary	2016-01-04 16:17:51 -05:00
Jake Moshenko	5c6e033d21	Fix indentation	2016-01-04 16:17:51 -05:00
Jake Moshenko	fe87d3c796	Hash and track layer file chunks for torrenting	2016-01-04 16:17:51 -05:00
josephschorr	f748d4348d	Merge pull request #1106 from coreos-inc/billingemail ... Add support for custom billing invoice email address	2016-01-04 14:34:30 -05:00
Joseph Schorr	31a8a0fba4	Better UX when recovering organization emails ... Fixes #291	2015-12-28 15:25:31 -05:00
Joseph Schorr	10efa96009	Add support for custom billing invoice email address ... Fixes #782	2015-12-28 13:59:50 -05:00
Joseph Schorr	01723d5546	Catch other cases where the queue item has been removed ... Fixes #1096	2015-12-22 15:58:51 -05:00
Jake Moshenko	9c1a2e7e1b	Improve performance by removing unnecessary group by fields	2015-12-22 11:35:49 -05:00
josephschorr	5ac7369bf5	Merge pull request #1068 from coreos-inc/slowqueryfix ... Remove check for derived image storages on image storage	2015-12-18 16:32:22 -05:00
Joseph Schorr	94ece129d4	Remove remaining recursive queries on repo delete and add test	2015-12-18 16:04:03 -05:00
josephschorr	16f814b7d9	Merge pull request #1075 from coreos-inc/userdeletefix ... Fix user deletion under MySQL	2015-12-17 15:09:18 -05:00
Joseph Schorr	2e7835c372	Fix user deletion under MySQL ... Fixes #973	2015-12-17 15:05:15 -05:00
Jimmy Zelinskie	e1f955a3f6	add a log rotation worker ... Fixes #609.	2015-12-16 17:22:28 -05:00
Joseph Schorr	f59f4e51e8	Remove check for derived image storages on image storage ... Derived image storages are now 1-to-1 with image storages, so we know they have already been removed at this point Fixes #1067	2015-12-16 13:41:25 -05:00
Joseph Schorr	73531d08b5	Add server default for the chunk_count column	2015-12-15 15:44:33 -05:00
Joseph Schorr	141f664bf7	Fix subquery delete which messes up MySQL ... Fixes #1061	2015-12-15 13:15:10 -05:00
Joseph Schorr	9698d6f6a0	Add created column to blob upload ... Fixes first half of #1054	2015-12-14 15:27:48 -05:00
Joseph Schorr	54095eb5cb	Handle the common case of one chunk when calculating the uncompressed size ... Reference #992	2015-12-14 15:27:48 -05:00
josephschorr	94effb5aaa	Merge pull request #1023 from coreos-inc/getblobopt ... Optimize blob lookup	2015-12-04 16:11:28 -05:00
Jake Moshenko	38cb63d195	Fix indentation on build model operations	2015-12-04 15:46:07 -05:00
Joseph Schorr	f07b940bc5	Optimize blob lookup ... Fixes #1013	2015-12-04 14:47:09 -05:00
Joseph Schorr	c324ebd7f6	Only write exceptions for manifest gen when a tag exists ... Fixes #1019 Currently, we just raise an exception to the logs regardless, which can make it appear as if there is an issue (when there isn't).	2015-12-03 16:04:17 -05:00
Silas Sewell	502e4c04d0	Fix seq migration down_revision	2015-11-30 17:59:04 -05:00
Silas Sewell	3833fb6530	Merge pull request #888 from coreos-inc/remove-hardcoded-ids ... Fix seq generators for enum tables in postgres	2015-11-30 17:54:13 -05:00
Joseph Schorr	0f7c8105b0	Remove DerivedImageStorage table	2015-11-25 11:46:59 -05:00
Joseph Schorr	762cd56e64	Change derived storage to be based on image ... Fixes #971	2015-11-24 12:44:07 -05:00
Jake Moshenko	3a29dfc535	Reducing in a tree to avoid recursion depth limits	2015-11-23 15:57:13 -05:00
Joseph Schorr	f4266d08d2	Fix handling of aggregate size in V2 ... Fixes #931	2015-11-20 11:44:03 -05:00
Joseph Schorr	4981ccbc4e	Fix issue with query when manifest count is 0	2015-11-19 17:44:16 -05:00
Jake Moshenko	c352050b07	For the last time, you can't delete with a subquery on the same table!	2015-11-19 16:44:27 -05:00
Jake Moshenko	7b53797677	Fix garbage collection when manifests may reference tags	2015-11-19 16:01:36 -05:00
Jake Moshenko	7ae94f414c	Alias our subqueries to appease the MySQL beast	2015-11-19 12:58:06 -05:00
Silas Sewell	1162814734	securityworker: mark children we can't analyze ... This allows us to differentiate between images that are queued and those we can't analyze in constant time.	2015-11-19 11:22:15 -05:00
Jake Moshenko	e6bd5488c9	Ensure that manifest tags are still alive	2015-11-19 11:01:47 -05:00
Jake Moshenko	b564492ea7	Improve the performance of fetching manifest blobs by checksum.	2015-11-19 11:01:47 -05:00
Quentin Machu	f2d874386b	Fix security worker (ok last time before I give up on engineering)	2015-11-18 21:21:00 -05:00
Quentin Machu	04f2688944	Merge pull request #917 from Quentin-M/fix_secwor ... Fix security worker (again?)	2015-11-18 19:45:36 -05:00
Quentin Machu	88e85cded0	Fix security worker (again?)	2015-11-18 19:45:09 -05:00
Quentin Machu	6d89f259f5	Merge pull request #894 from Quentin-M/fix_secwor ... Refactor security worker	2015-11-18 14:40:34 -05:00
Quentin Machu	605ed1fc77	Refactor security worker	2015-11-18 14:38:32 -05:00
Jake Moshenko	18b14001b4	Add indices for the security worker fields on Image ... Fixes #906	2015-11-18 13:29:51 -05:00
Jake Moshenko	206e18d160	Image parents do not have to be nulled transitively on repo delete	2015-11-17 16:48:26 -05:00
Jake Moshenko	e252397292	Switch parent back to a ForeignKeyField without a constraint	2015-11-17 16:09:33 -05:00
Jake Moshenko	3374e8c812	Do not constrain deferred fields in SQLAlchemy bridge	2015-11-17 15:55:18 -05:00
Jake Moshenko	ae61ebeac9	The translate placements query was renamed in v2	2015-11-17 12:24:05 -05:00
Jake Moshenko	7205bf5e7f	Merge pull request #885 from jakedt/python-registry-v2 ... Python registry v2 mega merge	2015-11-16 16:15:40 -05:00
Silas Sewell	30b0101584	Fix seq generators for enum tables in postgres ... This attempts to insert a temporary entry into each enum table until it succeeds. It re-synchronizes the postgres sequence generators with the max id of the table. Fixes #883 and #880	2015-11-16 15:29:51 -05:00
Matt Jibson	13aa6cfcfc	No PUT for logarchive ... fixes #862	2015-11-16 15:01:12 -05:00
Jake Moshenko	0459c3bc54	Merge remote-tracking branch 'upstream/master' into python-registry-v2	2015-11-16 14:22:54 -05:00
Jake Moshenko	52125bbfed	Fix gc by using the v1/v2 storage location helper everywhere	2015-11-16 14:13:37 -05:00
Joseph Schorr	819d461ed6	Remove migration re-added by merge accidentally	2015-11-12 22:02:26 -05:00
Joseph Schorr	030c69d7d2	Further merge fixes	2015-11-12 22:00:28 -05:00
Joseph Schorr	7816b0c657	Merge master into vulnerability-tool	2015-11-12 21:52:47 -05:00
Joseph Schorr	25b8b7590f	Fix all the things!	2015-11-12 20:55:41 -05:00
Jake Moshenko	44d06b0c2e	Fix v1 backward compatibility	2015-11-12 16:22:19 -05:00
Jake Moshenko	cf1ec68046	Correlate a specific blob storage with its placements	2015-11-12 16:20:59 -05:00
Jake Moshenko	ab340e20ea	Merge remote-tracking branch 'upstream/master' into python-registry-v2	2015-11-11 16:41:40 -05:00
Jake Moshenko	1d7be74a1f	Revert "Drop the v1 checksum column from imagestorage" ... This reverts commit d292a34343.	2015-11-11 16:39:46 -05:00
Jake Moshenko	a1ccd860e7	Merge pull request #823 from coreos-inc/phase3-11-07-2015 ... Phase3 11 07 2015	2015-11-11 14:22:19 -05:00
Jake Moshenko	1c6919dd93	We must fill in the parent_id on linking	2015-11-10 14:31:46 -05:00
Silas Sewell	e826b14ca4	Merge pull request #725 from coreos-inc/setup-tool-georeplication ... superuser: add storage replication config	2015-11-09 17:43:38 -05:00
Silas Sewell	5000b1621c	superuser: add storage replication config	2015-11-09 17:34:22 -05:00
Joseph Schorr	a69c9e12fd	Update quay sec code to fix problems identified in previous review ... - Change get_repository_images_recursive to operate over a single docker image and storage uuid - Move endpoints/sec to endpoints/secscan - Change notification system to work with new Quay-sec format Fixes #768	2015-11-09 17:14:35 -05:00
Silas Sewell	c739c453da	Merge pull request #807 from coreos-inc/storage-preference ... Enable storage preference	2015-11-09 16:30:47 -05:00
josephschorr	eb2e42dce9	Merge pull request #830 from coreos-inc/fix_parent_id ... Fix deleting repos and images under MySQL	2015-11-09 14:43:01 -05:00
Joseph Schorr	2d2662f53f	Fix deleting repos and images under MySQL ... MySQL doesn't handle constraints at the end of transactions, so deleting images currently fails. This removes the constraint and just leaves parent_id as an int	2015-11-09 14:42:05 -05:00
Matt Jibson	e5282a216f	Merge pull request #818 from mjibson/redis-socket-timeout ... Set timeout for redis commands	2015-11-09 14:39:00 -05:00
Joseph Schorr	b408cfd2cc	Ready for demo	2015-11-09 12:51:05 -05:00
Joseph Schorr	407eaae137	WIP: Towards sec demo	2015-11-09 12:50:39 -05:00
Joseph Schorr	d7ace69fe3	Add a vulnerability_found event for notice when we detect a vuln ... Fixes #637 Note: This PR does *not* actually raise the event; it merely adds support for it	2015-11-09 12:49:19 -05:00
Quentin Machu	37118423a5	Add support for Quay's vulnerability tool	2015-11-09 12:49:19 -05:00
Quentin Machu	ade664036c	Fix migration 57dad559ff2d's header ... Fixes #825	2015-11-08 15:22:15 -05:00
Jake Moshenko	b526e2a3cd	Merge pull request #822 from coreos-inc/phase2-11-07-2015 ... Phase2 11 07 2015	2015-11-08 13:21:59 -05:00
Joseph Schorr	8463514a09	Fix delete_user call to remove all user data	2015-11-08 13:10:01 -05:00
Matt Jibson	afa119d82e	Set timeout for redis commands ... fixes #779	2015-11-06 18:48:47 -05:00
Jake Moshenko	7efa6265bf	Merge branch 'newchanges' into python-registry-v2	2015-11-06 18:24:32 -05:00
Jake Moshenko	c2fcf8bead	Merge remote-tracking branch 'upstream/phase4-11-07-2015' into python-registry-v2	2015-11-06 18:18:29 -05:00
Jake Moshenko	4314882fa0	Reverse the order of get_parent_images	2015-11-06 17:47:08 -05:00
Jake Moshenko	d292a34343	Drop the v1 checksum column from imagestorage	2015-11-06 16:49:16 -05:00
Jake Moshenko	ad93425ead	Stop writing to v1 checksum on ImageStorage	2015-11-06 16:40:04 -05:00
Jake Moshenko	fd3f88f489	Re-enable parent id backfill, use new backfill style	2015-11-06 16:17:12 -05:00
Jake Moshenko	9036ca2f2f	Backfill the v1 checksums from imagestorage	2015-11-06 16:17:12 -05:00
Jimmy Zelinskie	d5e7f6bea7	resolve migration branches and run initdb	2015-11-06 16:10:31 -05:00
Jimmy Zelinskie	f3c3e684a1	prepare branch to be merged into phase1-11-07-2015 ... This removes the checksum backfill, removes the migration that runs the backfills, and defaults the security scan feature off.	2015-11-06 15:22:18 -05:00
Joseph Schorr	0f3db709ea	Add a vulnerability_found event for notice when we detect a vuln ... Fixes #637 Note: This PR does *not* actually raise the event; it merely adds support for it	2015-11-06 15:22:18 -05:00
Quentin Machu	3677947521	Add support for Quay's vulnerability tool	2015-11-06 15:22:18 -05:00
Quentin Machu	a99b8fcfe4	Fix migration	2015-11-06 15:22:18 -05:00
Quentin Machu	f59e35cc81	Add support for Quay's vulnerability tool	2015-11-06 15:22:18 -05:00
Jake Moshenko	3d0bcbaaeb	Move v1 checksums to image and track v2 separately	2015-11-06 15:17:55 -05:00
Joseph Schorr	2b3633b107	Remove the used_legacy_github column	2015-11-06 15:17:55 -05:00
Joseph Schorr	bbf4a1fac4	Remove the used_legacy_github column	2015-11-06 15:17:55 -05:00
Silas Sewell	a7fef8377c	Enable storage preference	2015-11-06 13:34:49 -05:00
Matt Jibson	57ffb39651	Merge pull request #714 from mjibson/queue-locking ... Refactor queue locking to not use select for update	2015-11-04 12:05:53 -05:00
Matt Jibson	a994b367da	Refactor queue locking to not use select for update ... The test suggests this works. fixes #622	2015-11-03 11:32:28 -05:00
Jimmy Zelinskie	c78c450211	UTF-8 v1_json_metadata, comment, manifest ... This will allow us to store unicode JSON blobs in the column on MySQL.	2015-11-02 15:40:19 -05:00
Jake Moshenko	2c10d28afc	Merge remote-tracking branch 'upstream/master' into python-registry-v2	2015-10-26 14:44:16 -04:00
Jake Moshenko	3186311669	Test postgres before mysql variations in migration	2015-10-24 15:00:19 -04:00
Jake Moshenko	b920bf6436	Fix references to mysql in migrations	2015-10-24 15:00:19 -04:00
Jake Moshenko	ddbe33e2ce	Switch Text to LongText for MySQL manifests	2015-10-24 15:00:19 -04:00
Jake Moshenko	cb7ec2f239	Backport remaining v2 changes to phase4	2015-10-24 15:00:13 -04:00
Jake Moshenko	e965ed9f3b	Remove the unused imagestorage columns from the db.	2015-10-24 14:51:35 -04:00
Jake Moshenko	9da64f3aba	Stop writing to deprecated columns for image data.	2015-10-24 14:45:15 -04:00
Jake Moshenko	fee95bc096	Merge remote-tracking branch 'upstream/v2-phase4' into python-registry-v2	2015-10-23 16:47:13 -04:00
Jake Moshenko	5dd377400e	Test postgres before mysql variations in migration	2015-10-23 16:26:47 -04:00
Jake Moshenko	a1e92f7150	Fix references to mysql in migrations	2015-10-23 16:23:27 -04:00
Jake Moshenko	4191d69055	Switch Text to LongText for MySQL manifests	2015-10-23 15:55:25 -04:00
Jimmy Zelinskie	e973289397	Revert "Revert "Merge pull request #682 from jzelinskie/revertrevert"" ... This reverts commit 278bc736e3.	2015-10-23 15:26:33 -04:00
Jake Moshenko	e9722c9468	Backport remaining v2 changes to phase4	2015-10-23 13:49:23 -04:00
Jake Moshenko	e7a6176594	Merge remote-tracking branch 'upstream/v2-phase4' into python-registry-v2	2015-10-22 16:59:28 -04:00
Jimmy Zelinskie	278bc736e3	Revert "Merge pull request #682 from jzelinskie/revertrevert" ... This reverts commit 627ad25c9c, reversing changes made to 31c392fecc.	2015-10-22 16:02:07 -04:00
Jimmy Zelinskie	d568697034	remove migration to get fixes into prod	2015-10-22 15:30:49 -04:00
Jimmy Zelinskie	a34ddc1f71	copy over v1 metadata when linking existing image	2015-10-22 15:21:15 -04:00
Joseph Schorr	c518874ded	I hate Redis! ... - Remove redis check from our health endpoint in prod entirely - Have the redis check have a maximum timeout of 1 second	2015-10-22 14:24:42 -04:00
Jake Moshenko	67ad7ecc88	Remove the unused imagestorage columns from the db.	2015-10-22 12:14:54 -04:00
Jake Moshenko	ce94931540	Stop writing to deprecated columns for image data.	2015-10-22 12:14:39 -04:00
Joseph Schorr	803a983126	Fix deletion of repos and users with V2 stuff ... Fixes #674	2015-10-22 11:58:52 -04:00
Jimmy Zelinskie	39cfe77d42	Revert "Merge pull request #557 from coreos-inc/revert-migration" ... This reverts commit c4f938898a, reversing changes made to 7ad2522dbe.	2015-10-21 15:29:57 -04:00
Joseph Schorr	147b7b26b4	Fix is_public in repo list ... Fixes #678	2015-10-21 14:13:39 -04:00
Joseph Schorr	4e5c8a9281	Reduce GC work time and make sure to use distinct query	2015-10-20 18:13:29 -04:00
Jimmy Zelinskie	2dea9cf05e	Merge pull request #666 from jzelinskie/fixbackfill ... Copy new fields over to the linked image	2015-10-19 17:18:06 -04:00
Jimmy Zelinskie	109d69abfd	Copy new fields over to the linked image ... This potentially fixes an issue with the v2 image field backfill. We should be safe to copy these fields over at link time so that hopefully it doesn't get skipped by the docker client. `_find_or_link_image` should NEVER be used by the registry v2 protocol.	2015-10-19 17:11:11 -04:00
Joseph Schorr	5941f3937c	Enable async GC for all ... Fixes #569	2015-10-19 14:22:41 -04:00
josephschorr	2f42a4d94d	Merge pull request #641 from coreos-inc/wildcardfix ... Make sure to filter wildcard queries	2015-10-15 14:26:51 -04:00
Joseph Schorr	6df7f60e4a	Make sure to filter wildcard queries ... Fixes #640	2015-10-15 14:26:33 -04:00
josephschorr	d3857e509f	Merge pull request #643 from coreos-inc/nullimage ... Check and handle NULL image_size	2015-10-15 13:26:13 -04:00
Joseph Schorr	fe79d5fb66	Check and handle NULL image_size ... Fixes #613	2015-10-15 13:25:54 -04:00
Joseph Schorr	c9daf7d8a9	Add additional tests for repo visibility and further simplify the query for perf	2015-10-15 12:12:57 -04:00
Joseph Schorr	e8cb359d96	Unionize the mega query - It needed more performance-based benefits	2015-10-09 14:45:05 -07:00
Jimmy Zelinskie	9818481b08	limit logs to a maximum number of pages	2015-10-06 14:13:23 -04:00
Matt Jibson	87cc3289a0	Remove transaction from metric reporting	2015-10-06 01:28:43 -04:00
Joseph Schorr	8ca92d6828	Remove old search API and switch V1 search to use the new search system	2015-10-05 14:36:43 -04:00
Joseph Schorr	6e0ca735a5	Add a better redis health check that reads and writes ... This will hopefully catch issues earlier with Redis	2015-09-30 15:23:19 -04:00
Joseph Schorr	35c35d9913	Load images and storage references in bulk during V1 synthesize ... Currently, we perform multiple queries for each layer, making it much slower (especially cross-region) Fixes #413	2015-09-29 17:53:39 -04:00
Silas Sewell	9000169b53	Revert "Merge pull request #491 from jakedt/migratebackp2" ... This reverts commit 7ad2522dbe, reversing changes made to a0b191ffa1.	2015-09-28 16:09:22 -04:00
josephschorr	7ad2522dbe	Merge pull request #491 from jakedt/migratebackp2 ... Migrate image data back phase 2	2015-09-26 15:11:46 -04:00
Matt Jibson	4da66c1219	Move the metric put outside the transaction	2015-09-21 13:37:49 -04:00
Jimmy Zelinskie	2ff77df946	Merge pull request #518 from jzelinskie/fixmysqlssl ... move UseThenDisconnect into queueworker	2015-09-21 13:35:35 -04:00
Jimmy Zelinskie	7c82e0b5b3	move UseThenDisconnect into queueworker ... This makes the tests pass while maintaining the same behavior.	2015-09-21 13:34:12 -04:00
Jimmy Zelinskie	0de17627d5	Merge pull request #517 from jzelinskie/fixmysqlssl ... close connections after getting queue metrics	2015-09-21 12:28:23 -04:00
Jimmy Zelinskie	98d6262a7f	close connections after getting queue metrics	2015-09-21 12:21:39 -04:00
Matt Jibson	bba1557437	Monitor queue adds and EC2 node starts ... fixes #157 see #304	2015-09-18 16:21:16 -04:00
Jake Moshenko	26cea9a07c	Merge remote-tracking branch 'upstream/master' into python-registry-v2	2015-09-17 16:16:27 -04:00
Jake Moshenko	8baacd2741	Migrate old data to new locations, read only new.	2015-09-17 15:47:13 -04:00
Joseph Schorr	b807accfb5	Fix migration head	2015-09-16 18:34:42 -04:00
josephschorr	217779273f	Merge pull request #503 from coreos-inc/ghmigrate ... Migrate all GitHub build triggers to use deploy keys	2015-09-16 18:32:32 -04:00
Joseph Schorr	eff9ff7a66	Migrate all GitHub build triggers to use deploy keys	2015-09-16 17:55:51 -04:00
Silas Sewell	0a48f1cfb0	Merge pull request #495 from coreos-inc/quay-versions ... Add quay releases	2015-09-16 17:29:58 -04:00
Silas Sewell	386c017d99	Add quay releases	2015-09-16 17:18:46 -04:00
Joseph Schorr	30379a2dd8	Fix interleaved repo delete with RAC via a transaction ... The RepositoryActionCount table can have entries added while a repository deletion is in progress. We now perform the repository deletion under a transaction and explicitly test for RAC entries in the deletion unit test (which doesn't test interleaving, but it was missing this check). Fixes #494	2015-09-16 15:34:32 -04:00
Jake Moshenko	502f5e4c8a	Missed one place to duplicate metadata.	2015-09-15 15:57:55 -04:00
Jake Moshenko	b56de3355c	Migrate data back to Image in preparation for v2	2015-09-15 11:53:31 -04:00
Matt Jibson	d36c7dcb4b	Merge pull request #425 from mjibson/monitor-queue-size ... Monitor various sizes for queues	2015-09-14 16:13:31 -04:00
Matt Jibson	39dc4c7d8d	Monitor various sizes for queues ... see #304	2015-09-14 15:57:08 -04:00
josephschorr	6d8752bdb5	Merge pull request #454 from coreos-inc/urlfor ... Remove uses of _external for url_for	2015-09-14 15:54:42 -04:00
Jake Moshenko	9c3ddf846f	Some fixes and tests for v2 auth ... Fixes #395	2015-09-10 15:38:57 -04:00
Joseph Schorr	96d5bbb155	Fix exceptions raised by the diffs worker ... Fixes #465	2015-09-10 14:12:16 -04:00
josephschorr	edef283697	Merge pull request #447 from coreos-inc/ronon ... Add support for Dex to Quay	2015-09-10 11:42:01 -04:00
Joseph Schorr	474fffd01f	Select the full RepositoryBuild record ... If we just return the ID, then peewee just fills in the other fields with defaults (such as UUID).	2015-09-09 21:43:48 -04:00
Jimmy Zelinskie	ece08f6e88	Merge pull request #463 from jzelinskie/fixpagination ... fix pagination of tags in API	2015-09-09 15:53:55 -04:00
Jimmy Zelinskie	ebdee55585	list_repository_tag_history fallback orderby name ... If tags are created at the same time (usually from a tight loop), it is possible that they will be order nondeterministically unless we fallback to another orderby.	2015-09-09 15:52:25 -04:00
Joseph Schorr	3ee4147117	Switch the build logs archiver to a more performant query ... Fixes #459	2015-09-09 13:59:45 -04:00
Joseph Schorr	f0c8552668	Remove uses of _external for url_for ... Fixes #439	2015-09-08 10:29:28 -04:00
Joseph Schorr	c0286d1ac3	Add support for Dex to Quay ... Fixes #306 - Adds support for Dex as an OAuth external login provider - Adds support for OIDC in general - Extract out external logins on the JS side into a service - Add a feature flag for disabling direct login - Add support for directing to the single external login service - Does *not* yet support the config in the superuser tool	2015-09-04 17:05:06 -04:00
Jake Moshenko	210ed7cf02	Merge remote-tracking branch 'upstream/master' into python-registry-v2	2015-09-04 16:32:01 -04:00
Jake Moshenko	82efc746b3	Make our JWT checking more strict.	2015-09-04 15:18:57 -04:00
josephschorr	9889ca268a	Merge pull request #432 from coreos-inc/oauthcheck ... Build the OAuth redirect URL ourselves, rather than relying on undocumented Flask behavior	2015-09-02 13:35:44 -04:00
Joseph Schorr	b7f487da42	Build the OAuth redirect URL ourselves, rather than relying on undocumented Flask behavior	2015-09-02 13:32:11 -04:00
josephschorr	62ea4a6cf4	Merge pull request #191 from coreos-inc/carmen ... Add automatic storage replication	2015-09-01 15:04:36 -04:00
Joseph Schorr	724b1607d7	Add automatic storage replication ... Adds a worker to automatically replicate data between storages and update the database accordingly	2015-09-01 14:53:32 -04:00
Joseph Schorr	51c507d02d	Add back the ability to retrieve information for an org member directly ... Fixes #427	2015-08-31 16:45:24 -04:00
Joseph Schorr	c0c1da3232	Change build logs load to using streaming Gzip	2015-08-28 14:08:13 -04:00
Jake Moshenko	398202e6fc	Implement some new methods on the storage engines.	2015-08-27 11:29:19 -04:00
Joseph Schorr	2b724509b9	Fix 500 error when pushing a manifest previously pushed ... Fixes #400	2015-08-25 15:34:49 -04:00
Joseph Schorr	1450b7e84c	Fix verbs support in V2	2015-08-24 12:05:09 -04:00
Joseph Schorr	cf030e2a98	Save the compressed image size on blob upload completion	2015-08-24 12:05:09 -04:00
Joseph Schorr	d246e68e68	Move shared V1/V2 code into common methods and fix verbs	2015-08-24 12:05:09 -04:00
Jake Moshenko	b998eca8e5	Fix the tests for registry v2 changes.	2015-08-24 11:59:12 -04:00
Jimmy Zelinskie	7787e1350d	Merge pull request #386 from coreos-inc/missingmigration2 ... Add missing migration	2015-08-21 14:21:51 -04:00
Joseph Schorr	e7c405f56b	Add missing migration ... Should have been in commit 84276ee945	2015-08-21 14:21:11 -04:00
Joseph Schorr	e5d2083912	Add new carrier billing plan ... Fixes #370	2015-08-21 14:10:48 -04:00
Matt Jibson	4cb4288672	Merge pull request #373 from mjibson/fix-metric-tests ... Fix test_queue.py tests	2015-08-18 14:05:29 -04:00
Joseph Schorr	0854d20cbd	SECURITY FIX FOR LDAP ... It appears the recent migration of the LDAP code and add of a check for the admin username/password being invalid *broke the LDAP password check*, allowing any password to succeed for login. This fixes the problem, add unit tests to verify the fix and add some tests to our other external auth test suite. A release will be needed immediately along with an announcement	2015-08-18 12:32:19 -04:00
Matt Jibson	fc671f3dde	Fix test_queue.py tests ... This restores the reporter class as was before the metrics changes.	2015-08-17 17:22:46 -04:00
Joseph Schorr	84276ee945	Better notifications UI ... Fixes #369	2015-08-17 17:08:58 -04:00
Jake Moshenko	2fd1d5969e	Merge pull request #351 from mjibson/more-metrics ... More metrics	2015-08-17 13:09:08 -04:00
Jake Moshenko	e1b3e9e6ae	Another huge batch of registry v2 changes ... Add patch support and resumeable sha Implement all actual registry methods Add a simple database generation option	2015-08-12 16:41:12 -04:00
Jake Moshenko	ec6bee35b6	Allow a stripe plan to be superseded ... If a plan has a direct corrolary, show that one as the selected plan instead of showing the plan as deprecated even though it has the same details	2015-08-12 15:01:15 -04:00
Matt Jibson	cfb6e884f2	Refactor metric collection ... This change adds a generic queue onto which metrics can be pushed. A separate module removes metrics from the queue and adds them to Cloudwatch. Since these are now separate ideas, we can easily change the consumer from Cloudwatch to anything else. This change maintains near feature parity (the only change is there is now just one queue instead of two - not a big deal).	2015-08-12 12:15:52 -04:00
Jake Moshenko	0cbc96a54b	Unify the free trial period ... Fixes #263	2015-08-10 17:36:13 -04:00
Joseph Schorr	ea25538646	MySQL and Postgres complain about the group by, so calculate dates ourselves	2015-08-06 12:52:55 -04:00
Joseph Schorr	d34afde954	Fix logs view and API ... - We needed to use an engine-agnostic way to extract the days - Joining with the LogEntryKind table has *horrible* performance in MySQL, so do it ourselves - Limit to 50 logs per page	2015-08-05 17:47:03 -04:00
Joseph Schorr	d480a204f5	Revert change to queue	2015-08-05 15:27:33 -04:00
josephschorr	ee53c04a45	Merge pull request #309 from coreos-inc/fasterqueue ... Improve the performance of queue candidate queries.	2015-08-04 18:24:28 -04:00
Jake Moshenko	ed62339f89	Improve the performance of queue candidate queries.	2015-08-04 18:20:54 -04:00
Joseph Schorr	9f2d6282bd	Add missing index on retries_remaining	2015-08-04 18:01:28 -04:00
josephschorr	f772bd0c9e	Merge pull request #300 from coreos-inc/toomanyutils ... Refactor the util directory to use subpackages.	2015-08-03 16:18:55 -04:00
Jake Moshenko	18100be481	Refactor the util directory to use subpackages.	2015-08-03 16:04:19 -04:00
Jimmy Zelinskie	8e6a0fbbee	Merge pull request #294 from coreos-inc/logsload ... Switch to using an aggregated logs query and infinite scrolling	2015-08-03 14:52:04 -04:00
Joseph Schorr	3d6c92901c	Switch to using an aggregated logs query and infinite scrolling ... This should allow users to work with large logs set. Fixes #294	2015-07-31 16:38:02 -04:00
Jake Moshenko	e133ea0962	Try not to throw any sets of data away when computing images to garbage collect.	2015-07-31 16:12:57 -04:00
Joseph Schorr	0fdc8b0f1f	Fix spelling of ancestors	2015-07-28 15:30:04 -04:00
Joseph Schorr	ba7686af99	Switch back to the read-then-write tag deletion code ... We changed to this originally to avoid locks	2015-07-28 15:30:04 -04:00
Joseph Schorr	70de107268	Make GC of repositories fully async for whitelisted namespaces ... This change adds a worker to conduct GC on repositories with garbage every 10s. Fixes #144	2015-07-28 15:30:04 -04:00
Joseph Schorr	acd86008c8	Switch tag deletion to use a single query	2015-07-28 15:30:04 -04:00
Joseph Schorr	378c83598d	Fix subquery issues in MySQL	2015-07-28 15:28:00 -04:00
Joseph Schorr	66b3d45fbc	Remove legacy.py that was misadded	2015-07-27 15:53:25 -04:00
Joseph Schorr	c3f269ee23	Add migration for BitBucket web hooks ... This needs to added only *after* we roll out #255	2015-07-23 14:45:12 -04:00
Joseph Schorr	ac1b46e7ec	Add missing migration	2015-07-22 16:19:10 -04:00
Joseph Schorr	687bab1c05	Support invite codes for verification of email ... Also changes the system so we don't apply the invite until it is called explicitly from the frontend Fixes #241	2015-07-22 13:41:27 -04:00
Jake Moshenko	5d86fa80e7	Merge pull request #197 from coreos-inc/keystone ... Add Keystone Auth	2015-07-22 13:38:47 -04:00
Joseph Schorr	38a6b3621c	Automatically link the superuser account to federated service for auth ... When the user commits the configuration, if they have chosen a non-DB auth system, we now auto-link the superuser account to that auth system, to ensure they can login again after restart.	2015-07-22 13:37:23 -04:00
Joseph Schorr	a0c4e72f13	Clean up the repository list API and loads stars with it ... We load stars with the same list API now so that we get the extra metadata needed in the repo list (popularity and last modified)	2015-07-22 13:05:02 -04:00
Joseph Schorr	7e4b23916a	Small SQL query fix ... Fixes #248	2015-07-20 14:17:26 -04:00
Joseph Schorr	33b54218cc	Refactor the users class into their own files, add a common base class for federated users and add a verify_credentials method which only does the verification, without the linking. We use this in the superuser verification pass	2015-07-20 11:39:59 -04:00
Joseph Schorr	1245385808	Fix typo	2015-07-20 10:55:21 -04:00
Joseph Schorr	066637f496	Basic Keystone Auth support ... Note: This has been verified as working by the end customer	2015-07-20 10:55:21 -04:00
Jake Moshenko	bc29561f8f	Fix and templatize the logic for external JWT AuthN and registry v2 Auth. ... Make it explicit that the registry-v2 stuff is not ready for prime time.	2015-07-17 11:56:15 -04:00
Jake Moshenko	3efaa255e8	Accidental refactor, split out legacy.py into separate sumodules and update all call sites.	2015-07-17 11:56:15 -04:00
Jake Moshenko	bea8b9ac53	More changes for registry-v2 in python. ... Implement the minimal changes to the local filesystem storage driver and feed them through the distributed storage driver. Create a digest package which contains digest_tools and checksums. Fix the tests to use the new v1 endpoint locations. Fix repository.delete_instance to properly filter the generated queries to avoid most subquery deletes, but still generate them when not explicitly filtered.	2015-07-17 11:50:41 -04:00
Jake Moshenko	acbcc2e206	Start of a v2 API.	2015-07-17 11:50:41 -04:00
Joseph Schorr	7a548ea101	Fix queries for repository list popularity and action count ... Before this change, we used extremely inefficient outer joins as part of a single query of lookup, which was spiking our CPU usage to nearly 100% on the query. We now issue two separate queries for popularity and action account, by doing a lookup of the previously found IDs. Interestingly enough, because of the way the queries are now written, MySQL can actually do both queries *directly from the indicies*, which means they each occur in approx 20ms! Verified by local tests, postgres tests, and testing on staging with monitoring of our CPU usage during lookup	2015-07-17 00:08:27 +03:00
Jimmy Zelinskie	2869e2a6ea	model: add missing params to validate_database_url	2015-07-15 17:39:26 -04:00
Jake Moshenko	eec7886e01	Add a server default for the broken migration.	2015-07-14 16:58:58 -04:00
Jimmy Zelinskie	bde781c98b	Merge pull request #205 from coreos-inc/delrobot ... Fix deletion of robot accounts when attached to builds	2015-07-13 12:19:01 -04:00
Joseph Schorr	3a59c99b08	Add a secondary tab to Teams for managing org members ... Also adds the ability to completely remove a user from an organization (repo permissions and teams), in a single click Fixes #212	2015-07-02 17:06:36 +03:00
Joseph Schorr	b535e222b8	Have the fetch tag dialog show a warning for robot accounts without access ... Before this change, we'd show the squash pulling command with the proper credentials, but it then 403s on the end user.	2015-07-01 19:37:52 +03:00
Joseph Schorr	3ba321934f	Fix deletion of robot accounts when attached to builds ... Fixes #204	2015-06-30 22:56:44 +03:00
josephschorr	7aeaf2344e	Merge pull request #200 from coreos-inc/tagapilimit ... Add pagination support to tag history API	2015-06-30 22:09:09 +03:00
Joseph Schorr	f7f10f4a6d	Add pagination support to tag history API ... Fixes #198	2015-06-30 19:44:43 +03:00
Jake Moshenko	38a5963afe	Merge pull request #190 from coreos-inc/timezone ... Fromtimestamp needs to be in UTC for JWT auth	2015-06-30 12:05:00 -04:00
Joseph Schorr	2b1bbcb579	Add a table view to the repos list page ... Fixes #104	2015-06-29 21:12:53 +03:00
Jimmy Zelinskie	756d6784ca	Merge pull request #192 from coreos-inc/sqlssl ... Allow SSL cert for the database to be configured	2015-06-29 13:33:31 -04:00
Joseph Schorr	dc5af7496c	Allow superusers to disable user accounts	2015-06-29 18:40:52 +03:00
Joseph Schorr	bb07d0965f	Allow SSL cert for the database to be configured ... This change adds a field for the SSL cert for the database in the setup tool. Fixes #89	2015-06-29 08:08:10 +03:00
Joseph Schorr	477e244eff	Fromtimestamp needs to be in UTC for JWT auth	2015-06-28 11:37:09 +03:00
Joseph Schorr	e7915baf8c	Have LDAP return a better error message if it fails to connect ... Currently, the error results in a 500 being raised when a user tries to login.	2015-06-23 17:41:53 -04:00
Joseph Schorr	07439328a4	Remove user_exists endpoint from all auth systems	2015-06-23 17:33:51 -04:00
Jake Moshenko	b21a033ef3	Merge pull request #131 from coreos-inc/moveapp ... Refactor JWT auth to not import app locally	2015-06-23 17:24:01 -04:00
Jake Moshenko	5f1d23c6e8	Use a UNION query instead of a multitude of left outer joins for performance reasons. ... Fixes #159	2015-06-23 17:18:37 -04:00
Joseph Schorr	331c300893	Refactor JWT auth to not import app locally	2015-06-17 15:53:21 -04:00
Joseph Schorr	e7fa560787	Add support for custom fields in billing invoices ... Customers (especially in Europe) need the ability to add Tax IDs, VAT IDs, and other custom fields to their invoices. Fixes #106	2015-06-12 16:45:01 -04:00
Jake Moshenko	79f1181a63	Switch build-scheduled to an official build phase.	2015-06-10 16:19:51 -04:00
Jake Moshenko	e09d84b3c8	Merge pull request #55 from coreos-inc/oauthdeny ... Fix OAuth redirect for denial action when generating for internal tokens	2015-06-05 14:00:16 -04:00
Joseph Schorr	8aac3fd86e	Add support for an external JWT-based authentication system ... This authentication system hits two HTTP endpoints to check and verify the existence of users: Existance endpoint: GET http://endpoint/ with Authorization: Basic (username:) => Returns 200 if the username/email exists, 4** otherwise Verification endpoint: GET http://endpoint/ with Authorization: Basic (username:password) => Returns 200 and a signed JWT with the user's username and email address if the username+password validates, 4** otherwise with the body containing an optional error message The JWT produced by the endpoint must be issued with an issuer matching that configured in the config.yaml, and the audience must be "quay.io/jwtauthn". The JWT is signed using a private key and then validated on the Quay.io side with the associated public key, found as "jwt-authn.cert" in the conf/stack directory.	2015-06-05 13:20:10 -04:00
Jake Moshenko	42da017d69	Merge pull request #48 from coreos-inc/nobots ... Change API calls that expect non-robots to explicitly filter	2015-06-02 12:31:19 -04:00
Joseph Schorr	5516911de9	Fix OAuth redirect for denial action when generating for internal tokens	2015-06-02 12:25:59 -04:00
Joseph Schorr	2a56790d38	Switch to using a named LDAP tuple for more readable code	2015-06-01 14:02:05 -04:00
Joseph Schorr	1aff701bc7	Fix LDAP referral and multiple pair handling ... Fixes two issues found with our LDAP handling code. First, we now follow referrals in both LDAP calls, as some LDAP systems will return a referral instead of the original record. Second, we now make sure to handle multiple search result pairs properly by further filtering based on the presence of the 'mail' attribute when we have multiple valid pairs. This CL also adds tests for all of the above cases.	2015-05-27 15:04:34 -04:00
Joseph Schorr	fdd43e2490	Change API calls that expect non-robots to explicitly filter ... Before this change, we'd filter in the UI but calls to the API could allow robots accounts where we only expect real users	2015-05-26 17:47:33 -04:00
Joseph Schorr	b0d763b5ff	Fix encrypted password generator to use the LDAP username, not the Quay username. ... Currently, we use the Quay username via `verify_user` when we go to create the encrypted password. This is only correct if Quay has not generated its own different username for the LDAP user, and fails if it has. We therefore add a new method `confirm_existing_user`, which looks up the federated login for the LDAP user and then runs the auth flow using that username.	2015-05-20 16:37:09 -04:00
Joseph Schorr	d5e70c6e2a	Explicitly enable LDAP referrals ... Note: The mock LDAP system doesn't support referrals, so we can't add a unit test for this.	2015-05-20 14:53:31 -04:00
Jimmy Zelinskie	3daa0e6a52	migration: use SA boolean value for server_default	2015-05-15 19:04:54 -04:00
Jimmy Zelinskie	e12d5af52c	migration: server_default for boolean field	2015-05-15 17:04:50 -04:00
Joseph Schorr	6a52ffa942	Add missing default value on the new reversion column	2015-05-13 13:49:21 -04:00
Joseph Schorr	07b4fb9105	LDAP sometimes has multiple records for a user	2015-05-12 12:02:09 -04:00
Joseph Schorr	efab02ae47	LDAP improvements: ... - Better logging - Better error messages - Add unit tests - Clean up the setup tool for LDAP	2015-05-11 21:23:18 -04:00
Joseph Schorr	3e1abba284	Add ability for super users to rename and delete organizations	2015-05-11 18:03:25 -04:00
Joseph Schorr	1c41d34b7c	Add ability for superusers to change user emails	2015-05-11 14:38:10 -04:00
Joseph Schorr	1c83def15b	LDAP should only show logs when asked.	2015-05-11 13:01:49 -04:00
Joseph Schorr	f858caf6cd	Only return the team and repo permissions when listing robots when we absolutely need them.	2015-05-08 16:43:07 -04:00
Joseph Schorr	561f2c7db0	Remove unnecessary sort	2015-05-08 15:10:31 -04:00
Joseph Schorr	3b93854c0b	MySQL returns a decimal type for the tuple, which we need to convert to an int	2015-05-08 14:55:54 -04:00
Joseph Schorr	36f33e8fd7	Merge branch 'master' of github.com:coreos-inc/quay	2015-05-08 14:19:39 -04:00
Joseph Schorr	f9c1f123c2	Add better debugging to LDAP	2015-05-08 14:19:32 -04:00
Joseph Schorr	8ed8367404	PR changes in response to comments	2015-05-08 13:38:34 -04:00
Joseph Schorr	c767aafcd6	Make the repository API faster by only checking the log entries table once for each kind of entry, rather than twice. We make use of a special subquery-like syntax, which allows us to count those entries that are both 30 days only and 1 day old in the same query. This was tested successfully on MySQL, Postgres and Sqlite.	2015-05-07 22:49:11 -04:00
Joseph Schorr	3627de103c	Minimize the queries used when retrieve builds. Previously, we'd call out to SQL extra times per build.	2015-05-07 21:11:15 -04:00
Joseph Schorr	7b35555776	Make sure to test for unicode usernames, since the collate on the username field is latin1	2015-05-07 18:13:45 -04:00
Joseph Schorr	729249f42b	Need to add the tables to the .select() to prevent N+1, not merely to the join	2015-05-07 16:59:13 -04:00
Joseph Schorr	b50600cde8	Make sure to join the Repo table on its namespace user and its visibility so we don't issue additional queries from the starred repos API call	2015-05-07 16:18:17 -04:00
Joseph Schorr	3cec22defe	Make sorting by repo ID, rather than description	2015-05-07 13:10:26 -04:00
Joseph Schorr	73193e2ab5	Remove sort by description, which is slow (and fairly useless anyway)	2015-05-06 20:55:31 -04:00
Joseph Schorr	c89760278f	Add migration to add gitlab trigger type	2015-05-03 10:49:13 -07:00
Joseph Schorr	a59100b231	Add the missing index on the peewee side. We already have the associated migration.	2015-04-30 15:56:40 -04:00
Joseph Schorr	17bc9b81bf	Add bitbucket trigger type	2015-04-30 15:55:57 -04:00
Joseph Schorr	b96e35b28c	Merge master into bitbucket	2015-04-30 15:52:08 -04:00
Joseph Schorr	31260d50f5	Rename the new images method to a slightly better name	2015-04-24 16:37:37 -04:00
Joseph Schorr	e70343d849	Faster cache lookup by removing a join with the ImagePlacementTable, removing the extra loop to add the locations and filtering the images looked up by the base image	2015-04-24 16:22:19 -04:00
Jimmy Zelinskie	fd65ca5916	migration: add custom-git service to database	2015-04-24 16:11:58 -04:00
Joseph Schorr	c480fb2105	Work in progress: bitbucket support	2015-04-24 15:13:08 -04:00
Jimmy Zelinskie	d25cc4db9c	migrations: fix percona migration test	2015-04-21 18:01:40 -04:00
Jimmy Zelinskie	93cd459460	Merge branch 'master' into git	2015-04-20 10:58:49 -04:00
Joseph Schorr	43ff6839b8	Hide hidden tags in the tags timeline	2015-04-19 18:12:06 -04:00
Joseph Schorr	ed342ae831	Add migration for properly creating the repository_id+datetime index	2015-04-19 16:03:06 -04:00
Joseph Schorr	d1e2d072ea	Add unit tests and a stronger restriction on the revert API call	2015-04-19 15:43:16 -04:00
Joseph Schorr	a8f8c317f9	Fix branch in alembic migrations	2015-04-19 15:20:01 -04:00
Joseph Schorr	2815ea2723	Merge branch 'master' of github.com:coreos-inc/quay	2015-04-19 15:16:39 -04:00
Joseph Schorr	8714eb207a	Merge branch 'actioncount'	2015-04-19 15:12:44 -04:00
Jimmy Zelinskie	4133924813	migrations: rebase on top of master's changes	2015-04-17 13:19:04 -04:00
Jimmy Zelinskie	ba2cb08904	Merge branch 'master' into git	2015-04-16 17:38:35 -04:00
Joseph Schorr	f19d2f684e	Add ability to revert tags via time machine	2015-04-16 17:18:00 -04:00
Joseph Schorr	f8c80f7d11	Add a history view to the tags page. Next step will add the ability to revert back in time	2015-04-15 15:21:09 -04:00
Joseph Schorr	3f1e8f3c27	Add a RepositoryActionCount table so we can use it (instead of LogEntry) when scoring repo search results	2015-04-13 13:31:07 -04:00
Joseph Schorr	1df025b57e	Change search to use a set of queries for repo lookup rather than a single monolithic query, in the hopes that this will make things significantly faster and actually useable. The individual queries have been tested by hand on MySQL, but the real test will be staging	2015-04-10 15:27:37 -04:00
Joseph Schorr	0be0aed17d	Move the repo sorting by pull count into the main matching query, to both make it more accurate and make the search faster	2015-04-09 14:41:59 -04:00
Joseph Schorr	396cba64e6	Fix search to return better results by searching for robots and namespaces in different queries.	2015-04-09 12:57:20 -04:00
Joseph Schorr	762fdaf319	Fix DB migration tests	2015-04-08 17:41:53 -04:00
Joseph Schorr	4f4bb05621	Fix search SQL issues	2015-04-08 17:41:08 -04:00
Joseph Schorr	19e25ac340	Merge branch 'master' into bing	2015-04-08 15:23:36 -04:00
Joseph Schorr	d09f2f6e22	Get the new context-sensitive new menu working	2015-04-07 18:33:43 -04:00
Joseph Schorr	40a6892a49	Add search tests	2015-04-07 14:05:12 -04:00
Joseph Schorr	1b56567268	Make sure also include teams from organizations that the user admins	2015-04-07 13:45:49 -04:00
Joseph Schorr	951b0cbab8	Start on new interactive search	2015-04-06 19:17:18 -04:00
Joseph Schorr	4cb7921c3a	Make sure to show public repos on the user and orgs pages	2015-04-03 14:55:09 -04:00
Joseph Schorr	094f91fb8b	Fix the tutorial's user events	2015-04-03 12:13:33 -04:00
Joseph Schorr	5cd500257d	Merge branch 'master' into orgview	2015-04-01 13:56:49 -04:00
Joseph Schorr	fde9666647	Add the team membership to the robots view	2015-04-01 13:56:30 -04:00
Joseph Schorr	1f5e6df678	- Fix tests ... - Add new endpoints for retrieving the repo permissions for a robot account - Have the robots list return the number of repositories for which there are permissions - Other UI fixes	2015-03-31 18:50:43 -04:00
Joseph Schorr	27a9b84587	Switch avatars to be built out of CSS and only overlayed with the gravatar when a non-default exists	2015-03-30 17:55:04 -04:00
Jimmy Zelinskie	da15eda2bf	migration: make auth_token for builds nullable	2015-03-27 11:20:30 -04:00
Joseph Schorr	7b1d35737e	Merge branch 'master' of github.com:coreos-inc/quay	2015-03-26 17:45:49 -04:00
Joseph Schorr	a7b6cb5c23	Fix handling of byte strings and large ints	2015-03-26 17:45:43 -04:00
Jimmy Zelinskie	4d4947edaf	Merge branch 'master' of github.com:coreos-inc/quay	2015-03-26 16:31:20 -04:00
Jimmy Zelinskie	bcc7a9580b	models: change UUID of user on password change ... This prevents old cookies from continuing to work after a password has been changed.	2015-03-26 16:30:41 -04:00
Joseph Schorr	c4a2574b0d	Clarify unencrypted password error message	2015-03-26 16:23:28 -04:00
Joseph Schorr	f8afd8b5ce	Make sure to parse the big int into a byte string	2015-03-26 16:13:35 -04:00
Joseph Schorr	4d1792db1c	getrandbits creates an int, not a float	2015-03-26 15:47:44 -04:00
Joseph Schorr	aaf1b23e98	Address CL concerns and switch to a real encryption system	2015-03-26 15:10:58 -04:00
Joseph Schorr	d23bb6616d	Fix error message to exactly match current output	2015-03-26 13:22:16 -04:00
Joseph Schorr	e4b659f107	Add support for encrypted client tokens via basic auth (for the docker CLI) and a feature flag to disable normal passwords	2015-03-25 18:43:12 -04:00
Jake Moshenko	3d44416016	Add the migration for the unique index which helps prevent tag deadlocks.	2015-03-24 18:00:04 -04:00
Jake Moshenko	201943ed1c	Fix deadlocks with tags and garbage collection.	2015-03-24 18:00:04 -04:00
Jimmy Zelinskie	9c55aca011	migration: make resource_key nullable	2015-03-23 15:46:35 -04:00
Jimmy Zelinskie	93a9e9d01a	migration: add private key to build triggers	2015-03-19 14:30:25 -04:00
Joseph Schorr	ab2331a486	Performance improvements for the repo API and the new repo UI	2015-03-18 14:47:53 -04:00
Joseph Schorr	fe8d006855	Fix subquery on MySQL	2015-03-17 13:54:51 -04:00
Joseph Schorr	44ff85d044	Remove migration (temporarily), fix a broken test, and make the aggregate size calculation use the entire image ancestry (for now).	2015-03-17 12:13:01 -04:00
Joseph Schorr	b8d88c0f4e	Add aggregate size column and a migration to backfill it	2015-03-16 18:03:17 -04:00
Joseph Schorr	333e0acd6d	Add the builds tab	2015-03-13 15:34:28 -07:00
Joseph Schorr	002dc083f2	Get the main repo page design working	2015-03-10 17:22:46 -07:00
Joseph Schorr	afc8e95e19	Start on new tag view	2015-03-09 22:03:39 -07:00
Joseph Schorr	929bad57f5	Fix revision	2015-03-05 15:06:46 -05:00
Joseph Schorr	86447c0a99	Merge branch 'master' into pagesnew	2015-03-05 14:22:10 -05:00
Joseph Schorr	edafa26e6e	Fix email confirmation for users created through the superuser API	2015-03-03 14:26:48 -05:00
Joseph Schorr	a36266f758	Add LDAP tracing	2015-02-27 17:01:46 -05:00
Jake Moshenko	6c4e78ec0c	Fix the deadlock in tags.	2015-02-25 17:49:46 -05:00
Jimmy Zelinskie	fb0d3d69c2	changes to reflect PR comments (not finished)	2015-02-24 17:50:54 -05:00
Jake Moshenko	d8a34427ec	Try to get around the tag deadlock by using a select for update.	2015-02-24 17:18:42 -05:00
Jake Moshenko	246ff556b9	Fix some other list reifications to use nested subqueries for performance and query size safety reasons.	2015-02-23 15:07:38 -05:00
Joseph Schorr	4020cc1102	Reset the number of invalid login attempts when the user change's their password	2015-02-23 13:56:42 -05:00
Joseph Schorr	a0706d20ae	Merge branch 'master' of github.com:coreos-inc/quay	2015-02-23 13:38:04 -05:00
Joseph Schorr	5f605b7cc8	Fix queue handling to remove the dependency from repobuild, and have a cancel method	2015-02-23 13:38:01 -05:00
Jake Moshenko	75d41ca371	Switch to an inner inner subquery to make mysql arbitrarily happy.	2015-02-23 12:36:14 -05:00
Jimmy Zelinskie	7fbbf14e49	migrations: resolve branched migration tree ... This moves the migration adding stars to HEAD.	2015-02-19 17:10:27 -05:00
Jimmy Zelinskie	917dd6b674	Merge branch 'master' into star	2015-02-18 17:36:58 -05:00
Jake Moshenko	41108a0856	Allow tags to be marked as hidden. Create a hidden tag on every image during a push to prevent them from getting GCed.	2015-02-18 17:05:16 -05:00
Jake Moshenko	59b794dd61	Move the creation of images to when the JSON is uploaded.	2015-02-18 17:04:25 -05:00
Joseph Schorr	89eb5bdcc5	Merge branch 'quark'	2015-02-18 15:59:36 -05:00
Joseph Schorr	a5ff765f3b	Validate that we have a valid JSON body	2015-02-18 15:57:05 -05:00
Jake Moshenko	2dd03f1bed	Merge remote-tracking branch 'origin/master' into rockyhorror ... Conflicts: test/data/test.db	2015-02-18 10:56:01 -05:00
Jake Moshenko	2cd5bdb563	Address concerns from pull request.	2015-02-18 10:43:08 -05:00
Joseph Schorr	ff7e97faf7	Merge branch 'master' into quark	2015-02-17 19:16:12 -05:00
Joseph Schorr	524705b88c	Get dashboard working and upgrade bootstrap. Note: the bootstrap fixes will be coming in the followup CL	2015-02-17 19:15:54 -05:00
Joseph Schorr	83e05d2342	Add tracking of the kind of temporary access tokens, so we can display if a pull/push by token is for a build worker	2015-02-17 12:35:16 -05:00
Joseph Schorr	3e04e3cfd7	Merge branch 'master' of github.com:coreos-inc/quay	2015-02-13 15:54:09 -05:00
Joseph Schorr	81ce4c771e	Add ability to cancel builds that are in the waiting state	2015-02-13 15:54:01 -05:00
Jake Moshenko	579d900ef2	Add server defaults to the new fields which are not nullable.	2015-02-13 14:45:42 -05:00
Jake Moshenko	2ce6e76d9d	Add the required migration for time machine tag lifetimes.	2015-02-13 14:41:08 -05:00
Jake Moshenko	5aedd1fabc	Merge remote-tracking branch 'origin/master' into rockyhorror ... Conflicts: test/data/test.db	2015-02-13 13:17:10 -05:00
Jake Moshenko	2743fe6f4d	Trap migration exceptions for the migration that we expect to fail in prod.	2015-02-13 13:11:50 -05:00
Jake Moshenko	cd245e62bf	Remove the access_token_id column from log entries.	2015-02-13 11:56:59 -05:00
Jake Moshenko	4a921a49b9	Merge remote-tracking branch 'origin/master' into rockyhorror	2015-02-12 16:24:30 -05:00
Joseph Schorr	f84d1bad45	Handle internal errors in a better fashion: If a build would be marked as internal error, only do so if there are retries remaining. Otherwise, we mark it as failed (since it won't be rebuilt anyway)	2015-02-12 16:19:44 -05:00
Jake Moshenko	b154e7acef	Merge remote-tracking branch 'origin/master' into rockyhorror	2015-02-12 15:27:28 -05:00
Jake Moshenko	990739b1e5	Add the APIs required to change the time machine policy for users and organizations.	2015-02-12 14:37:11 -05:00
Jake Moshenko	872539bdbf	Switch to a per-namespace configurable expiration policy for time machine, and switch the tag gc to respect it.	2015-02-12 14:11:56 -05:00
Joseph Schorr	f107b50a46	Merge branch 'master' into ackbar	2015-02-12 12:04:45 -05:00
Jake Moshenko	f32bd748e4	Use a consistent concept of tag liveness everywhere. Fix the tests.	2015-02-11 15:02:50 -05:00
Jake Moshenko	90c0a9c1e0	First stab at time machine using fixed two week expiration policy.	2015-02-11 14:15:40 -05:00
Joseph Schorr	045614c6c8	Merge branch 'master' into ackbar	2015-02-09 17:16:42 -05:00
Joseph Schorr	3d31c64da2	Make sure to select the latest image in the repository with the matching comment	2015-02-09 16:45:06 -05:00
Joseph Schorr	48949627e0	Merge master in delta	2015-02-09 12:07:43 -05:00
Joseph Schorr	e1c5ccb7d6	Fixes	2015-02-05 17:37:58 -05:00
Joseph Schorr	cf774e23df	Merge branch 'master' into v2	2015-02-05 15:37:14 -05:00
Joseph Schorr	a12bfa7623	Add migration for the new tables for signatures	2015-02-05 15:30:45 -05:00
Joseph Schorr	9dfe523615	Merge master changes	2015-02-05 13:11:16 -05:00
Joseph Schorr	bfb0784abc	Add signing to the ACI converter	2015-02-04 15:29:24 -05:00
Jake Moshenko	ce7033489b	Hopefully fix the deadlock in the queue.	2015-02-03 14:50:01 -05:00
Jake Moshenko	64750e31fc	Add the ability to select for update within transactions to fix some write after read hazards. Fix a bug in extend_processing.	2015-01-30 16:32:13 -05:00
Joseph Schorr	3872d29de9	Add a transaction around the extend_processing call	2015-01-29 18:40:41 -05:00
Joseph Schorr	30b895b795	Merge branch 'grunt-js-folder' of https://github.com/coreos-inc/quay into ackbar	2015-01-23 17:26:14 -05:00
Joseph Schorr	c8229b9c8a	Implement new step-by-step setup	2015-01-23 17:19:15 -05:00
Jake Moshenko	44f7ab53a2	Merge remote-tracking branch 'origin/master' into ephemeral	2015-01-21 13:39:27 -05:00
Joseph Schorr	92d32bc636	Make the DB health check first attempt a simple DB connection. If the database is in the middle of a failover, this will fail after 3 seconds (the connection timeout specified), rather than hanging and causing the ELB health checks to timeout and fail.	2015-01-20 14:46:22 -05:00
Joseph Schorr	15a69ac872	Change robot deletions to set the performer to null, rather than attempting to delete the rows from the large log entries table	2015-01-14 12:56:06 -05:00
Joseph Schorr	bfd273d16f	- Make validation a bit nicer: ... - Add timeout to the DB validation - Make DB validation exception handling a bit nicer - Move the DB validation error message - Fix bug around RADOS config default for Is Secure - Allow hiding of the validation box	2015-01-08 15:27:49 -05:00
Joseph Schorr	47fb10b79f	Merge branch 'master' into ackbar	2015-01-08 13:57:39 -05:00
Jimmy Zelinskie	b18986c72b	Continuation of baa4918d89.	2015-01-06 15:11:37 -05:00
Jimmy Zelinskie	baa4918d89	Explicitly declare length of large MySQL indices. ... Most MySQL installs silently truncate indices at a specific number of bytes. This value is 767 bytes unless either the innodb_large_prefix option is turned on or the innodb_page_size has been changed. This change explicitly limits the size of large indices to 767 characters due to the latin1 charset being 1 byte per char.	2015-01-06 14:55:31 -05:00
Joseph Schorr	1bf25f25c1	WIP	2015-01-04 14:38:41 -05:00
Jake Moshenko	cc70225043	Generalize the ephemeral build managers so that any manager may manage a builder spawned by any other manager.	2014-12-31 11:33:56 -05:00
Jimmy Zelinskie	e8cd24781a	remove 'include_starred' option on repo listings	2014-12-29 14:11:46 -05:00
Jimmy Zelinskie	84f6cc7385	add percona to migration tests	2014-12-23 18:34:02 -05:00
Jimmy Zelinskie	977782d047	Add MariaDB to migration.sh	2014-12-22 15:34:17 -05:00
Jimmy Zelinskie	eeeb2e620c	move slackwebhook migration from tools to util ... tools isn't shipped inside of the container because it contains private keys	2014-12-18 13:22:13 -05:00
Joseph Schorr	107847a4bb	Fix Slack notification setup to support the new slack web hook format and convert all existing data to use the new format (so we only have one code path)	2014-12-16 14:34:43 +02:00
Jimmy Zelinskie	5a484cfe11	Initial redesigned UI for repo listings w/ stars.	2014-12-11 15:07:41 -05:00
Joseph Schorr	6601e83285	When speaking to version 0.2-beta of the build worker, properly lookup the cached commands and see if we have a matching image/tag in the repository	2014-12-11 18:03:40 +02:00
Joseph Schorr	dbac8c7e3d	Fix build code: ... - Fix issue with the queue_item in extend processing - Add the new compiled docker binary with the lxc volume fix	2014-12-04 17:49:39 +01:00
Jimmy Zelinskie	97b605ca8d	Fix starring after rebase.	2014-12-03 12:20:51 -08:00
Jimmy Zelinskie	bd9f529e38	Add missing Star import.	2014-12-02 17:31:55 -08:00
Jimmy Zelinskie	08d84298e2	Fix indentation in star migration.	2014-12-02 17:31:21 -08:00
Jimmy Zelinskie	eb956e5b7d	initial work on adding models for starring repos. ... I'm sick of using `git stash`.	2014-12-02 17:31:21 -08:00
Jimmy Zelinskie	f3259c862b	Merge branch 'koh' ... Conflicts: auth/scopes.py requirements-nover.txt requirements.txt static/css/quay.css static/directives/namespace-selector.html static/js/app.js static/partials/manage-application.html templates/oauthorize.html	2014-12-01 12:30:09 -08:00
Joseph Schorr	72d613614d	Merge branch 'bagger'	2014-12-01 12:48:59 -05:00
Joseph Schorr	7bf96c506f	Merge branch 'bees' into koh	2014-11-24 19:25:53 -05:00
Joseph Schorr	e9cac407df	Add a configurable avatar system and add an internal avatar system for enterprise	2014-11-24 19:25:13 -05:00
Jimmy Zelinskie	716d7a737b	Strip whitespace from ALL the things.	2014-11-24 16:07:38 -05:00
Joseph Schorr	b8e873b00b	Add support to the build system for tracking if/when the build manager crashes and make sure builds are restarted within a few minutes	2014-11-21 14:27:06 -05:00
Jimmy Zelinskie	c918d15979	Add a migration for backfilling user UUIDs.	2014-11-20 18:28:54 -05:00
Jake Moshenko	2b8c246476	Temporarily put user rename behind a feature flag. Switch queue names back to using the username for namespace while we figure out a real migration strategy.	2014-11-20 15:36:39 -05:00
Jimmy Zelinskie	e6742b00b0	Remove backfill from UUID migration	2014-11-20 14:04:12 -05:00
Joseph Schorr	8fab3b6d34	Make sure to switch the context back to LogEntry and make sure to only include the performer if actually present	2014-11-20 12:04:53 -05:00
Joseph Schorr	cc7bccea4c	Add the logentrykind and performer information to log listing. This makes the system much faster, since new queries are not needed for every log entry.	2014-11-20 12:03:16 -05:00
Joseph Schorr	9b31b9805a	Fix performance problem with looking up org members and add some tests	2014-11-20 11:33:42 -05:00
Jake Moshenko	ac0d0ce36a	Change the order of migrations to allow migration code to work with the updated model.	2014-11-19 15:58:47 -05:00
Jake Moshenko	e863b96166	Tweak the uuid backfill to leave the uuid column nullable.	2014-11-19 15:32:30 -05:00
Jimmy Zelinskie	7c8a438b58	Rebase migration on top of master and add backfill.	2014-11-19 14:07:54 -05:00
Jimmy Zelinskie	606ad21bec	Apply reviewed changes. ... Adds a length to the UUID field, renames QuayDeferredPermissionUser parameter id->uuid, adds transactions to backfill script.	2014-11-19 13:28:16 -05:00
Jimmy Zelinskie	9d677b8eb3	Add UUID to User model and use in cookie.	2014-11-19 13:28:16 -05:00
Jake Moshenko	17fc72d262	Switch postgres to a non-transactional DDL to allow us to use peewee to modify data in migrations: enterprise customers are running postgres migrations offline already. Move the image backfill script back to a migration since it will now work. Unify the interface to sending a DB URI to env.py for the migration script.	2014-11-18 14:07:33 -05:00
Jake Moshenko	a7bae6c1d9	Fix robot renaming. Allow for trigger URLs to contain or omit the repository in the path. Fix calls to get_trigger to remove the namespace and repository.	2014-11-18 10:24:48 -05:00
Jake Moshenko	f4681f2c18	Merge branch 'master' into nomenclature ... Conflicts: test/data/test.db	2014-11-17 17:59:59 -05:00
Joseph Schorr	58ca76239b	Add ability to one-click generate an authorization access token in the applications panel	2014-11-17 14:54:07 -05:00
Joseph Schorr	178c5a7ac0	Add an index to the docker_image_id for faster lookup	2014-11-13 12:51:50 -05:00
Joseph Schorr	d73747ce1d	- Fix some other group_by clauses ... - Fix garbage_collect for an empty list (fixes a test) - Add a script which runs the full test suite against mysql and postgres (note: QueueTest's are broken for MySQL, but they obviously work in production, so they need to be fixed)	2014-11-13 12:51:37 -05:00
Joseph Schorr	a2e2dcb010	Fix group_by clauses for postgres	2014-11-12 20:32:06 -05:00
Joseph Schorr	079289c53c	HACK: Don't join on the visibility table because it is horrendously slow for some odd reason.	2014-11-12 14:51:24 -05:00
Joseph Schorr	eddcc02ea6	Make repository deletes much faster by adding custom deletion code and have additional tests to verify the deletion code paths	2014-11-10 23:05:20 -05:00
Joseph Schorr	e7cbda86f7	Merge branch 'perf'	2014-11-10 21:52:27 -05:00
Joseph Schorr	091f821a6a	- Rename get_repo_image to get_repo_image_extended and get_repo_image_directly to get_repo_image ... - Remove the configure call from CloseForLongOperation - Other small fixes	2014-11-10 13:44:36 -05:00
Joseph Schorr	72fedef097	Move the robot deletion code into a delete_instance method on the User object	2014-11-10 13:18:17 -05:00
Jake Moshenko	ca435fc7a6	Rename robots when we rename a user. Do not use the namespace from the path to check permissions from the incoming webhooks since the namespace may have changed and we cannot recreate them in remote services easily.	2014-11-09 17:50:57 -05:00
Joseph Schorr	17f605a9ef	Select only a single token.	2014-11-09 15:50:50 -05:00
Joseph Schorr	691be49817	Fix issues with the perf updated code	2014-11-07 14:36:32 -05:00
Joseph Schorr	158acd4f41	- Turn on foreign key constraint checking in the tests ... - Change all ForeignKeyField's that refer to users to use our custom class, and mark those that allow robots - Change robot delete to only execute the subset of queries necessary to actually delete robots	2014-11-07 12:05:21 -05:00
Joseph Schorr	d5bbb57481	Change registry code to disconnect from the DB before long I/O operations	2014-11-06 18:00:52 -05:00
Joseph Schorr	23d9bd2b42	Change verbs to use short lived database connections	2014-11-06 17:50:48 -05:00
Joseph Schorr	c569299e5c	Database optimizations around image creation and logs lookup	2014-11-06 14:48:16 -05:00
Joseph Schorr	a35bc11912	Add perf comments	2014-11-05 12:27:38 -05:00
Joseph Schorr	29c30b336e	Move the uncompressed image size migration call outside of alembic, since it will sometimes deadlock with certain kinds of DBs (because alembic is running things inside a transaction)	2014-11-03 21:01:41 -05:00
Joseph Schorr	9aa72c5cc2	Fix migration issues: ... - MySQL 5.5 doesn't support the now() call as a default - Postgres migration isn't auto-committed, so we have to check if the table exists first	2014-11-03 15:25:55 -05:00
Joseph Schorr	4eedd54b66	- Make usage language more accurate by stating "repositories" ... - Have usage counter be based on a 4 weeks TTL - Add a simple usage counter breakage test	2014-10-30 13:26:02 -04:00
Joseph Schorr	c1398c6d2b	- Add a log entry for repo verb handling and make the container usage calculation take it into account ... - Move all the repo push/pull/verb logging into a central track_and_log method - Readd images accidentally deleted in the last CL - Make the uncompressed size migration script better handle exceptions	2014-10-29 15:42:44 -04:00
Joseph Schorr	e9c2e54dbc	Change delete to once again not perform everything under a transaction. A recent revision accidentally moved everything back under it.	2014-10-29 11:59:02 -04:00
Joseph Schorr	109850b428	Add a basic usage counter for enterprise	2014-10-28 16:33:13 -04:00
Joseph Schorr	c06f57a6e7	Make sure builders close the db handle when no work comes in and make the metrics transaction smaller in scope	2014-10-24 11:40:02 -04:00
Jake Moshenko	1461310ab8	Merge remote-tracking branch 'origin/master' into nomenclature ... Conflicts: endpoints/common.py endpoints/notificationhelper.py test/data/test.db workers/dockerfilebuild.py	2014-10-23 13:25:37 -04:00
Joseph Schorr	5db9cd948b	Add better (jinja-based) messaging to the notifications and add some fixes for the email templates	2014-10-22 19:01:56 -04:00
Joseph Schorr	28a463f998	Make the 'created' field optional, in case the user does not specify it via the API	2014-10-20 12:07:49 -04:00
Jake Moshenko	dc5ee43a3a	MySQL still doesn't allow us to have empty IN clauses.	2014-10-17 17:49:18 -04:00
Jake Moshenko	6ca0115b5e	MySQL doesnt allow us to use the table we are modifying in a subquery.	2014-10-17 17:01:55 -04:00
Jake Moshenko	efe8825a15	We can't even use an empty in query for images.	2014-10-17 16:37:05 -04:00
Jake Moshenko	acd5a1e07e	Only garbage collect storage when it might result in something. Return the count of the removed images to fulfill the contract expected by some callers.	2014-10-17 16:13:32 -04:00
Jake Moshenko	18a944427f	Merge branch 'master' of bitbucket.org:yackob03/quay	2014-10-17 15:26:58 -04:00
Jake Moshenko	c093e5a326	Add a whitelist of candidate storages which will speed up the orphan queries and limit the damage of GC run amok.	2014-10-17 15:26:51 -04:00
Joseph Schorr	26a9bb50d5	Fix spelling mistake	2014-10-17 14:44:17 -04:00
Jake Moshenko	d8149295ab	Update the GC code to do everything with subqueries, making each GC run a bounded finite number of queries with a fixed length.	2014-10-17 14:33:04 -04:00
Jake Moshenko	380eb49e58	Split out the GC code into smaller transactions. Remove from the actual storage component after the fact, outside of the transactions.	2014-10-17 11:42:09 -04:00
Jake Moshenko	fa6a06502d	Change the default redis host to localhost. Fix some whitespace issues in the userevents module.	2014-10-14 14:37:02 -04:00
Jake Moshenko	328db8b660	Split the app into separate backends, which can use different worker types and different timeouts.	2014-10-14 13:58:08 -04:00
Joseph Schorr	bb9502ee77	Make sure to raise the exception for WSGI and make the delete method's intent more clear	2014-10-08 13:51:50 -04:00
Joseph Schorr	d16fdde528	Fix bug in dockerloadformat and make sure we handle exceptions properly in the verb call	2014-10-08 13:43:12 -04:00
Jake Moshenko	c34a8b6727	Fix the squashed image migration to insert the proper image transformation type.	2014-10-08 11:47:02 -04:00
Jake Moshenko	1a8287af55	Missed a table import.	2014-10-07 17:33:20 -04:00
Jake Moshenko	bbcdf18fdd	Update the migration to reflect the state of the database.	2014-10-07 16:19:05 -04:00
Jake Moshenko	1968c67cba	Merge remote-tracking branch 'origin/laffa' into nomenclature	2014-10-07 16:09:45 -04:00
Jake Moshenko	626d984fb2	Fixes to the migration generator script.	2014-10-07 16:09:30 -04:00
Jake Moshenko	fe59ad4fb5	Rectify our prod database, previous migrations, and database definition files.	2014-10-07 16:03:57 -04:00
Jake Moshenko	c77a3dc6ea	Update the migration for translating queue names to take into account previous migrations.	2014-10-07 15:57:17 -04:00
Jake Moshenko	a0d94f9d59	Merge remote-tracking branch 'origin/laffa' into nomenclature ... Conflicts: test/data/test.db	2014-10-07 15:56:28 -04:00
Jake Moshenko	ee9973a395	Merge remote-tracking branch 'origin/master' into nomenclature	2014-10-07 15:43:59 -04:00
Joseph Schorr	f4daa5e97b	- Update the migrations tool to verify migrations work up and down for both MySQL and PostgresSQL. ... - Add migrations for the squashed image tables and for backfilling the uncompressed sizes - Make sure gzip stream uses a max length when determining the uncompressed size	2014-10-07 15:29:56 -04:00
Joseph Schorr	f38ce51943	Merge master into laffa	2014-10-07 14:03:17 -04:00
Jake Moshenko	4ad592e7ce	Add an index to the image storage uuid to improve performance.	2014-10-06 18:44:37 -04:00
Jake Moshenko	c4266140e2	Fix all of the upgrades and downgrades to work on both mysql and postgres.	2014-10-06 16:41:49 -04:00