vbatts/quay
Archived
1
0
Fork 0
Code Issues Pull requests Releases Wiki Activity
3886 commits 2 branches 62 tags 205 MiB
Commit graph

713 commits

Author SHA1 Message Date
Joseph Schorr
b8e873b00b Add support to the build system for tracking if/when the build manager crashes and make sure builds are restarted within a few minutes 2014-11-21 14:27:06 -05:00
Jimmy Zelinskie
c918d15979 Add a migration for backfilling user UUIDs. 2014-11-20 18:28:54 -05:00
Jake Moshenko
2b8c246476 Temporarily put user rename behind a feature flag. Switch queue names back to using the username for namespace while we figure out a real migration strategy. 2014-11-20 15:36:39 -05:00
Jimmy Zelinskie
e6742b00b0 Remove backfill from UUID migration 2014-11-20 14:04:12 -05:00
Joseph Schorr
8fab3b6d34 Make sure to switch the context back to LogEntry and make sure to only include the performer if actually present 2014-11-20 12:04:53 -05:00
Joseph Schorr
cc7bccea4c Add the logentrykind and performer information to log listing. This makes the system much faster, since new queries are not needed for every log entry. 2014-11-20 12:03:16 -05:00
Joseph Schorr
9b31b9805a Fix performance problem with looking up org members and add some tests 2014-11-20 11:33:42 -05:00
Jake Moshenko
ac0d0ce36a Change the order of migrations to allow migration code to work with the updated model. 2014-11-19 15:58:47 -05:00
Jake Moshenko
e863b96166 Tweak the uuid backfill to leave the uuid column nullable. 2014-11-19 15:32:30 -05:00
Jimmy Zelinskie
7c8a438b58 Rebase migration on top of master and add backfill. 2014-11-19 14:07:54 -05:00
Jimmy Zelinskie
606ad21bec Apply reviewed changes. 
Adds a length to the UUID field, renames QuayDeferredPermissionUser
parameter id->uuid, adds transactions to backfill script.
 2014-11-19 13:28:16 -05:00
Jimmy Zelinskie
9d677b8eb3 Add UUID to User model and use in cookie. 2014-11-19 13:28:16 -05:00
Jake Moshenko
17fc72d262 Switch postgres to a non-transactional DDL to allow us to use peewee to modify data in migrations: enterprise customers are running postgres migrations offline already. Move the image backfill script back to a migration since it will now work. Unify the interface to sending a DB URI to env.py for the migration script. 2014-11-18 14:07:33 -05:00
Jake Moshenko
a7bae6c1d9 Fix robot renaming. Allow for trigger URLs to contain or omit the repository in the path. Fix calls to get_trigger to remove the namespace and repository. 2014-11-18 10:24:48 -05:00
Jake Moshenko
f4681f2c18 Merge branch 'master' into nomenclature 
Conflicts:
	test/data/test.db
 2014-11-17 17:59:59 -05:00
Joseph Schorr
58ca76239b Add ability to one-click generate an authorization access token in the applications panel 2014-11-17 14:54:07 -05:00
Joseph Schorr
178c5a7ac0 Add an index to the docker_image_id for faster lookup 2014-11-13 12:51:50 -05:00
Joseph Schorr
d73747ce1d - Fix some other group_by clauses 
- Fix garbage_collect for an empty list (fixes a test)
- Add a script which runs the full test suite against mysql and postgres (note: QueueTest's are broken for MySQL, but they obviously work in production, so they need to be fixed)
 2014-11-13 12:51:37 -05:00
Joseph Schorr
a2e2dcb010 Fix group_by clauses for postgres 2014-11-12 20:32:06 -05:00
Joseph Schorr
079289c53c HACK: Don't join on the visibility table because it is horrendously slow for some odd reason. 2014-11-12 14:51:24 -05:00
Joseph Schorr
eddcc02ea6 Make repository deletes much faster by adding custom deletion code and have additional tests to verify the deletion code paths 2014-11-10 23:05:20 -05:00
Joseph Schorr
e7cbda86f7 Merge branch 'perf' 2014-11-10 21:52:27 -05:00
Joseph Schorr
091f821a6a - Rename get_repo_image to get_repo_image_extended and get_repo_image_directly to get_repo_image 
- Remove the configure call from CloseForLongOperation
- Other small fixes
 2014-11-10 13:44:36 -05:00
Joseph Schorr
72fedef097 Move the robot deletion code into a delete_instance method on the User object 2014-11-10 13:18:17 -05:00
Jake Moshenko
ca435fc7a6 Rename robots when we rename a user. Do not use the namespace from the path to check permissions from the incoming webhooks since the namespace may have changed and we cannot recreate them in remote services easily. 2014-11-09 17:50:57 -05:00
Joseph Schorr
17f605a9ef Select only a single token. 2014-11-09 15:50:50 -05:00
Joseph Schorr
691be49817 Fix issues with the perf updated code 2014-11-07 14:36:32 -05:00
Joseph Schorr
158acd4f41 - Turn on foreign key constraint checking in the tests 
- Change all ForeignKeyField's that refer to users to use our custom class, and mark those that allow robots
- Change robot delete to only execute the subset of queries necessary to actually delete robots
 2014-11-07 12:05:21 -05:00
Joseph Schorr
d5bbb57481 Change registry code to disconnect from the DB before long I/O operations 2014-11-06 18:00:52 -05:00
Joseph Schorr
23d9bd2b42 Change verbs to use short lived database connections 2014-11-06 17:50:48 -05:00
Joseph Schorr
c569299e5c Database optimizations around image creation and logs lookup 2014-11-06 14:48:16 -05:00
Joseph Schorr
a35bc11912 Add perf comments 2014-11-05 12:27:38 -05:00
Joseph Schorr
29c30b336e Move the uncompressed image size migration call outside of alembic, since it will sometimes deadlock with certain kinds of DBs (because alembic is running things inside a transaction) 2014-11-03 21:01:41 -05:00
Joseph Schorr
9aa72c5cc2 Fix migration issues: 
- MySQL 5.5 doesn't support the now() call as a default
  - Postgres migration isn't auto-committed, so we have to check if the table exists first
 2014-11-03 15:25:55 -05:00
Joseph Schorr
4eedd54b66 - Make usage language more accurate by stating "repositories" 
- Have usage counter be based on a 4 weeks TTL
- Add a simple usage counter breakage test
 2014-10-30 13:26:02 -04:00
Joseph Schorr
c1398c6d2b - Add a log entry for repo verb handling and make the container usage calculation take it into account 
- Move all the repo push/pull/verb logging into a central track_and_log method
- Readd images accidentally deleted in the last CL
- Make the uncompressed size migration script better handle exceptions
 2014-10-29 15:42:44 -04:00
Joseph Schorr
e9c2e54dbc Change delete to once again not perform everything under a transaction. A recent revision accidentally moved everything back under it. 2014-10-29 11:59:02 -04:00
Joseph Schorr
109850b428 Add a basic usage counter for enterprise 2014-10-28 16:33:13 -04:00
Joseph Schorr
c06f57a6e7 Make sure builders close the db handle when no work comes in and make the metrics transaction smaller in scope 2014-10-24 11:40:02 -04:00
Jake Moshenko
1461310ab8 Merge remote-tracking branch 'origin/master' into nomenclature 
Conflicts:
	endpoints/common.py
	endpoints/notificationhelper.py
	test/data/test.db
	workers/dockerfilebuild.py
 2014-10-23 13:25:37 -04:00
Joseph Schorr
5db9cd948b Add better (jinja-based) messaging to the notifications and add some fixes for the email templates 2014-10-22 19:01:56 -04:00
Joseph Schorr
28a463f998 Make the 'created' field optional, in case the user does not specify it via the API 2014-10-20 12:07:49 -04:00
Jake Moshenko
dc5ee43a3a MySQL still doesn't allow us to have empty IN clauses. 2014-10-17 17:49:18 -04:00
Jake Moshenko
6ca0115b5e MySQL doesnt allow us to use the table we are modifying in a subquery. 2014-10-17 17:01:55 -04:00
Jake Moshenko
efe8825a15 We can't even use an empty in query for images. 2014-10-17 16:37:05 -04:00
Jake Moshenko
acd5a1e07e Only garbage collect storage when it might result in something. Return the count of the removed images to fulfill the contract expected by some callers. 2014-10-17 16:13:32 -04:00
Jake Moshenko
18a944427f Merge branch 'master' of bitbucket.org:yackob03/quay 2014-10-17 15:26:58 -04:00
Jake Moshenko
c093e5a326 Add a whitelist of candidate storages which will speed up the orphan queries and limit the damage of GC run amok. 2014-10-17 15:26:51 -04:00
Joseph Schorr
26a9bb50d5 Fix spelling mistake 2014-10-17 14:44:17 -04:00
Jake Moshenko
d8149295ab Update the GC code to do everything with subqueries, making each GC run a bounded finite number of queries with a fixed length. 2014-10-17 14:33:04 -04:00
Jake Moshenko
380eb49e58 Split out the GC code into smaller transactions. Remove from the actual storage component after the fact, outside of the transactions. 2014-10-17 11:42:09 -04:00
Jake Moshenko
fa6a06502d Change the default redis host to localhost. Fix some whitespace issues in the userevents module. 2014-10-14 14:37:02 -04:00
Jake Moshenko
328db8b660 Split the app into separate backends, which can use different worker types and different timeouts. 2014-10-14 13:58:08 -04:00
Joseph Schorr
bb9502ee77 Make sure to raise the exception for WSGI and make the delete method's intent more clear 2014-10-08 13:51:50 -04:00
Joseph Schorr
d16fdde528 Fix bug in dockerloadformat and make sure we handle exceptions properly in the verb call 2014-10-08 13:43:12 -04:00
Jake Moshenko
c34a8b6727 Fix the squashed image migration to insert the proper image transformation type. 2014-10-08 11:47:02 -04:00
Jake Moshenko
1a8287af55 Missed a table import. 2014-10-07 17:33:20 -04:00
Jake Moshenko
bbcdf18fdd Update the migration to reflect the state of the database. 2014-10-07 16:19:05 -04:00
Jake Moshenko
1968c67cba Merge remote-tracking branch 'origin/laffa' into nomenclature 2014-10-07 16:09:45 -04:00
Jake Moshenko
626d984fb2 Fixes to the migration generator script. 2014-10-07 16:09:30 -04:00
Jake Moshenko
fe59ad4fb5 Rectify our prod database, previous migrations, and database definition files. 2014-10-07 16:03:57 -04:00
Jake Moshenko
c77a3dc6ea Update the migration for translating queue names to take into account previous migrations. 2014-10-07 15:57:17 -04:00
Jake Moshenko
a0d94f9d59 Merge remote-tracking branch 'origin/laffa' into nomenclature 
Conflicts:
	test/data/test.db
 2014-10-07 15:56:28 -04:00
Jake Moshenko
ee9973a395 Merge remote-tracking branch 'origin/master' into nomenclature 2014-10-07 15:43:59 -04:00
Joseph Schorr
f4daa5e97b - Update the migrations tool to verify migrations work up and down for both MySQL and PostgresSQL. 
- Add migrations for the squashed image tables and for backfilling the uncompressed sizes
- Make sure gzip stream uses a max length when determining the uncompressed size
 2014-10-07 15:29:56 -04:00
Joseph Schorr
f38ce51943 Merge master into laffa 2014-10-07 14:03:17 -04:00
Jake Moshenko
4ad592e7ce Add an index to the image storage uuid to improve performance. 2014-10-06 18:44:37 -04:00
Jake Moshenko
c4266140e2 Fix all of the upgrades and downgrades to work on both mysql and postgres. 2014-10-06 16:41:49 -04:00
Jake Moshenko
ed8bcff39e Merge remote-tracking branch 'origin/master' into nomenclature 
Conflicts:
	test/data/test.db
	workers/dockerfilebuild.py
 2014-10-06 10:29:39 -04:00
Jake Moshenko
5c18ffe67d Allow the namespace column to be null, and also non-unique. Fix the uncompressed size clobbering the size on the wire field. Add metadata constraints so that foreign key constraints get predictable names. Fix all downgrade migrations. 2014-10-02 10:46:20 -04:00
Jake Moshenko
e8b3d1cc4a Phase 4 of the namespace to user migration: actually remove the column from the db and remove the dependence on serialized namespaces in the workers and queues 2014-10-01 14:23:46 -04:00
Joseph Schorr
2c5cc7990f Allow for additional REDIS config such as password and port 2014-10-01 14:16:42 -04:00
Joseph Schorr
6c520b8b0b Fixed broken query 2014-09-30 13:19:32 -04:00
Joseph Schorr
5f776771d1 Merge branch 'master' of https://bitbucket.org/yackob03/quay 2014-09-29 17:01:07 -04:00
Joseph Schorr
474add0fb1 Have the layer PUT method calculate the uncompressed size in realtime, as trusting the JSON is fraught with complications 2014-09-29 17:00:47 -04:00
Jake Moshenko
fbc7ee3bce Phase 3 of the namespace migration, remove write references to the namespace string column. 2014-09-29 15:44:25 -04:00
Jake Moshenko
ae9026ee91 MySQL will not allow us to revoke the possibility for the namespace_user column to be nullable. 2014-09-26 14:50:26 -04:00
Jake Moshenko
21d8c41320 Add the existing types to the alter column to make MySQL happy. 2014-09-26 12:58:03 -04:00
Jake Moshenko
e9745dbc96 Fix the SQL migration. 2014-09-26 11:17:42 -04:00
Jake Moshenko
03190efde3 Phase 2 of migrating repo namespaces to referencing user objects, backfilling the rows without a value for namespace_user, and changing all accesses to go through the namespace_user object. All tests are passing, manual testing still required. 2014-09-24 18:01:35 -04:00
Jake Moshenko
98d8e0fe37 Fixes for the uncompressed size backfill script. 2014-09-23 16:06:59 -04:00
Joseph Schorr
ec484e3efc Move size and checksum updates into the metadata call 2014-09-23 15:49:28 -04:00
Jake Moshenko
74e35f917e Switch socket_timeout to socket_connect_timeout for redis. 2014-09-23 14:59:00 -04:00
Jake Moshenko
798c13aec8 Fix the migration to use the proper column name that peewee expects. 2014-09-23 14:20:15 -04:00
Jake Moshenko
b6d635b648 Fix the migration to use the proper base. 2014-09-23 14:04:20 -04:00
Joseph Schorr
86dfca2e3e Add uncompressed size field to the image storage and add a backfill script (which is not yet automatically called) 2014-09-23 14:01:27 -04:00
Joseph Schorr
87bc37f6c8 Merge branch 'sunday' 2014-09-23 11:29:03 -04:00
Joseph Schorr
ba0963a81c Update the worker code to better handle exceptions, fix the utcdate issue and make sure we send the proper retry. Also updates notification workers to send JobExceptions rather than returning true or false 2014-09-23 11:01:50 -04:00
Joseph Schorr
f3b03ebc34 Add a feature flag for disabling all emails 2014-09-22 19:11:48 -04:00
Jake Moshenko
c68d49dfce Fix the get_matching_users query to work with peewee 2.3+ 2014-09-22 18:42:48 -04:00
Jake Moshenko
a6225ad34a Set an aggressive timeout on the redis connections to allow the unit tests to run when communication with redis is disabled. 2014-09-22 18:42:22 -04:00
Jake Moshenko
3259cda000 The new strategy is to do a three phase migration. This is the first phase: getting the namespace user in the db and written for all new repositories. 2014-09-22 17:27:02 -04:00
Joseph Schorr
297c8ad29c Add migration to backfill uncompressed image sizes on the storage 2014-09-22 15:04:28 -04:00
Joseph Schorr
1658475ac1 Previous revision should not have the image storage col in it 2014-09-22 14:39:44 -04:00
Joseph Schorr
f16878cce9 Add migration for synthetic image tables 2014-09-22 14:38:42 -04:00
Joseph Schorr
70e0aba257 Add a script for generating schema migrations. Should be run from the root quay directory. 2014-09-22 14:36:52 -04:00
Joseph Schorr
f23038c6ee Update the worker code to better handle exceptions, fix the utcdate issue and make sure we send the proper retry. Also updates notification workers to send JobExceptions rather than returning true or false 2014-09-22 12:52:57 -04:00
Jake Moshenko
8626d1cd70 Initial changes to move repositories from using a namespace string to referencing a user object. Also stores the user id in the cookie rather than the username, to allow users to be renamed. This commit must not be used unmodified because the database migration is too aggressive for live migration. 2014-09-19 10:17:23 -04:00
Jake Moshenko
8c00eabedd Merge remote-tracking branch 'origin/better-emails' 2014-09-19 10:04:02 -04:00
Joseph Schorr
6b05b55225 Add unicode password support 2014-09-18 17:36:26 -04:00
Jake Moshenko
11bb8e6448 Actually store the generated image storage in the database, and allow it to be garbage collected when the parent image storage is collected. 2014-09-18 17:26:40 -04:00
Joseph Schorr
b212dbb2ab Merge branch 'master' into better-emails 2014-09-18 13:20:32 -04:00
Jake Moshenko
8b5e7621d2 Add the required notification kind for team invites to the migration that was previously run. 2014-09-16 10:11:18 -04:00
Joseph Schorr
e3c52fa0eb Work in progress. This is currently broken! 2014-09-16 00:18:57 -04:00
Jake Moshenko
e13c027631 Add a migration for team member invites. 2014-09-15 23:57:05 -04:00
Jake Moshenko
75d2ef377e Merge remote-tracking branch 'origin/master' into comewithmeifyouwanttowork 
Conflicts:
	data/model/legacy.py
 2014-09-15 17:52:17 -04:00
Jake Moshenko
2b59a0cbe1 Merge branch 'master' of bitbucket.org:yackob03/quay 2014-09-15 15:59:18 -04:00
Jake Moshenko
efb66f7c1e Select the random row function based on DB driver. 2014-09-15 15:58:56 -04:00
Joseph Schorr
913b3e472f Add ability to detach external login services 2014-09-15 12:01:02 -04:00
Joseph Schorr
e8ad01cb41 Lots of small NPE and other exception fixes 2014-09-15 11:27:33 -04:00
Joseph Schorr
10faa7de84 Only allow users matching the team invite to accept, if the invite was specified for a user (rather than an email) 2014-09-12 14:29:01 -04:00
Jake Moshenko
c01de4a916 Set redis logs entries to expire rather than to immediately delete them to make the logs archiver idempotent. 2014-09-12 13:13:14 -04:00
Jake Moshenko
353da13b16 Set up the service monitoring for the log migration script. Add a database migration to add the logs_archived column. 2014-09-12 11:57:47 -04:00
Jake Moshenko
b55e79e5d3 Merge remote-tracking branch 'origin/master' into waltermitty 2014-09-12 11:45:07 -04:00
Jake Moshenko
c5ca46a14b Merge remote-tracking branch 'origin/master' into comewithmeifyouwanttowork 
Conflicts:
	data/model/legacy.py
	static/js/app.js
 2014-09-12 11:03:30 -04:00
Jake Moshenko
5388633f9a Merge remote-tracking branch 'origin/master' into pullfail 2014-09-12 10:36:38 -04:00
Joseph Schorr
8d3ce44682 Address comments on code review 2014-09-11 15:45:41 -04:00
Jake Moshenko
8b3a3178b0 Finish the build logs archiver, add handlers for cloud and local that handle gzip encoded archived content. 2014-09-11 15:33:10 -04:00
Jake Moshenko
2455c17f96 Merge remote-tracking branch 'origin/master' into waltermitty 
Conflicts:
	app.py
	data/userfiles.py
 2014-09-11 11:18:28 -04:00
Jake Moshenko
548f855f71 Use the pure python io module to avoid some interaction between gunicorn, wsgi, and bufferedreader that prevents gunicorn from properly sending the files. 2014-09-09 22:28:25 -04:00
Jake Moshenko
c9e1648781 Small fixes to bugs in the streaming handler for use with magic and radosgw. 2014-09-09 18:30:14 -04:00
Jake Moshenko
756e8ec848 Send the content type through to the cloud engines. 2014-09-09 16:52:53 -04:00
Jake Moshenko
29d40db5ea Add a new RadosGW storage engine. Allow engines to distinguish not only between those that can support direct uploads and downloads, but those that support doing it through the browser. Rename resumeable->resumable. 2014-09-09 15:54:03 -04:00
Joseph Schorr
7c45aca405 Code review changes 2014-09-08 17:20:01 -04:00
Jake Moshenko
451e034ca1 Archived logs commit 1. Squash me. 2014-09-08 16:43:17 -04:00
Jake Moshenko
54fbb2a4c0 Rename collections to morecollections to avoid a conflict with the built in module. 2014-09-08 16:42:43 -04:00
Joseph Schorr
3c20402b32 Add a common base email template, translate the emails over to using jinja and add emails when e-mail addresses and passwords are changed. 2014-09-05 19:57:33 -04:00
Jake Moshenko
64480fd4ed Merge remote-tracking branch 'origin/master' into yellowalert 
Conflicts:
	data/migrations/versions/82297d834ad_add_us_west_location.py
	test/data/test.db
 2014-09-05 11:30:30 -04:00
Jake Moshenko
c7e873366d Inject the tables metadata into the upgrade and downgrade functions. Fix a bunch of the downgrades to actually work. 2014-09-04 20:58:29 -04:00
Joseph Schorr
6fa5a365b3 Add loginservice for Google 2014-09-04 18:45:23 -04:00
Joseph Schorr
fa1abd5eda Don't require the invite email to match the invited user 2014-09-04 18:42:23 -04:00
Joseph Schorr
b9a4d2835f Add migration for the new DB field 2014-09-04 18:18:19 -04:00
Joseph Schorr
e028d4ae0a Merge master into branch 2014-09-04 18:08:18 -04:00
Jake Moshenko
8910c6ff01 Add a migration to remove the webhooks table. 2014-09-03 13:44:05 -04:00
Jake Moshenko
21f7acf7ca Fix the default value for the migration to use a string 2014-09-03 13:34:36 -04:00
Jake Moshenko
0bd9ba523e Add a migration for the brute force prevention fields to the user table. 2014-09-03 13:07:53 -04:00
Jake Moshenko
2dcdd7ba5b Add exponential backoff of login attempts. 2014-09-02 15:27:05 -04:00
Jake Moshenko
68ecebc704 Merge branch 'master' of ssh://bitbucket.org/yackob03/quay 2014-08-29 13:16:41 -04:00
Jake Moshenko
ce7e3a8733 Do not link against layers that are still marked as uploading, there is no guarantee that they will ever be completed and their ancestry may be incomplete. 2014-08-29 13:16:32 -04:00
Joseph Schorr
3b72b26836 Merge branch 'master' into comewithmeifyouwanttowork 2014-08-28 20:50:13 -04:00
Joseph Schorr
f15b3f345e Review updates 2014-08-28 19:07:22 -04:00
Joseph Schorr
85ab7a8c8d Fix migration downgrade for the regenerating robot kind 2014-08-28 18:40:33 -04:00
Joseph Schorr
9142ff69a3 Add migration for new notification kinds 2014-08-28 18:39:35 -04:00
Joseph Schorr
d76d4704a0 Add pagination to the notifications API and make the UI only show a maximum of 5 notifications (beyond that, it shows "5+"). 2014-08-26 15:19:39 -04:00
Joseph Schorr
a129aac94b Add ability to regenerate robot account credentials 2014-08-25 17:19:23 -04:00
Joseph Schorr
d2880807b2 - Further fixes for license stuff 
- Small fixes to ensure Quay works for Postgres
 2014-08-21 19:21:20 -04:00
Joseph Schorr
d5027d2383 Add a migration script for the new table and log entry kinds 2014-08-18 20:45:48 -04:00
Joseph Schorr
daa43c3bb9 Add better messaging around pulling of base images when they fail due to invalid or missing credentials 2014-08-18 20:34:39 -04:00
Joseph Schorr
02d3b70013 Make sure to search teams as well when determining the robots which have access to a private repo 2014-08-18 19:19:01 -04:00
Joseph Schorr
43b6695f9c Get team invite confirmation working and fully tested 2014-08-18 17:24:00 -04:00
Joseph Schorr
56d7a3524d Work in progress: Require invite acceptance to join an org 2014-08-15 17:47:43 -04:00
Joseph Schorr
f6f857eec2 Add US West region 2014-08-15 15:02:26 -04:00
Joseph Schorr
27c2680380 Plan update and fix about page 2014-08-13 14:25:48 -04:00
Joseph Schorr
bd4bbe0bdb Add missing image location for enterprise 2014-08-12 22:29:17 -04:00
Joseph Schorr
389c88a7c4 Update federated login to store metadata and have the UI pull the information from the metadata 2014-08-11 18:25:01 -04:00
yackob03
420d02cd71 Fix the migration script to add and remove the logentrykinds for notifications. 2014-08-05 14:21:29 -07:00
yackob03
b07eff41f6 Switch the table names to use lower case. 2014-08-05 13:44:18 -07:00
yackob03
cd5af72ba8 Temporarily prevent the deletion of the webhooks table to allow a migration path which keeps prod working. 2014-08-04 13:44:22 -07:00
Joseph Schorr
2c3fe2e60f Switch web hook migration script to fully use the alembic context and connection 2014-08-01 13:37:27 -04:00
Joseph Schorr
a44345338d Add bidirectional migration code of webhooks to/from repo-push webhook notifications. 2014-07-31 16:02:14 -04:00
Jake Moshenko
ee2abf590a Merge branch 'redalert' of ssh://bitbucket.org/yackob03/quay into redalert 2014-07-31 14:01:17 -04:00
Jake Moshenko
3f6466fccd Add a migration to move the existing webhooks over and another migration to remove the legacy webhooks table. 2014-07-31 14:01:03 -04:00
Joseph Schorr
346d3491ff Merge branch 'redalert' of https://bitbucket.org/yackob03/quay into redalert 2014-07-31 13:31:16 -04:00
Joseph Schorr
49801bc2c4 - Add web hook queue code back in. We'll remove it and turn it off after this CL goes to prod 
- Make notification lookup always be by repo and its UUID, rather than the internal DB ID
- Add the init script for the notification worker
 2014-07-31 13:30:54 -04:00
Jake Moshenko
53a1f62614 Create a migration to prepare the database for the new notification types. 2014-07-31 13:27:05 -04:00
Joseph Schorr
7de1dd7dc0 Merge branch 'master' into redalert 2014-07-28 18:35:39 -04:00
Joseph Schorr
32b2ecdfa6 Add ability to dismiss notifications 2014-07-28 18:23:46 -04:00
Joseph Schorr
34fc279092 Add e-mail authorization to the repository notification flow. Also validates the creation of the other notification methods. 2014-07-28 14:58:12 -04:00
Jake Moshenko
9ad7fc7a54 Prevent extra queries to load image storage data. 2014-07-22 16:46:36 -04:00
Joseph Schorr
752efb9e0f Fix the spawn_notification to work in all cases and clean up some of the remaining code 2014-07-18 16:34:52 -04:00
Joseph Schorr
591cd020b8 Merge branch 'master' into redalert 2014-07-18 15:58:56 -04:00
Jake Moshenko
8a2c88488a Filter tokens with an associated build trigger from the list API. Delete associated tokens when deleting a trigger. 2014-07-18 14:56:26 -04:00
Joseph Schorr
8d7493cb86 Convert over to notifications system. Note this is incomplete 2014-07-17 22:51:58 -04:00
Joseph Schorr
de8e898ad0 Add UI for managing repo notifications 2014-07-17 13:32:39 -04:00
Joseph Schorr
a84fe0681a Start on data model changes and API changes for the new repository notification system 2014-07-16 16:30:47 -04:00
Jake Moshenko
5645b6da32 Add support for read slave databases. 2014-07-02 19:10:24 -04:00
Jake Moshenko
d851feef6e One of the calls which returns an image was not using the base format which includes locations yet. 2014-06-30 14:51:01 -04:00
Jake Moshenko
684c26bc1a Populate image storage objects with locations everywhere. 2014-06-27 20:04:26 -04:00
Jake Moshenko
e68d6a7302 Merge branch 'master' of ssh://bitbucket.org/yackob03/quay 2014-06-27 19:18:35 -04:00
Jake Moshenko
4e5e8a08de Add a maintenance notification kind and make it of the level warning. 2014-06-27 19:18:27 -04:00
Joseph Schorr
76165b5d2b Add API and UI support for displaying image locations 2014-06-24 18:48:42 -04:00
Jake Moshenko
6047f3759f Remove the placement fallback since the DB has been fully backfilled. 2014-06-24 17:01:23 -04:00
Jake Moshenko
6e3a545c7f Add a migration to upgrade to the version of the database supporting placements. 2014-06-18 17:46:20 -04:00
Jake Moshenko
0a62f7f725 Add the ability to look up images which do not have a placement yet. 2014-06-18 12:40:23 -04:00
Jake Moshenko
bf0e01fba5 Fix bugs with pushing, pulling, and deleting. 2014-06-17 16:37:48 -04:00
Jake Moshenko
bf98575feb Add the basics of geographic data distribution and get the tests to work. 2014-06-17 16:03:43 -04:00
Jake Moshenko
6d07cf94d2 Add a migration to remove the image fields that were migrated to image storage, and add some uniqueness constraints to some indexes. 2014-06-12 19:51:57 -04:00
Jake Moshenko
78c5aec5b9 Switch the checksums to use the registry computed value, remove all assumptions of namespaced paths for legacy storage, fix an upload race condition in the registry code. 2014-06-11 15:37:45 -04:00
Jake Moshenko
d1f4fbdacc Split out the redis hostname for user events and build logs as a string config. Modularize the user events and fix all callers. 2014-05-30 14:25:29 -04:00
Jake Moshenko
0ba4201020 Add a module which will create notifications for all users when the license is at its expiration period, and terminate the process when the license expires. 2014-05-29 11:24:10 -04:00
Jake Moshenko
22cc7a85d4 Add exp_month and exp_year to the fake card to fix the tests. 2014-05-28 14:06:37 -04:00
Jake Moshenko
33b43b75c0 Eliminate a lot of the if cases in create_user by separating them out. Add a limit to the number of users which can be created based on the license. Add support for creating and loading licenses. 2014-05-28 13:51:52 -04:00
Jake Moshenko
0b6552d6cc Fix the metrics so they are usable for scaling the workers down and up. Switch all datetimes which touch the database from now to utcnow. Fix the worker Dockerfile. 2014-05-23 14:16:26 -04:00
Jake Moshenko
f4c488f9b6 Fix the queue query for old jobs which won't run. 2014-05-22 13:50:06 -04:00
Jake Moshenko
f6726bd0a4 Merge branch 'ldapper' 
Conflicts:
	Dockerfile
	app.py
	data/database.py
	endpoints/index.py
	test/data/test.db
 2014-05-22 12:13:41 -04:00
Jake Moshenko
d14798de1d Add a queue capacity reporter plugin to the queue. Move the queue definitions to app. Add a cloudwatch reporter to the dockerfile build queue. 2014-05-21 19:50:37 -04:00
Jake Moshenko
e1a5a3e543 Temporarily remove the db pool until we can figure out why they are running out of connections. 2014-05-19 17:14:23 -04:00
Jake Moshenko
8b5c781f84 Change the health check to ping the db and the redis build logs. 2014-05-13 18:53:42 -04:00
Jake Moshenko
11c6c5fa52 Merge remote-tracking branch 'origin/master' into ldapper 
Conflicts:
	app.py
 2014-05-13 16:55:02 -04:00
Jake Moshenko
628d09afe0 Remove the passwd attr ldap config. 2014-05-13 15:52:20 -04:00
Jake Moshenko
2da8b4737e Fix the registry to work with unicode usernames in LDAP. 2014-05-13 15:22:31 -04:00
Jake Moshenko
f049f738da Run the db migrations on container start unless we're running against Sqlite. 2014-05-13 15:20:17 -04:00
Jake Moshenko
553ef36e9b Fix a remaining direct usage of transaction factory in the data model. 2014-05-13 15:17:16 -04:00
Jake Moshenko
5fdccfe3e6 Add an alembic migration for the full initial database with the data. Switch LDAP to using bind and creating a federated login entry. Add LDAP support to the registry and index endpoints. Add a username transliteration and suggestion mechanism. Switch the database and model to require a manual initialization call. 2014-05-13 12:17:26 -04:00
Jake Moshenko
bcb993a914 Set up the build logs to use our fake build logs on test and local. 2014-05-09 18:45:11 -04:00
Jake Moshenko
027ada1f5c First stab at LDAP integration. 2014-05-09 17:39:43 -04:00
Jake Moshenko
feb7ad8dd1 Use the pooled mysql database. 2014-05-07 12:37:45 -04:00
Jake Moshenko
8a3af93b8c Improve the builder response to being terminated or dying. 2014-05-06 18:46:19 -04:00
Jake Moshenko
ef9fe871fc Move the upload flag to the database, and use the database stored image size rather than going to s3. 2014-05-02 16:59:46 -04:00
jakedt
44e514f49f Fix the test stripe to suppor trial properties. 2014-04-25 14:29:08 -04:00
jakedt
893e5136a0 Make the local userfiles 404 if the file is not there. Remove some extraneous logging from the trigger. 2014-04-16 22:43:57 -04:00
jakedt
be728ceccb Fix the implementation of local userfiles and switch master_branch to default_branch to match the github api. 2014-04-16 22:35:47 -04:00
jakedt
0827e0fbac Merge remote-tracking branch 'origin/master' into ncc1701 
Conflicts:
	endpoints/web.py
	static/directives/signup-form.html
	static/js/app.js
	static/js/controllers.js
	static/partials/landing.html
	static/partials/view-repo.html
	test/data/test.db
 2014-04-14 19:37:22 -04:00
jakedt
4b8217d4ad Add config to allow for setting the queue names at runtime. Fix a bug in the data model. 2014-04-11 19:23:57 -04:00
jakedt
61a6db236f Finish the implementation of local userfiles. Strip charsets from mimetypes in the build worker. Add canonical name ordering to the build queue. Port all queues to the canonical naming version. 2014-04-11 18:34:47 -04:00
jakedt
14fba3ae7c Rename SERVER_NAME to SERVER_HOSTNAME to fix the subdomain routing problems. 2014-04-11 11:17:45 -04:00
jakedt
8fac0474b5 Get staging to run under docker on an EC2 host. 2014-04-10 18:30:09 -04:00
jakedt
73f23f155c Merge branch 'ncc1701' of ssh://bitbucket.org/yackob03/quay into ncc1701 2014-04-10 15:20:26 -04:00
jakedt
d39f3cc5d4 Fix the tests and implement a fake stripe. 2014-04-10 15:20:16 -04:00
Joseph Schorr
bdf2b02c1a Merge branch 'ncc1701' of https://bitbucket.org/yackob03/quay into ncc1701 2014-04-10 00:27:51 -04:00
Joseph Schorr
0e320c964f - Add support for super users 
- Add a super user API
- Add a super user interface
 2014-04-10 00:26:55 -04:00
jakedt
4f3fa34206 Remove test field from the database definition. 2014-04-09 19:13:46 -04:00
jakedt
fc7756a3c2 Add alembic plumbing for database schema migrations. 2014-04-09 19:11:33 -04:00
Joseph Schorr
0e54b0501c Return the reason a username validation failed and add tests to verify we are sending the reason to client 2014-04-07 20:37:02 -04:00
jakedt
265fa5070a Fix support for multiple stack configurations and move most secrets into the quay-config project. 2014-04-07 16:59:22 -04:00
jakedt
e87ffa20cf First attempt at making config loadable through string config overrides in an env variable. 2014-04-03 17:31:46 -04:00
Joseph Schorr
9a79d1562a Change to store the pull robot on the repository build and only add the credentials to the queue item. This prevents the credentials from being exposed to the end user. Also fixes the restart build option 2014-04-01 21:49:06 -04:00
jakedt
ca1970a2f4 Use the real registry endpoint in the login command. 2014-04-01 19:00:11 -04:00
jakedt
3525e383df Merge remote-tracking branch 'origin/master' into pullinprivate 
Conflicts:
	test/data/test.db
 2014-04-01 18:28:48 -04:00
jakedt
126363dce9 Use our patched version of docker-py with build credentials patched in. Fix the hostname in the build credentials block to be variable. 2014-04-01 11:38:17 -04:00
Joseph Schorr
35f69b9f5b Add support for .tar.gz build packs in the build package viewer 2014-04-01 00:23:53 -04:00
jakedt
d67a1cddc2 Merge remote-tracking branch 'origin/master' into pullinprivate 
Conflicts:
	workers/dockerfilebuild.py
 2014-03-31 18:10:34 -04:00
jakedt
00ac3fb639 Merge branch 'master' of ssh://bitbucket.org/yackob03/quay 2014-03-28 17:54:07 -04:00
jakedt
fd941ed3a4 Strip tzinfo data from datetimes to make them compatible with peewee on sqlite. 2014-03-28 17:52:47 -04:00
Joseph Schorr
795de4235d Change "cannot connect to redid" to be a raised exception 2014-03-28 14:20:06 -04:00
Joseph Schorr
2006917e03 Add support for pull credentials on builds and build triggers 2014-03-27 18:33:13 -04:00
jakedt
41cfadac23 Protect the search and repository list endpoints appropriately. Add more differentiating data to some need types. Remove the notification about password change from the user admin page. Select the dependent models for the visible repo list. 2014-03-25 17:26:45 -04:00
jakedt
afb3a67b7b Switch the data to a textfield for authorization codes. 2014-03-25 16:06:34 -04:00
jakedt
5f98bf8dab Merge remote-tracking branch 'origin/master' into swaggerlikeus 
Conflicts:
	endpoints/api.py
 2014-03-25 15:50:03 -04:00
jakedt
669ec9c382 Change the token expiration time to 10 years. 2014-03-25 15:38:16 -04:00
jakedt
0097daebc2 Formatting changes. 2014-03-25 14:32:02 -04:00
Joseph Schorr
16d3ddd8cc Nicely handle the case where we cannot connect to Redis 2014-03-25 13:29:06 -04:00
jakedt
b81e48cb41 Merge branch 'swaggerlikeus' of ssh://bitbucket.org/yackob03/quay into swaggerlikeus 
Conflicts:
	test/data/test.db
 2014-03-25 12:43:09 -04:00
jakedt
cbc40588cb Finally figure out what the data field is supposed to be for and use it to implement and fix 3LO. 2014-03-25 12:42:40 -04:00
Joseph Schorr
c82d1ffe98 Add ability for users to see their authorized applications and revoke the access 2014-03-24 20:57:02 -04:00
Joseph Schorr
e92cf37583 Add cancel button to the oauth authorization page, add the org icon to said page, and fix some other minor bugs 2014-03-24 18:30:22 -04:00
Joseph Schorr
f7c27f250b Add full application management API, UI and test cases 2014-03-20 15:46:13 -04:00
jakedt
3b7b12085d User scope objects everywhere. Switch scope objects to namedtuples. Pass the user when validating whether the user has authorized such scopes in the past. Make sure we calculate the scope string using all user scopes form all previously granted tokens. 2014-03-19 18:09:09 -04:00
jakedt
c93c62600d Merge remote-tracking branch 'origin/master' into swaggerlikeus 
Conflicts:
	data/database.py
	endpoints/api.py
	endpoints/common.py
	templates/base.html
	test/data/test.db
	test/specs.py
 2014-03-19 15:39:44 -04:00
Joseph Schorr
b0dcb5d7e3 Merge branch 'swaggerlikeus' of https://bitbucket.org/yackob03/quay into swaggerlikeus 2014-03-18 16:46:28 -04:00
Joseph Schorr
9ae4506a0d Add OAuth usage information the API logs, have it be displayed in the logs UI and start on the code to display application information when clicked. Note that this does not (yet) do anything with the information returned as we need to wait for the mainline merge of Angular 1.2.9 (which is in master) before I can continue on the display 2014-03-18 16:45:18 -04:00
jakedt
6f39e158d6 Eliminate all of the exceptions when running the tests. 2014-03-18 15:58:37 -04:00
Joseph Schorr
d469b41899 Add an oauth authorization page 2014-03-14 18:57:28 -04:00
Joseph Schorr
525ef8d14f Add support for targeting notifications to organizations and remove the password_required notification for new orbs 2014-03-12 19:00:24 -04:00
jakedt
e74eb3ee87 Add scope ordinality and translations. Process oauth tokens and limit scopes accordingly. 2014-03-12 16:31:37 -04:00
Joseph Schorr
578add3b9e Finish basic notifications system and verify it works for the "password_required" notification. 2014-03-12 16:05:32 -04:00
jakedt
25ceb90fc6 Add some sort of oauth. 2014-03-12 12:37:06 -04:00
Joseph Schorr
368a8da7db - Add data classes for notifications 
- Add basic API for notifications
- Change the password required to be a notification
 2014-03-12 00:49:03 -04:00
jakedt
bb413126ec Fix list_trigger_builds. 2014-03-06 15:04:59 -05:00
jakedt
fdccba9d8e Reverse the order of the builds from the API. 2014-03-05 16:17:01 -05:00
jakedt
fde96c736a Fix a problem with initdb and add a limit to the builds API. 2014-03-05 15:50:32 -05:00
Joseph Schorr
3f806b10c2 Add support for build status tags, which link to the Quay.io repo 2014-02-28 16:23:36 -05:00
jakedt
c13f7cd9df Add the resource_key back to the repository build. 2014-02-25 18:22:02 -05:00
jakedt
de49ce151b Merge remote-tracking branch 'origin/master' into rustedbuilds 2014-02-25 17:00:38 -05:00
jakedt
4b689a7a7c Fix the translation of old ids to new ids, again. 2014-02-25 16:31:52 -05:00
jakedt
b094480164 Add code that will allow build triggers to be deactivated or deleted from their services. 2014-02-24 16:36:49 -05:00
jakedt
13dea98499 Prepare the build worker to support multiple tags and subdirectories. Change the build database config to accept a job config object instead of breaking out the parameters into independent blocks. 2014-02-24 16:11:23 -05:00
jakedt
86e93a2c0f Write triggers are successfully installing on GitHub, noice! 2014-02-21 17:09:56 -05:00
jakedt
d5304f7db0 Merge remote-tracking branch 'origin/master' into rustedbuilds 
Conflicts:
	data/database.py
	endpoints/api.py
	endpoints/common.py
	test/data/test.db
 2014-02-21 14:52:40 -05:00
jakedt
f339160ab9 Fix for mapping translations for existing images. 2014-02-21 08:00:47 -05:00
jakedt
f8eb0c983f Short circuit the ancestry computation for empty ancestry strings. 2014-02-20 22:35:54 -05:00
jakedt
a63a49caa1 Fix up the shared base images stuff. 2014-02-20 22:26:10 -05:00
jakedt
9e426816a5 Pass trigger information on build status. Set up a trigger for the sample building repository. Allow to list the builds started from a trigger. Protect the callback with the proper auth for creating a trigger on a repo. 2014-02-19 16:08:33 -05:00
jakedt
57dc1e045a Try to fix the bad query that makes mysql a sad panda. 2014-02-18 20:41:00 -05:00
jakedt
f60f9eb62a Properly connect the github push webhook with the build worker. Still need to resolve the archive format. 2014-02-18 18:09:14 -05:00
Joseph Schorr
a897fb1dd3 Add tests for image sharing 2014-02-18 16:46:10 -05:00
jakedt
b5d4919364 Split out callbacks into their own blueprint. Add build trigger DB information and connect it with some APIs. Stub out the UI to allow for generation of triggers. Split out the triggers into a plugin-ish architecture for easily adding new triggers. 2014-02-18 15:50:15 -05:00
jakedt
46de02a9ec Merge remote-tracking branch 'origin/master' into allyourbaseimage 2014-02-17 15:18:01 -05:00
jakedt
50d130f199 Fix another image storage test. Reverse a test to make the conditional smaller. 2014-02-17 14:58:45 -05:00
jakedt
7428d2aa50 Fix the garbage collect method to remove unused old images. 2014-02-17 14:38:35 -05:00
jakedt
e7064f1191 Fix the tests and the one bug that it highlighted. 2014-02-16 18:59:24 -05:00
jakedt
b619356907 Get the base image stuff working. Checkpoint before fixing the tests. 2014-02-16 17:38:47 -05:00
Joseph Schorr
55d846061e Add ability to re-run a dockerfile build 2014-02-14 18:37:06 -05:00
jakedt
5742e6ea4e First set of changes toward shared base images. 2014-02-14 16:06:30 -05:00
yackob03
ade20952e2 Merge branch 'master' into tutorial 
Conflicts:
	config.py
	static/js/app.js
	test/data/test.db
 2014-02-13 14:35:20 -05:00
yackob03
8f6cdabde3 Merge branch 'bobthe' 
Conflicts:
	test/data/test.db
 2014-02-13 12:47:59 -05:00
yackob03
b920a0cb1f Really try to emulate the logs format with the test logs. Switch to a simplified flat logs format. 2014-02-12 18:58:40 -05:00
yackob03
85694dd110 Add a build display name, set it from the etag reported by s3 for file uploads. 2014-02-12 13:52:12 -05:00
yackob03
e1a4efe35c Change the build queue identifier to disambiguate between old and new build jobs. 2014-02-12 12:51:30 -05:00
yackob03
7f89a5b7f0 Add the start time to the repository build object and APIs. 2014-02-11 12:25:38 -05:00
yackob03
6fd343741b Change to the new paging format with the commands available at the top. 2014-02-10 19:12:43 -05:00
Joseph Schorr
98e57b9d2b Merge branch 'master' into tutorial 
Conflicts:
	endpoints/index.py
	static/css/quay.css
	static/js/app.js
	static/js/controllers.js
	test/data/test.db
 2014-02-06 21:23:27 -05:00
Joseph Schorr
fa1bf94af1 Get the basic tutorial working completely, including reacting to server-side events 2014-02-06 20:58:26 -05:00
yackob03
b9a5e355eb Improve the error message for when a new org name doesn't pass the regex. 2014-02-06 19:59:10 -05:00
yackob03
c8f0780aaf Fix a subtle bug with create_or_update_tag. 2014-02-06 14:49:53 -05:00
yackob03
2a89c2bb35 Refactor the garbage collection code a tiny bit. 2014-02-06 14:22:13 -05:00
yackob03
ef68982728 Garbage collect repositories on push and on tag deletion. 2014-02-06 14:13:35 -05:00
yackob03
5270066d6d Switch to the redis backed build logs and status. 2014-02-03 19:08:37 -05:00
Joseph Schorr
0833c88065 Make testing much faster by using a save point, rather than recreating the database every test 2014-01-30 20:57:40 -05:00
yackob03
ea6ec52689 Fix OR predicate for mysql on prototype permissions. 2014-01-22 12:13:51 -05:00