vbatts/quay
Archived
1
0
Fork 0
Code Issues Pull requests Releases Wiki Activity
3990 commits 2 branches 62 tags 205 MiB
Commit graph

731 commits

Author SHA1 Message Date
Joseph Schorr
f19d2f684e Add ability to revert tags via time machine 2015-04-16 17:18:00 -04:00
Joseph Schorr
f8c80f7d11 Add a history view to the tags page. Next step will add the ability to revert back in time 2015-04-15 15:21:09 -04:00
Joseph Schorr
3f1e8f3c27 Add a RepositoryActionCount table so we can use it (instead of LogEntry) when scoring repo search results 2015-04-13 13:31:07 -04:00
Joseph Schorr
1df025b57e Change search to use a set of queries for repo lookup rather than a single monolithic query, in the hopes that this will make things significantly faster and actually useable. The individual queries have been tested by hand on MySQL, but the real test will be staging 2015-04-10 15:27:37 -04:00
Joseph Schorr
0be0aed17d Move the repo sorting by pull count into the main matching query, to both make it more accurate and make the search faster 2015-04-09 14:41:59 -04:00
Joseph Schorr
396cba64e6 Fix search to return better results by searching for robots and namespaces in different queries. 2015-04-09 12:57:20 -04:00
Joseph Schorr
762fdaf319 Fix DB migration tests 2015-04-08 17:41:53 -04:00
Joseph Schorr
4f4bb05621 Fix search SQL issues 2015-04-08 17:41:08 -04:00
Joseph Schorr
19e25ac340 Merge branch 'master' into bing 2015-04-08 15:23:36 -04:00
Joseph Schorr
d09f2f6e22 Get the new context-sensitive new menu working 2015-04-07 18:33:43 -04:00
Joseph Schorr
40a6892a49 Add search tests 2015-04-07 14:05:12 -04:00
Joseph Schorr
1b56567268 Make sure also include teams from organizations that the user admins 2015-04-07 13:45:49 -04:00
Joseph Schorr
951b0cbab8 Start on new interactive search 2015-04-06 19:17:18 -04:00
Joseph Schorr
4cb7921c3a Make sure to show public repos on the user and orgs pages 2015-04-03 14:55:09 -04:00
Joseph Schorr
094f91fb8b Fix the tutorial's user events 2015-04-03 12:13:33 -04:00
Joseph Schorr
5cd500257d Merge branch 'master' into orgview 2015-04-01 13:56:49 -04:00
Joseph Schorr
fde9666647 Add the team membership to the robots view 2015-04-01 13:56:30 -04:00
Joseph Schorr
1f5e6df678 - Fix tests 
- Add new endpoints for retrieving the repo permissions for a robot account
- Have the robots list return the number of repositories for which there are permissions
- Other UI fixes
 2015-03-31 18:50:43 -04:00
Joseph Schorr
27a9b84587 Switch avatars to be built out of CSS and only overlayed with the gravatar when a non-default exists 2015-03-30 17:55:04 -04:00
Jimmy Zelinskie
da15eda2bf migration: make auth_token for builds nullable 2015-03-27 11:20:30 -04:00
Joseph Schorr
7b1d35737e Merge branch 'master' of github.com:coreos-inc/quay 2015-03-26 17:45:49 -04:00
Joseph Schorr
a7b6cb5c23 Fix handling of byte strings and large ints 2015-03-26 17:45:43 -04:00
Jimmy Zelinskie
4d4947edaf Merge branch 'master' of github.com:coreos-inc/quay 2015-03-26 16:31:20 -04:00
Jimmy Zelinskie
bcc7a9580b models: change UUID of user on password change 
This prevents old cookies from continuing to work after a password has
been changed.
 2015-03-26 16:30:41 -04:00
Joseph Schorr
c4a2574b0d Clarify unencrypted password error message 2015-03-26 16:23:28 -04:00
Joseph Schorr
f8afd8b5ce Make sure to parse the big int into a byte string 2015-03-26 16:13:35 -04:00
Joseph Schorr
4d1792db1c getrandbits creates an int, not a float 2015-03-26 15:47:44 -04:00
Joseph Schorr
aaf1b23e98 Address CL concerns and switch to a real encryption system 2015-03-26 15:10:58 -04:00
Joseph Schorr
d23bb6616d Fix error message to exactly match current output 2015-03-26 13:22:16 -04:00
Joseph Schorr
e4b659f107 Add support for encrypted client tokens via basic auth (for the docker CLI) and a feature flag to disable normal passwords 2015-03-25 18:43:12 -04:00
Jake Moshenko
3d44416016 Add the migration for the unique index which helps prevent tag deadlocks. 2015-03-24 18:00:04 -04:00
Jake Moshenko
201943ed1c Fix deadlocks with tags and garbage collection. 2015-03-24 18:00:04 -04:00
Jimmy Zelinskie
9c55aca011 migration: make resource_key nullable 2015-03-23 15:46:35 -04:00
Jimmy Zelinskie
93a9e9d01a migration: add private key to build triggers 2015-03-19 14:30:25 -04:00
Joseph Schorr
ab2331a486 Performance improvements for the repo API and the new repo UI 2015-03-18 14:47:53 -04:00
Joseph Schorr
fe8d006855 Fix subquery on MySQL 2015-03-17 13:54:51 -04:00
Joseph Schorr
44ff85d044 Remove migration (temporarily), fix a broken test, and make the aggregate size calculation use the entire image ancestry (for now). 2015-03-17 12:13:01 -04:00
Joseph Schorr
b8d88c0f4e Add aggregate size column and a migration to backfill it 2015-03-16 18:03:17 -04:00
Joseph Schorr
333e0acd6d Add the builds tab 2015-03-13 15:34:28 -07:00
Joseph Schorr
002dc083f2 Get the main repo page design working 2015-03-10 17:22:46 -07:00
Joseph Schorr
afc8e95e19 Start on new tag view 2015-03-09 22:03:39 -07:00
Joseph Schorr
929bad57f5 Fix revision 2015-03-05 15:06:46 -05:00
Joseph Schorr
86447c0a99 Merge branch 'master' into pagesnew 2015-03-05 14:22:10 -05:00
Joseph Schorr
edafa26e6e Fix email confirmation for users created through the superuser API 2015-03-03 14:26:48 -05:00
Joseph Schorr
a36266f758 Add LDAP tracing 2015-02-27 17:01:46 -05:00
Jake Moshenko
6c4e78ec0c Fix the deadlock in tags. 2015-02-25 17:49:46 -05:00
Jimmy Zelinskie
fb0d3d69c2 changes to reflect PR comments (not finished) 2015-02-24 17:50:54 -05:00
Jake Moshenko
d8a34427ec Try to get around the tag deadlock by using a select for update. 2015-02-24 17:18:42 -05:00
Jake Moshenko
246ff556b9 Fix some other list reifications to use nested subqueries for performance and query size safety reasons. 2015-02-23 15:07:38 -05:00
Joseph Schorr
4020cc1102 Reset the number of invalid login attempts when the user change's their password 2015-02-23 13:56:42 -05:00
Joseph Schorr
a0706d20ae Merge branch 'master' of github.com:coreos-inc/quay 2015-02-23 13:38:04 -05:00
Joseph Schorr
5f605b7cc8 Fix queue handling to remove the dependency from repobuild, and have a cancel method 2015-02-23 13:38:01 -05:00
Jake Moshenko
75d41ca371 Switch to an inner inner subquery to make mysql arbitrarily happy. 2015-02-23 12:36:14 -05:00
Jimmy Zelinskie
7fbbf14e49 migrations: resolve branched migration tree 
This moves the migration adding stars to HEAD.
 2015-02-19 17:10:27 -05:00
Jimmy Zelinskie
917dd6b674 Merge branch 'master' into star 2015-02-18 17:36:58 -05:00
Jake Moshenko
41108a0856 Allow tags to be marked as hidden. Create a hidden tag on every image during a push to prevent them from getting GCed. 2015-02-18 17:05:16 -05:00
Jake Moshenko
59b794dd61 Move the creation of images to when the JSON is uploaded. 2015-02-18 17:04:25 -05:00
Joseph Schorr
89eb5bdcc5 Merge branch 'quark' 2015-02-18 15:59:36 -05:00
Joseph Schorr
a5ff765f3b Validate that we have a valid JSON body 2015-02-18 15:57:05 -05:00
Jake Moshenko
2dd03f1bed Merge remote-tracking branch 'origin/master' into rockyhorror 
Conflicts:
	test/data/test.db
 2015-02-18 10:56:01 -05:00
Jake Moshenko
2cd5bdb563 Address concerns from pull request. 2015-02-18 10:43:08 -05:00
Joseph Schorr
ff7e97faf7 Merge branch 'master' into quark 2015-02-17 19:16:12 -05:00
Joseph Schorr
524705b88c Get dashboard working and upgrade bootstrap. Note: the bootstrap fixes will be coming in the followup CL 2015-02-17 19:15:54 -05:00
Joseph Schorr
83e05d2342 Add tracking of the kind of temporary access tokens, so we can display if a pull/push by token is for a build worker 2015-02-17 12:35:16 -05:00
Joseph Schorr
3e04e3cfd7 Merge branch 'master' of github.com:coreos-inc/quay 2015-02-13 15:54:09 -05:00
Joseph Schorr
81ce4c771e Add ability to cancel builds that are in the waiting state 2015-02-13 15:54:01 -05:00
Jake Moshenko
579d900ef2 Add server defaults to the new fields which are not nullable. 2015-02-13 14:45:42 -05:00
Jake Moshenko
2ce6e76d9d Add the required migration for time machine tag lifetimes. 2015-02-13 14:41:08 -05:00
Jake Moshenko
5aedd1fabc Merge remote-tracking branch 'origin/master' into rockyhorror 
Conflicts:
	test/data/test.db
 2015-02-13 13:17:10 -05:00
Jake Moshenko
2743fe6f4d Trap migration exceptions for the migration that we expect to fail in prod. 2015-02-13 13:11:50 -05:00
Jake Moshenko
cd245e62bf Remove the access_token_id column from log entries. 2015-02-13 11:56:59 -05:00
Jake Moshenko
4a921a49b9 Merge remote-tracking branch 'origin/master' into rockyhorror 2015-02-12 16:24:30 -05:00
Joseph Schorr
f84d1bad45 Handle internal errors in a better fashion: If a build would be marked as internal error, only do so if there are retries remaining. Otherwise, we mark it as failed (since it won't be rebuilt anyway) 2015-02-12 16:19:44 -05:00
Jake Moshenko
b154e7acef Merge remote-tracking branch 'origin/master' into rockyhorror 2015-02-12 15:27:28 -05:00
Jake Moshenko
990739b1e5 Add the APIs required to change the time machine policy for users and organizations. 2015-02-12 14:37:11 -05:00
Jake Moshenko
872539bdbf Switch to a per-namespace configurable expiration policy for time machine, and switch the tag gc to respect it. 2015-02-12 14:11:56 -05:00
Joseph Schorr
f107b50a46 Merge branch 'master' into ackbar 2015-02-12 12:04:45 -05:00
Jake Moshenko
f32bd748e4 Use a consistent concept of tag liveness everywhere. Fix the tests. 2015-02-11 15:02:50 -05:00
Jake Moshenko
90c0a9c1e0 First stab at time machine using fixed two week expiration policy. 2015-02-11 14:15:40 -05:00
Joseph Schorr
045614c6c8 Merge branch 'master' into ackbar 2015-02-09 17:16:42 -05:00
Joseph Schorr
3d31c64da2 Make sure to select the latest image in the repository with the matching comment 2015-02-09 16:45:06 -05:00
Joseph Schorr
48949627e0 Merge master in delta 2015-02-09 12:07:43 -05:00
Joseph Schorr
e1c5ccb7d6 Fixes 2015-02-05 17:37:58 -05:00
Joseph Schorr
cf774e23df Merge branch 'master' into v2 2015-02-05 15:37:14 -05:00
Joseph Schorr
a12bfa7623 Add migration for the new tables for signatures 2015-02-05 15:30:45 -05:00
Joseph Schorr
9dfe523615 Merge master changes 2015-02-05 13:11:16 -05:00
Joseph Schorr
bfb0784abc Add signing to the ACI converter 2015-02-04 15:29:24 -05:00
Jake Moshenko
ce7033489b Hopefully fix the deadlock in the queue. 2015-02-03 14:50:01 -05:00
Jake Moshenko
64750e31fc Add the ability to select for update within transactions to fix some write after read hazards. Fix a bug in extend_processing. 2015-01-30 16:32:13 -05:00
Joseph Schorr
3872d29de9 Add a transaction around the extend_processing call 2015-01-29 18:40:41 -05:00
Joseph Schorr
30b895b795 Merge branch 'grunt-js-folder' of https://github.com/coreos-inc/quay into ackbar 2015-01-23 17:26:14 -05:00
Joseph Schorr
c8229b9c8a Implement new step-by-step setup 2015-01-23 17:19:15 -05:00
Jake Moshenko
44f7ab53a2 Merge remote-tracking branch 'origin/master' into ephemeral 2015-01-21 13:39:27 -05:00
Joseph Schorr
92d32bc636 Make the DB health check first attempt a simple DB connection. If the database is in the middle of a failover, this will fail after 3 seconds (the connection timeout specified), rather than hanging and causing the ELB health checks to timeout and fail. 2015-01-20 14:46:22 -05:00
Joseph Schorr
15a69ac872 Change robot deletions to set the performer to null, rather than attempting to delete the rows from the large log entries table 2015-01-14 12:56:06 -05:00
Joseph Schorr
bfd273d16f - Make validation a bit nicer: 
- Add timeout to the DB validation
  - Make DB validation exception handling a bit nicer
  - Move the DB validation error message

- Fix bug around RADOS config default for Is Secure
- Allow hiding of the validation box
 2015-01-08 15:27:49 -05:00
Joseph Schorr
47fb10b79f Merge branch 'master' into ackbar 2015-01-08 13:57:39 -05:00
Jimmy Zelinskie
b18986c72b Continuation of baa4918d89. 2015-01-06 15:11:37 -05:00
Jimmy Zelinskie
baa4918d89 Explicitly declare length of large MySQL indices. 
Most MySQL installs silently truncate indices at a specific number of bytes.
This value is 767 bytes unless either the innodb_large_prefix option is turned
on or the innodb_page_size has been changed. This change explicitly limits the
size of large indices to 767 characters due to the latin1 charset being 1 byte
per char.
 2015-01-06 14:55:31 -05:00
Joseph Schorr
1bf25f25c1 WIP 2015-01-04 14:38:41 -05:00
Jake Moshenko
cc70225043 Generalize the ephemeral build managers so that any manager may manage a builder spawned by any other manager. 2014-12-31 11:33:56 -05:00
Jimmy Zelinskie
e8cd24781a remove 'include_starred' option on repo listings 2014-12-29 14:11:46 -05:00
Jimmy Zelinskie
84f6cc7385 add percona to migration tests 2014-12-23 18:34:02 -05:00
Jimmy Zelinskie
977782d047 Add MariaDB to migration.sh 2014-12-22 15:34:17 -05:00
Jimmy Zelinskie
eeeb2e620c move slackwebhook migration from tools to util 
tools isn't shipped inside of the container because it contains private
keys
 2014-12-18 13:22:13 -05:00
Joseph Schorr
107847a4bb Fix Slack notification setup to support the new slack web hook format and convert all existing data to use the new format (so we only have one code path) 2014-12-16 14:34:43 +02:00
Jimmy Zelinskie
5a484cfe11 Initial redesigned UI for repo listings w/ stars. 2014-12-11 15:07:41 -05:00
Joseph Schorr
6601e83285 When speaking to version 0.2-beta of the build worker, properly lookup the cached commands and see if we have a matching image/tag in the repository 2014-12-11 18:03:40 +02:00
Joseph Schorr
dbac8c7e3d Fix build code: 
- Fix issue with the queue_item in extend processing
  - Add the new compiled docker binary with the lxc volume fix
 2014-12-04 17:49:39 +01:00
Jimmy Zelinskie
97b605ca8d Fix starring after rebase. 2014-12-03 12:20:51 -08:00
Jimmy Zelinskie
bd9f529e38 Add missing Star import. 2014-12-02 17:31:55 -08:00
Jimmy Zelinskie
08d84298e2 Fix indentation in star migration. 2014-12-02 17:31:21 -08:00
Jimmy Zelinskie
eb956e5b7d initial work on adding models for starring repos. 
I'm sick of using `git stash`.
 2014-12-02 17:31:21 -08:00
Jimmy Zelinskie
f3259c862b Merge branch 'koh' 
Conflicts:
	auth/scopes.py
	requirements-nover.txt
	requirements.txt
	static/css/quay.css
	static/directives/namespace-selector.html
	static/js/app.js
	static/partials/manage-application.html
	templates/oauthorize.html
 2014-12-01 12:30:09 -08:00
Joseph Schorr
72d613614d Merge branch 'bagger' 2014-12-01 12:48:59 -05:00
Joseph Schorr
7bf96c506f Merge branch 'bees' into koh 2014-11-24 19:25:53 -05:00
Joseph Schorr
e9cac407df Add a configurable avatar system and add an internal avatar system for enterprise 2014-11-24 19:25:13 -05:00
Jimmy Zelinskie
716d7a737b Strip whitespace from ALL the things. 2014-11-24 16:07:38 -05:00
Joseph Schorr
b8e873b00b Add support to the build system for tracking if/when the build manager crashes and make sure builds are restarted within a few minutes 2014-11-21 14:27:06 -05:00
Jimmy Zelinskie
c918d15979 Add a migration for backfilling user UUIDs. 2014-11-20 18:28:54 -05:00
Jake Moshenko
2b8c246476 Temporarily put user rename behind a feature flag. Switch queue names back to using the username for namespace while we figure out a real migration strategy. 2014-11-20 15:36:39 -05:00
Jimmy Zelinskie
e6742b00b0 Remove backfill from UUID migration 2014-11-20 14:04:12 -05:00
Joseph Schorr
8fab3b6d34 Make sure to switch the context back to LogEntry and make sure to only include the performer if actually present 2014-11-20 12:04:53 -05:00
Joseph Schorr
cc7bccea4c Add the logentrykind and performer information to log listing. This makes the system much faster, since new queries are not needed for every log entry. 2014-11-20 12:03:16 -05:00
Joseph Schorr
9b31b9805a Fix performance problem with looking up org members and add some tests 2014-11-20 11:33:42 -05:00
Jake Moshenko
ac0d0ce36a Change the order of migrations to allow migration code to work with the updated model. 2014-11-19 15:58:47 -05:00
Jake Moshenko
e863b96166 Tweak the uuid backfill to leave the uuid column nullable. 2014-11-19 15:32:30 -05:00
Jimmy Zelinskie
7c8a438b58 Rebase migration on top of master and add backfill. 2014-11-19 14:07:54 -05:00
Jimmy Zelinskie
606ad21bec Apply reviewed changes. 
Adds a length to the UUID field, renames QuayDeferredPermissionUser
parameter id->uuid, adds transactions to backfill script.
 2014-11-19 13:28:16 -05:00
Jimmy Zelinskie
9d677b8eb3 Add UUID to User model and use in cookie. 2014-11-19 13:28:16 -05:00
Jake Moshenko
17fc72d262 Switch postgres to a non-transactional DDL to allow us to use peewee to modify data in migrations: enterprise customers are running postgres migrations offline already. Move the image backfill script back to a migration since it will now work. Unify the interface to sending a DB URI to env.py for the migration script. 2014-11-18 14:07:33 -05:00
Jake Moshenko
a7bae6c1d9 Fix robot renaming. Allow for trigger URLs to contain or omit the repository in the path. Fix calls to get_trigger to remove the namespace and repository. 2014-11-18 10:24:48 -05:00
Jake Moshenko
f4681f2c18 Merge branch 'master' into nomenclature 
Conflicts:
	test/data/test.db
 2014-11-17 17:59:59 -05:00
Joseph Schorr
58ca76239b Add ability to one-click generate an authorization access token in the applications panel 2014-11-17 14:54:07 -05:00
Joseph Schorr
178c5a7ac0 Add an index to the docker_image_id for faster lookup 2014-11-13 12:51:50 -05:00
Joseph Schorr
d73747ce1d - Fix some other group_by clauses 
- Fix garbage_collect for an empty list (fixes a test)
- Add a script which runs the full test suite against mysql and postgres (note: QueueTest's are broken for MySQL, but they obviously work in production, so they need to be fixed)
 2014-11-13 12:51:37 -05:00
Joseph Schorr
a2e2dcb010 Fix group_by clauses for postgres 2014-11-12 20:32:06 -05:00
Joseph Schorr
079289c53c HACK: Don't join on the visibility table because it is horrendously slow for some odd reason. 2014-11-12 14:51:24 -05:00
Joseph Schorr
eddcc02ea6 Make repository deletes much faster by adding custom deletion code and have additional tests to verify the deletion code paths 2014-11-10 23:05:20 -05:00
Joseph Schorr
e7cbda86f7 Merge branch 'perf' 2014-11-10 21:52:27 -05:00
Joseph Schorr
091f821a6a - Rename get_repo_image to get_repo_image_extended and get_repo_image_directly to get_repo_image 
- Remove the configure call from CloseForLongOperation
- Other small fixes
 2014-11-10 13:44:36 -05:00
Joseph Schorr
72fedef097 Move the robot deletion code into a delete_instance method on the User object 2014-11-10 13:18:17 -05:00
Jake Moshenko
ca435fc7a6 Rename robots when we rename a user. Do not use the namespace from the path to check permissions from the incoming webhooks since the namespace may have changed and we cannot recreate them in remote services easily. 2014-11-09 17:50:57 -05:00
Joseph Schorr
17f605a9ef Select only a single token. 2014-11-09 15:50:50 -05:00
Joseph Schorr
691be49817 Fix issues with the perf updated code 2014-11-07 14:36:32 -05:00
Joseph Schorr
158acd4f41 - Turn on foreign key constraint checking in the tests 
- Change all ForeignKeyField's that refer to users to use our custom class, and mark those that allow robots
- Change robot delete to only execute the subset of queries necessary to actually delete robots
 2014-11-07 12:05:21 -05:00
Joseph Schorr
d5bbb57481 Change registry code to disconnect from the DB before long I/O operations 2014-11-06 18:00:52 -05:00
Joseph Schorr
23d9bd2b42 Change verbs to use short lived database connections 2014-11-06 17:50:48 -05:00
Joseph Schorr
c569299e5c Database optimizations around image creation and logs lookup 2014-11-06 14:48:16 -05:00
Joseph Schorr
a35bc11912 Add perf comments 2014-11-05 12:27:38 -05:00
Joseph Schorr
29c30b336e Move the uncompressed image size migration call outside of alembic, since it will sometimes deadlock with certain kinds of DBs (because alembic is running things inside a transaction) 2014-11-03 21:01:41 -05:00
Joseph Schorr
9aa72c5cc2 Fix migration issues: 
- MySQL 5.5 doesn't support the now() call as a default
  - Postgres migration isn't auto-committed, so we have to check if the table exists first
 2014-11-03 15:25:55 -05:00
Joseph Schorr
4eedd54b66 - Make usage language more accurate by stating "repositories" 
- Have usage counter be based on a 4 weeks TTL
- Add a simple usage counter breakage test
 2014-10-30 13:26:02 -04:00
Joseph Schorr
c1398c6d2b - Add a log entry for repo verb handling and make the container usage calculation take it into account 
- Move all the repo push/pull/verb logging into a central track_and_log method
- Readd images accidentally deleted in the last CL
- Make the uncompressed size migration script better handle exceptions
 2014-10-29 15:42:44 -04:00
Joseph Schorr
e9c2e54dbc Change delete to once again not perform everything under a transaction. A recent revision accidentally moved everything back under it. 2014-10-29 11:59:02 -04:00
Joseph Schorr
109850b428 Add a basic usage counter for enterprise 2014-10-28 16:33:13 -04:00
Joseph Schorr
c06f57a6e7 Make sure builders close the db handle when no work comes in and make the metrics transaction smaller in scope 2014-10-24 11:40:02 -04:00
Jake Moshenko
1461310ab8 Merge remote-tracking branch 'origin/master' into nomenclature 
Conflicts:
	endpoints/common.py
	endpoints/notificationhelper.py
	test/data/test.db
	workers/dockerfilebuild.py
 2014-10-23 13:25:37 -04:00
Joseph Schorr
5db9cd948b Add better (jinja-based) messaging to the notifications and add some fixes for the email templates 2014-10-22 19:01:56 -04:00
Joseph Schorr
28a463f998 Make the 'created' field optional, in case the user does not specify it via the API 2014-10-20 12:07:49 -04:00
Jake Moshenko
dc5ee43a3a MySQL still doesn't allow us to have empty IN clauses. 2014-10-17 17:49:18 -04:00
Jake Moshenko
6ca0115b5e MySQL doesnt allow us to use the table we are modifying in a subquery. 2014-10-17 17:01:55 -04:00
Jake Moshenko
efe8825a15 We can't even use an empty in query for images. 2014-10-17 16:37:05 -04:00
Jake Moshenko
acd5a1e07e Only garbage collect storage when it might result in something. Return the count of the removed images to fulfill the contract expected by some callers. 2014-10-17 16:13:32 -04:00
Jake Moshenko
18a944427f Merge branch 'master' of bitbucket.org:yackob03/quay 2014-10-17 15:26:58 -04:00
Jake Moshenko
c093e5a326 Add a whitelist of candidate storages which will speed up the orphan queries and limit the damage of GC run amok. 2014-10-17 15:26:51 -04:00
Joseph Schorr
26a9bb50d5 Fix spelling mistake 2014-10-17 14:44:17 -04:00
Jake Moshenko
d8149295ab Update the GC code to do everything with subqueries, making each GC run a bounded finite number of queries with a fixed length. 2014-10-17 14:33:04 -04:00
Jake Moshenko
380eb49e58 Split out the GC code into smaller transactions. Remove from the actual storage component after the fact, outside of the transactions. 2014-10-17 11:42:09 -04:00
Jake Moshenko
fa6a06502d Change the default redis host to localhost. Fix some whitespace issues in the userevents module. 2014-10-14 14:37:02 -04:00
Jake Moshenko
328db8b660 Split the app into separate backends, which can use different worker types and different timeouts. 2014-10-14 13:58:08 -04:00
Joseph Schorr
bb9502ee77 Make sure to raise the exception for WSGI and make the delete method's intent more clear 2014-10-08 13:51:50 -04:00
Joseph Schorr
d16fdde528 Fix bug in dockerloadformat and make sure we handle exceptions properly in the verb call 2014-10-08 13:43:12 -04:00
Jake Moshenko
c34a8b6727 Fix the squashed image migration to insert the proper image transformation type. 2014-10-08 11:47:02 -04:00
Jake Moshenko
1a8287af55 Missed a table import. 2014-10-07 17:33:20 -04:00
Jake Moshenko
bbcdf18fdd Update the migration to reflect the state of the database. 2014-10-07 16:19:05 -04:00
Jake Moshenko
1968c67cba Merge remote-tracking branch 'origin/laffa' into nomenclature 2014-10-07 16:09:45 -04:00
Jake Moshenko
626d984fb2 Fixes to the migration generator script. 2014-10-07 16:09:30 -04:00
Jake Moshenko
fe59ad4fb5 Rectify our prod database, previous migrations, and database definition files. 2014-10-07 16:03:57 -04:00
Jake Moshenko
c77a3dc6ea Update the migration for translating queue names to take into account previous migrations. 2014-10-07 15:57:17 -04:00
Jake Moshenko
a0d94f9d59 Merge remote-tracking branch 'origin/laffa' into nomenclature 
Conflicts:
	test/data/test.db
 2014-10-07 15:56:28 -04:00
Jake Moshenko
ee9973a395 Merge remote-tracking branch 'origin/master' into nomenclature 2014-10-07 15:43:59 -04:00
Joseph Schorr
f4daa5e97b - Update the migrations tool to verify migrations work up and down for both MySQL and PostgresSQL. 
- Add migrations for the squashed image tables and for backfilling the uncompressed sizes
- Make sure gzip stream uses a max length when determining the uncompressed size
 2014-10-07 15:29:56 -04:00
Joseph Schorr
f38ce51943 Merge master into laffa 2014-10-07 14:03:17 -04:00
Jake Moshenko
4ad592e7ce Add an index to the image storage uuid to improve performance. 2014-10-06 18:44:37 -04:00
Jake Moshenko
c4266140e2 Fix all of the upgrades and downgrades to work on both mysql and postgres. 2014-10-06 16:41:49 -04:00
Jake Moshenko
ed8bcff39e Merge remote-tracking branch 'origin/master' into nomenclature 
Conflicts:
	test/data/test.db
	workers/dockerfilebuild.py
 2014-10-06 10:29:39 -04:00
Jake Moshenko
5c18ffe67d Allow the namespace column to be null, and also non-unique. Fix the uncompressed size clobbering the size on the wire field. Add metadata constraints so that foreign key constraints get predictable names. Fix all downgrade migrations. 2014-10-02 10:46:20 -04:00
Jake Moshenko
e8b3d1cc4a Phase 4 of the namespace to user migration: actually remove the column from the db and remove the dependence on serialized namespaces in the workers and queues 2014-10-01 14:23:46 -04:00
Joseph Schorr
2c5cc7990f Allow for additional REDIS config such as password and port 2014-10-01 14:16:42 -04:00
Joseph Schorr
6c520b8b0b Fixed broken query 2014-09-30 13:19:32 -04:00
Joseph Schorr
5f776771d1 Merge branch 'master' of https://bitbucket.org/yackob03/quay 2014-09-29 17:01:07 -04:00
Joseph Schorr
474add0fb1 Have the layer PUT method calculate the uncompressed size in realtime, as trusting the JSON is fraught with complications 2014-09-29 17:00:47 -04:00
Jake Moshenko
fbc7ee3bce Phase 3 of the namespace migration, remove write references to the namespace string column. 2014-09-29 15:44:25 -04:00
Jake Moshenko
ae9026ee91 MySQL will not allow us to revoke the possibility for the namespace_user column to be nullable. 2014-09-26 14:50:26 -04:00
Jake Moshenko
21d8c41320 Add the existing types to the alter column to make MySQL happy. 2014-09-26 12:58:03 -04:00
Jake Moshenko
e9745dbc96 Fix the SQL migration. 2014-09-26 11:17:42 -04:00
Jake Moshenko
03190efde3 Phase 2 of migrating repo namespaces to referencing user objects, backfilling the rows without a value for namespace_user, and changing all accesses to go through the namespace_user object. All tests are passing, manual testing still required. 2014-09-24 18:01:35 -04:00
Jake Moshenko
98d8e0fe37 Fixes for the uncompressed size backfill script. 2014-09-23 16:06:59 -04:00
Joseph Schorr
ec484e3efc Move size and checksum updates into the metadata call 2014-09-23 15:49:28 -04:00