Joseph Schorr
d73747ce1d
- Fix some other group_by clauses
...
- Fix garbage_collect for an empty list (fixes a test)
- Add a script which runs the full test suite against mysql and postgres (note: QueueTest's are broken for MySQL, but they obviously work in production, so they need to be fixed)
2014-11-13 12:51:37 -05:00
Joseph Schorr
a2e2dcb010
Fix group_by clauses for postgres
2014-11-12 20:32:06 -05:00
Joseph Schorr
079289c53c
HACK: Don't join on the visibility table because it is horrendously slow for some odd reason.
2014-11-12 14:51:24 -05:00
Joseph Schorr
eddcc02ea6
Make repository deletes much faster by adding custom deletion code and have additional tests to verify the deletion code paths
2014-11-10 23:05:20 -05:00
Joseph Schorr
e7cbda86f7
Merge branch 'perf'
2014-11-10 21:52:27 -05:00
Joseph Schorr
091f821a6a
- Rename get_repo_image to get_repo_image_extended and get_repo_image_directly to get_repo_image
...
- Remove the configure call from CloseForLongOperation
- Other small fixes
2014-11-10 13:44:36 -05:00
Joseph Schorr
72fedef097
Move the robot deletion code into a delete_instance method on the User object
2014-11-10 13:18:17 -05:00
Joseph Schorr
17f605a9ef
Select only a single token.
2014-11-09 15:50:50 -05:00
Joseph Schorr
691be49817
Fix issues with the perf updated code
2014-11-07 14:36:32 -05:00
Joseph Schorr
158acd4f41
- Turn on foreign key constraint checking in the tests
...
- Change all ForeignKeyField's that refer to users to use our custom class, and mark those that allow robots
- Change robot delete to only execute the subset of queries necessary to actually delete robots
2014-11-07 12:05:21 -05:00
Joseph Schorr
d5bbb57481
Change registry code to disconnect from the DB before long I/O operations
2014-11-06 18:00:52 -05:00
Joseph Schorr
23d9bd2b42
Change verbs to use short lived database connections
2014-11-06 17:50:48 -05:00
Joseph Schorr
c569299e5c
Database optimizations around image creation and logs lookup
2014-11-06 14:48:16 -05:00
Joseph Schorr
a35bc11912
Add perf comments
2014-11-05 12:27:38 -05:00
Joseph Schorr
29c30b336e
Move the uncompressed image size migration call outside of alembic, since it will sometimes deadlock with certain kinds of DBs (because alembic is running things inside a transaction)
2014-11-03 21:01:41 -05:00
Joseph Schorr
9aa72c5cc2
Fix migration issues:
...
- MySQL 5.5 doesn't support the now() call as a default
- Postgres migration isn't auto-committed, so we have to check if the table exists first
2014-11-03 15:25:55 -05:00
Joseph Schorr
4eedd54b66
- Make usage language more accurate by stating "repositories"
...
- Have usage counter be based on a 4 weeks TTL
- Add a simple usage counter breakage test
2014-10-30 13:26:02 -04:00
Joseph Schorr
c1398c6d2b
- Add a log entry for repo verb handling and make the container usage calculation take it into account
...
- Move all the repo push/pull/verb logging into a central track_and_log method
- Readd images accidentally deleted in the last CL
- Make the uncompressed size migration script better handle exceptions
2014-10-29 15:42:44 -04:00
Joseph Schorr
e9c2e54dbc
Change delete to once again not perform everything under a transaction. A recent revision accidentally moved everything back under it.
2014-10-29 11:59:02 -04:00
Joseph Schorr
109850b428
Add a basic usage counter for enterprise
2014-10-28 16:33:13 -04:00
Joseph Schorr
c06f57a6e7
Make sure builders close the db handle when no work comes in and make the metrics transaction smaller in scope
2014-10-24 11:40:02 -04:00
Joseph Schorr
5db9cd948b
Add better (jinja-based) messaging to the notifications and add some fixes for the email templates
2014-10-22 19:01:56 -04:00
Joseph Schorr
28a463f998
Make the 'created' field optional, in case the user does not specify it via the API
2014-10-20 12:07:49 -04:00
Jake Moshenko
dc5ee43a3a
MySQL still doesn't allow us to have empty IN clauses.
2014-10-17 17:49:18 -04:00
Jake Moshenko
6ca0115b5e
MySQL doesnt allow us to use the table we are modifying in a subquery.
2014-10-17 17:01:55 -04:00
Jake Moshenko
efe8825a15
We can't even use an empty in query for images.
2014-10-17 16:37:05 -04:00
Jake Moshenko
acd5a1e07e
Only garbage collect storage when it might result in something. Return the count of the removed images to fulfill the contract expected by some callers.
2014-10-17 16:13:32 -04:00
Jake Moshenko
18a944427f
Merge branch 'master' of bitbucket.org:yackob03/quay
2014-10-17 15:26:58 -04:00
Jake Moshenko
c093e5a326
Add a whitelist of candidate storages which will speed up the orphan queries and limit the damage of GC run amok.
2014-10-17 15:26:51 -04:00
Joseph Schorr
26a9bb50d5
Fix spelling mistake
2014-10-17 14:44:17 -04:00
Jake Moshenko
d8149295ab
Update the GC code to do everything with subqueries, making each GC run a bounded finite number of queries with a fixed length.
2014-10-17 14:33:04 -04:00
Jake Moshenko
380eb49e58
Split out the GC code into smaller transactions. Remove from the actual storage component after the fact, outside of the transactions.
2014-10-17 11:42:09 -04:00
Jake Moshenko
fa6a06502d
Change the default redis host to localhost. Fix some whitespace issues in the userevents module.
2014-10-14 14:37:02 -04:00
Jake Moshenko
328db8b660
Split the app into separate backends, which can use different worker types and different timeouts.
2014-10-14 13:58:08 -04:00
Joseph Schorr
bb9502ee77
Make sure to raise the exception for WSGI and make the delete method's intent more clear
2014-10-08 13:51:50 -04:00
Joseph Schorr
d16fdde528
Fix bug in dockerloadformat and make sure we handle exceptions properly in the verb call
2014-10-08 13:43:12 -04:00
Jake Moshenko
c34a8b6727
Fix the squashed image migration to insert the proper image transformation type.
2014-10-08 11:47:02 -04:00
Jake Moshenko
1a8287af55
Missed a table import.
2014-10-07 17:33:20 -04:00
Jake Moshenko
626d984fb2
Fixes to the migration generator script.
2014-10-07 16:09:30 -04:00
Jake Moshenko
fe59ad4fb5
Rectify our prod database, previous migrations, and database definition files.
2014-10-07 16:03:57 -04:00
Joseph Schorr
f4daa5e97b
- Update the migrations tool to verify migrations work up and down for both MySQL and PostgresSQL.
...
- Add migrations for the squashed image tables and for backfilling the uncompressed sizes
- Make sure gzip stream uses a max length when determining the uncompressed size
2014-10-07 15:29:56 -04:00
Joseph Schorr
f38ce51943
Merge master into laffa
2014-10-07 14:03:17 -04:00
Jake Moshenko
4ad592e7ce
Add an index to the image storage uuid to improve performance.
2014-10-06 18:44:37 -04:00
Jake Moshenko
c4266140e2
Fix all of the upgrades and downgrades to work on both mysql and postgres.
2014-10-06 16:41:49 -04:00
Jake Moshenko
5c18ffe67d
Allow the namespace column to be null, and also non-unique. Fix the uncompressed size clobbering the size on the wire field. Add metadata constraints so that foreign key constraints get predictable names. Fix all downgrade migrations.
2014-10-02 10:46:20 -04:00
Joseph Schorr
2c5cc7990f
Allow for additional REDIS config such as password and port
2014-10-01 14:16:42 -04:00
Joseph Schorr
6c520b8b0b
Fixed broken query
2014-09-30 13:19:32 -04:00
Joseph Schorr
5f776771d1
Merge branch 'master' of https://bitbucket.org/yackob03/quay
2014-09-29 17:01:07 -04:00
Joseph Schorr
474add0fb1
Have the layer PUT method calculate the uncompressed size in realtime, as trusting the JSON is fraught with complications
2014-09-29 17:00:47 -04:00
Jake Moshenko
fbc7ee3bce
Phase 3 of the namespace migration, remove write references to the namespace string column.
2014-09-29 15:44:25 -04:00
