Joseph Schorr
e7cbda86f7
Merge branch 'perf'
2014-11-10 21:52:27 -05:00
Joseph Schorr
091f821a6a
- Rename get_repo_image to get_repo_image_extended and get_repo_image_directly to get_repo_image
...
- Remove the configure call from CloseForLongOperation
- Other small fixes
2014-11-10 13:44:36 -05:00
Joseph Schorr
72fedef097
Move the robot deletion code into a delete_instance method on the User object
2014-11-10 13:18:17 -05:00
Jake Moshenko
ca435fc7a6
Rename robots when we rename a user. Do not use the namespace from the path to check permissions from the incoming webhooks since the namespace may have changed and we cannot recreate them in remote services easily.
2014-11-09 17:50:57 -05:00
Joseph Schorr
17f605a9ef
Select only a single token.
2014-11-09 15:50:50 -05:00
Joseph Schorr
691be49817
Fix issues with the perf updated code
2014-11-07 14:36:32 -05:00
Joseph Schorr
158acd4f41
- Turn on foreign key constraint checking in the tests
...
- Change all ForeignKeyField's that refer to users to use our custom class, and mark those that allow robots
- Change robot delete to only execute the subset of queries necessary to actually delete robots
2014-11-07 12:05:21 -05:00
Joseph Schorr
23d9bd2b42
Change verbs to use short lived database connections
2014-11-06 17:50:48 -05:00
Joseph Schorr
c569299e5c
Database optimizations around image creation and logs lookup
2014-11-06 14:48:16 -05:00
Joseph Schorr
a35bc11912
Add perf comments
2014-11-05 12:27:38 -05:00
Joseph Schorr
4eedd54b66
- Make usage language more accurate by stating "repositories"
...
- Have usage counter be based on a 4 weeks TTL
- Add a simple usage counter breakage test
2014-10-30 13:26:02 -04:00
Joseph Schorr
c1398c6d2b
- Add a log entry for repo verb handling and make the container usage calculation take it into account
...
- Move all the repo push/pull/verb logging into a central track_and_log method
- Readd images accidentally deleted in the last CL
- Make the uncompressed size migration script better handle exceptions
2014-10-29 15:42:44 -04:00
Joseph Schorr
e9c2e54dbc
Change delete to once again not perform everything under a transaction. A recent revision accidentally moved everything back under it.
2014-10-29 11:59:02 -04:00
Joseph Schorr
109850b428
Add a basic usage counter for enterprise
2014-10-28 16:33:13 -04:00
Jake Moshenko
1461310ab8
Merge remote-tracking branch 'origin/master' into nomenclature
...
Conflicts:
endpoints/common.py
endpoints/notificationhelper.py
test/data/test.db
workers/dockerfilebuild.py
2014-10-23 13:25:37 -04:00
Joseph Schorr
5db9cd948b
Add better (jinja-based) messaging to the notifications and add some fixes for the email templates
2014-10-22 19:01:56 -04:00
Joseph Schorr
28a463f998
Make the 'created' field optional, in case the user does not specify it via the API
2014-10-20 12:07:49 -04:00
Jake Moshenko
dc5ee43a3a
MySQL still doesn't allow us to have empty IN clauses.
2014-10-17 17:49:18 -04:00
Jake Moshenko
6ca0115b5e
MySQL doesnt allow us to use the table we are modifying in a subquery.
2014-10-17 17:01:55 -04:00
Jake Moshenko
efe8825a15
We can't even use an empty in query for images.
2014-10-17 16:37:05 -04:00
Jake Moshenko
acd5a1e07e
Only garbage collect storage when it might result in something. Return the count of the removed images to fulfill the contract expected by some callers.
2014-10-17 16:13:32 -04:00
Jake Moshenko
18a944427f
Merge branch 'master' of bitbucket.org:yackob03/quay
2014-10-17 15:26:58 -04:00
Jake Moshenko
c093e5a326
Add a whitelist of candidate storages which will speed up the orphan queries and limit the damage of GC run amok.
2014-10-17 15:26:51 -04:00
Joseph Schorr
26a9bb50d5
Fix spelling mistake
2014-10-17 14:44:17 -04:00
Jake Moshenko
d8149295ab
Update the GC code to do everything with subqueries, making each GC run a bounded finite number of queries with a fixed length.
2014-10-17 14:33:04 -04:00
Jake Moshenko
380eb49e58
Split out the GC code into smaller transactions. Remove from the actual storage component after the fact, outside of the transactions.
2014-10-17 11:42:09 -04:00
Joseph Schorr
bb9502ee77
Make sure to raise the exception for WSGI and make the delete method's intent more clear
2014-10-08 13:51:50 -04:00
Joseph Schorr
d16fdde528
Fix bug in dockerloadformat and make sure we handle exceptions properly in the verb call
2014-10-08 13:43:12 -04:00
Jake Moshenko
1a8287af55
Missed a table import.
2014-10-07 17:33:20 -04:00
Jake Moshenko
a0d94f9d59
Merge remote-tracking branch 'origin/laffa' into nomenclature
...
Conflicts:
test/data/test.db
2014-10-07 15:56:28 -04:00
Joseph Schorr
f4daa5e97b
- Update the migrations tool to verify migrations work up and down for both MySQL and PostgresSQL.
...
- Add migrations for the squashed image tables and for backfilling the uncompressed sizes
- Make sure gzip stream uses a max length when determining the uncompressed size
2014-10-07 15:29:56 -04:00
Joseph Schorr
f38ce51943
Merge master into laffa
2014-10-07 14:03:17 -04:00
Jake Moshenko
ed8bcff39e
Merge remote-tracking branch 'origin/master' into nomenclature
...
Conflicts:
test/data/test.db
workers/dockerfilebuild.py
2014-10-06 10:29:39 -04:00
Jake Moshenko
5c18ffe67d
Allow the namespace column to be null, and also non-unique. Fix the uncompressed size clobbering the size on the wire field. Add metadata constraints so that foreign key constraints get predictable names. Fix all downgrade migrations.
2014-10-02 10:46:20 -04:00
Jake Moshenko
e8b3d1cc4a
Phase 4 of the namespace to user migration: actually remove the column from the db and remove the dependence on serialized namespaces in the workers and queues
2014-10-01 14:23:46 -04:00
Joseph Schorr
6c520b8b0b
Fixed broken query
2014-09-30 13:19:32 -04:00
Joseph Schorr
5f776771d1
Merge branch 'master' of https://bitbucket.org/yackob03/quay
2014-09-29 17:01:07 -04:00
Joseph Schorr
474add0fb1
Have the layer PUT method calculate the uncompressed size in realtime, as trusting the JSON is fraught with complications
2014-09-29 17:00:47 -04:00
Jake Moshenko
fbc7ee3bce
Phase 3 of the namespace migration, remove write references to the namespace string column.
2014-09-29 15:44:25 -04:00
Jake Moshenko
03190efde3
Phase 2 of migrating repo namespaces to referencing user objects, backfilling the rows without a value for namespace_user, and changing all accesses to go through the namespace_user object. All tests are passing, manual testing still required.
2014-09-24 18:01:35 -04:00
Jake Moshenko
98d8e0fe37
Fixes for the uncompressed size backfill script.
2014-09-23 16:06:59 -04:00
Joseph Schorr
ec484e3efc
Move size and checksum updates into the metadata call
2014-09-23 15:49:28 -04:00
Joseph Schorr
87bc37f6c8
Merge branch 'sunday'
2014-09-23 11:29:03 -04:00
Joseph Schorr
f3b03ebc34
Add a feature flag for disabling all emails
2014-09-22 19:11:48 -04:00
Jake Moshenko
c68d49dfce
Fix the get_matching_users query to work with peewee 2.3+
2014-09-22 18:42:48 -04:00
Jake Moshenko
3259cda000
The new strategy is to do a three phase migration. This is the first phase: getting the namespace user in the db and written for all new repositories.
2014-09-22 17:27:02 -04:00
Jake Moshenko
8626d1cd70
Initial changes to move repositories from using a namespace string to referencing a user object. Also stores the user id in the cookie rather than the username, to allow users to be renamed. This commit must not be used unmodified because the database migration is too aggressive for live migration.
2014-09-19 10:17:23 -04:00
Jake Moshenko
8c00eabedd
Merge remote-tracking branch 'origin/better-emails'
2014-09-19 10:04:02 -04:00
Joseph Schorr
6b05b55225
Add unicode password support
2014-09-18 17:36:26 -04:00
Jake Moshenko
11bb8e6448
Actually store the generated image storage in the database, and allow it to be garbage collected when the parent image storage is collected.
2014-09-18 17:26:40 -04:00
Joseph Schorr
b212dbb2ab
Merge branch 'master' into better-emails
2014-09-18 13:20:32 -04:00
Jake Moshenko
75d2ef377e
Merge remote-tracking branch 'origin/master' into comewithmeifyouwanttowork
...
Conflicts:
data/model/legacy.py
2014-09-15 17:52:17 -04:00
Joseph Schorr
913b3e472f
Add ability to detach external login services
2014-09-15 12:01:02 -04:00
Joseph Schorr
e8ad01cb41
Lots of small NPE and other exception fixes
2014-09-15 11:27:33 -04:00
Joseph Schorr
10faa7de84
Only allow users matching the team invite to accept, if the invite was specified for a user (rather than an email)
2014-09-12 14:29:01 -04:00
Jake Moshenko
c5ca46a14b
Merge remote-tracking branch 'origin/master' into comewithmeifyouwanttowork
...
Conflicts:
data/model/legacy.py
static/js/app.js
2014-09-12 11:03:30 -04:00
Joseph Schorr
7c45aca405
Code review changes
2014-09-08 17:20:01 -04:00
Jake Moshenko
451e034ca1
Archived logs commit 1. Squash me.
2014-09-08 16:43:17 -04:00
Joseph Schorr
3c20402b32
Add a common base email template, translate the emails over to using jinja and add emails when e-mail addresses and passwords are changed.
2014-09-05 19:57:33 -04:00
Joseph Schorr
fa1abd5eda
Don't require the invite email to match the invited user
2014-09-04 18:42:23 -04:00
Joseph Schorr
e028d4ae0a
Merge master into branch
2014-09-04 18:08:18 -04:00
Jake Moshenko
2dcdd7ba5b
Add exponential backoff of login attempts.
2014-09-02 15:27:05 -04:00
Jake Moshenko
ce7e3a8733
Do not link against layers that are still marked as uploading, there is no guarantee that they will ever be completed and their ancestry may be incomplete.
2014-08-29 13:16:32 -04:00
Joseph Schorr
3b72b26836
Merge branch 'master' into comewithmeifyouwanttowork
2014-08-28 20:50:13 -04:00
Joseph Schorr
f15b3f345e
Review updates
2014-08-28 19:07:22 -04:00
Joseph Schorr
d76d4704a0
Add pagination to the notifications API and make the UI only show a maximum of 5 notifications (beyond that, it shows "5+").
2014-08-26 15:19:39 -04:00
Joseph Schorr
a129aac94b
Add ability to regenerate robot account credentials
2014-08-25 17:19:23 -04:00
Joseph Schorr
d2880807b2
- Further fixes for license stuff
...
- Small fixes to ensure Quay works for Postgres
2014-08-21 19:21:20 -04:00
Joseph Schorr
02d3b70013
Make sure to search teams as well when determining the robots which have access to a private repo
2014-08-18 19:19:01 -04:00
Joseph Schorr
43b6695f9c
Get team invite confirmation working and fully tested
2014-08-18 17:24:00 -04:00
Joseph Schorr
56d7a3524d
Work in progress: Require invite acceptance to join an org
2014-08-15 17:47:43 -04:00
Joseph Schorr
389c88a7c4
Update federated login to store metadata and have the UI pull the information from the metadata
2014-08-11 18:25:01 -04:00
Joseph Schorr
49801bc2c4
- Add web hook queue code back in. We'll remove it and turn it off after this CL goes to prod
...
- Make notification lookup always be by repo and its UUID, rather than the internal DB ID
- Add the init script for the notification worker
2014-07-31 13:30:54 -04:00
Joseph Schorr
7de1dd7dc0
Merge branch 'master' into redalert
2014-07-28 18:35:39 -04:00
Joseph Schorr
32b2ecdfa6
Add ability to dismiss notifications
2014-07-28 18:23:46 -04:00
Joseph Schorr
34fc279092
Add e-mail authorization to the repository notification flow. Also validates the creation of the other notification methods.
2014-07-28 14:58:12 -04:00
Jake Moshenko
9ad7fc7a54
Prevent extra queries to load image storage data.
2014-07-22 16:46:36 -04:00
Joseph Schorr
752efb9e0f
Fix the spawn_notification to work in all cases and clean up some of the remaining code
2014-07-18 16:34:52 -04:00
Joseph Schorr
591cd020b8
Merge branch 'master' into redalert
2014-07-18 15:58:56 -04:00
Jake Moshenko
8a2c88488a
Filter tokens with an associated build trigger from the list API. Delete associated tokens when deleting a trigger.
2014-07-18 14:56:26 -04:00
Joseph Schorr
8d7493cb86
Convert over to notifications system. Note this is incomplete
2014-07-17 22:51:58 -04:00
Joseph Schorr
de8e898ad0
Add UI for managing repo notifications
2014-07-17 13:32:39 -04:00
Joseph Schorr
a84fe0681a
Start on data model changes and API changes for the new repository notification system
2014-07-16 16:30:47 -04:00
Jake Moshenko
d851feef6e
One of the calls which returns an image was not using the base format which includes locations yet.
2014-06-30 14:51:01 -04:00
Jake Moshenko
684c26bc1a
Populate image storage objects with locations everywhere.
2014-06-27 20:04:26 -04:00
Jake Moshenko
e68d6a7302
Merge branch 'master' of ssh://bitbucket.org/yackob03/quay
2014-06-27 19:18:35 -04:00
Joseph Schorr
76165b5d2b
Add API and UI support for displaying image locations
2014-06-24 18:48:42 -04:00
Jake Moshenko
6047f3759f
Remove the placement fallback since the DB has been fully backfilled.
2014-06-24 17:01:23 -04:00
Jake Moshenko
0a62f7f725
Add the ability to look up images which do not have a placement yet.
2014-06-18 12:40:23 -04:00
Jake Moshenko
bf0e01fba5
Fix bugs with pushing, pulling, and deleting.
2014-06-17 16:37:48 -04:00
Jake Moshenko
bf98575feb
Add the basics of geographic data distribution and get the tests to work.
2014-06-17 16:03:43 -04:00
Jake Moshenko
78c5aec5b9
Switch the checksums to use the registry computed value, remove all assumptions of namespaced paths for legacy storage, fix an upload race condition in the registry code.
2014-06-11 15:37:45 -04:00
Jake Moshenko
0ba4201020
Add a module which will create notifications for all users when the license is at its expiration period, and terminate the process when the license expires.
2014-05-29 11:24:10 -04:00
Jake Moshenko
33b43b75c0
Eliminate a lot of the if cases in create_user by separating them out. Add a limit to the number of users which can be created based on the license. Add support for creating and loading licenses.
2014-05-28 13:51:52 -04:00
Jake Moshenko
0b6552d6cc
Fix the metrics so they are usable for scaling the workers down and up. Switch all datetimes which touch the database from now to utcnow. Fix the worker Dockerfile.
2014-05-23 14:16:26 -04:00
Jake Moshenko
f6726bd0a4
Merge branch 'ldapper'
...
Conflicts:
Dockerfile
app.py
data/database.py
endpoints/index.py
test/data/test.db
2014-05-22 12:13:41 -04:00
Jake Moshenko
8b5c781f84
Change the health check to ping the db and the redis build logs.
2014-05-13 18:53:42 -04:00
Jake Moshenko
553ef36e9b
Fix a remaining direct usage of transaction factory in the data model.
2014-05-13 15:17:16 -04:00
Jake Moshenko
5fdccfe3e6
Add an alembic migration for the full initial database with the data. Switch LDAP to using bind and creating a federated login entry. Add LDAP support to the registry and index endpoints. Add a username transliteration and suggestion mechanism. Switch the database and model to require a manual initialization call.
2014-05-13 12:17:26 -04:00
Jake Moshenko
ef9fe871fc
Move the upload flag to the database, and use the database stored image size rather than going to s3.
2014-05-02 16:59:46 -04:00
jakedt
0827e0fbac
Merge remote-tracking branch 'origin/master' into ncc1701
...
Conflicts:
endpoints/web.py
static/directives/signup-form.html
static/js/app.js
static/js/controllers.js
static/partials/landing.html
static/partials/view-repo.html
test/data/test.db
2014-04-14 19:37:22 -04:00
jakedt
4b8217d4ad
Add config to allow for setting the queue names at runtime. Fix a bug in the data model.
2014-04-11 19:23:57 -04:00
jakedt
14fba3ae7c
Rename SERVER_NAME to SERVER_HOSTNAME to fix the subdomain routing problems.
2014-04-11 11:17:45 -04:00
Joseph Schorr
bdf2b02c1a
Merge branch 'ncc1701' of https://bitbucket.org/yackob03/quay into ncc1701
2014-04-10 00:27:51 -04:00
Joseph Schorr
0e320c964f
- Add support for super users
...
- Add a super user API
- Add a super user interface
2014-04-10 00:26:55 -04:00
jakedt
fc7756a3c2
Add alembic plumbing for database schema migrations.
2014-04-09 19:11:33 -04:00
Joseph Schorr
0e54b0501c
Return the reason a username validation failed and add tests to verify we are sending the reason to client
2014-04-07 20:37:02 -04:00
jakedt
e87ffa20cf
First attempt at making config loadable through string config overrides in an env variable.
2014-04-03 17:31:46 -04:00
Joseph Schorr
9a79d1562a
Change to store the pull robot on the repository build and only add the credentials to the queue item. This prevents the credentials from being exposed to the end user. Also fixes the restart build option
2014-04-01 21:49:06 -04:00
jakedt
ca1970a2f4
Use the real registry endpoint in the login command.
2014-04-01 19:00:11 -04:00
jakedt
126363dce9
Use our patched version of docker-py with build credentials patched in. Fix the hostname in the build credentials block to be variable.
2014-04-01 11:38:17 -04:00
jakedt
d67a1cddc2
Merge remote-tracking branch 'origin/master' into pullinprivate
...
Conflicts:
workers/dockerfilebuild.py
2014-03-31 18:10:34 -04:00
jakedt
fd941ed3a4
Strip tzinfo data from datetimes to make them compatible with peewee on sqlite.
2014-03-28 17:52:47 -04:00
Joseph Schorr
2006917e03
Add support for pull credentials on builds and build triggers
2014-03-27 18:33:13 -04:00
jakedt
41cfadac23
Protect the search and repository list endpoints appropriately. Add more differentiating data to some need types. Remove the notification about password change from the user admin page. Select the dependent models for the visible repo list.
2014-03-25 17:26:45 -04:00
jakedt
669ec9c382
Change the token expiration time to 10 years.
2014-03-25 15:38:16 -04:00
jakedt
0097daebc2
Formatting changes.
2014-03-25 14:32:02 -04:00
jakedt
b81e48cb41
Merge branch 'swaggerlikeus' of ssh://bitbucket.org/yackob03/quay into swaggerlikeus
...
Conflicts:
test/data/test.db
2014-03-25 12:43:09 -04:00
jakedt
cbc40588cb
Finally figure out what the data field is supposed to be for and use it to implement and fix 3LO.
2014-03-25 12:42:40 -04:00
Joseph Schorr
c82d1ffe98
Add ability for users to see their authorized applications and revoke the access
2014-03-24 20:57:02 -04:00
Joseph Schorr
e92cf37583
Add cancel button to the oauth authorization page, add the org icon to said page, and fix some other minor bugs
2014-03-24 18:30:22 -04:00
Joseph Schorr
f7c27f250b
Add full application management API, UI and test cases
2014-03-20 15:46:13 -04:00
jakedt
3b7b12085d
User scope objects everywhere. Switch scope objects to namedtuples. Pass the user when validating whether the user has authorized such scopes in the past. Make sure we calculate the scope string using all user scopes form all previously granted tokens.
2014-03-19 18:09:09 -04:00
jakedt
c93c62600d
Merge remote-tracking branch 'origin/master' into swaggerlikeus
...
Conflicts:
data/database.py
endpoints/api.py
endpoints/common.py
templates/base.html
test/data/test.db
test/specs.py
2014-03-19 15:39:44 -04:00
Joseph Schorr
b0dcb5d7e3
Merge branch 'swaggerlikeus' of https://bitbucket.org/yackob03/quay into swaggerlikeus
2014-03-18 16:46:28 -04:00
Joseph Schorr
9ae4506a0d
Add OAuth usage information the API logs, have it be displayed in the logs UI and start on the code to display application information when clicked. Note that this does not (yet) do anything with the information returned as we need to wait for the mainline merge of Angular 1.2.9 (which is in master) before I can continue on the display
2014-03-18 16:45:18 -04:00
jakedt
6f39e158d6
Eliminate all of the exceptions when running the tests.
2014-03-18 15:58:37 -04:00
Joseph Schorr
d469b41899
Add an oauth authorization page
2014-03-14 18:57:28 -04:00
jakedt
e74eb3ee87
Add scope ordinality and translations. Process oauth tokens and limit scopes accordingly.
2014-03-12 16:31:37 -04:00
jakedt
25ceb90fc6
Add some sort of oauth.
2014-03-12 12:37:06 -04:00