Joseph Schorr
38a6b3621c
Automatically link the superuser account to federated service for auth
...
When the user commits the configuration, if they have chosen a non-DB auth system, we now auto-link the superuser account to that auth system, to ensure they can login again after restart.
2015-07-22 13:37:23 -04:00
Joseph Schorr
a0c4e72f13
Clean up the repository list API and loads stars with it
...
We load stars with the same list API now so that we get the extra metadata needed in the repo list (popularity and last modified)
2015-07-22 13:05:02 -04:00
Joseph Schorr
7e4b23916a
Small SQL query fix
...
Fixes #248
2015-07-20 14:17:26 -04:00
Joseph Schorr
33b54218cc
Refactor the users class into their own files, add a common base class for federated users and add a verify_credentials
method which only does the verification, without the linking. We use this in the superuser verification pass
2015-07-20 11:39:59 -04:00
Joseph Schorr
1245385808
Fix typo
2015-07-20 10:55:21 -04:00
Joseph Schorr
066637f496
Basic Keystone Auth support
...
Note: This has been verified as working by the end customer
2015-07-20 10:55:21 -04:00
Jake Moshenko
bc29561f8f
Fix and templatize the logic for external JWT AuthN and registry v2 Auth.
...
Make it explicit that the registry-v2 stuff is not ready for prime time.
2015-07-17 11:56:15 -04:00
Jake Moshenko
3efaa255e8
Accidental refactor, split out legacy.py into separate sumodules and update all call sites.
2015-07-17 11:56:15 -04:00
Jake Moshenko
bea8b9ac53
More changes for registry-v2 in python.
...
Implement the minimal changes to the local filesystem storage driver and feed them through the distributed storage driver.
Create a digest package which contains digest_tools and checksums.
Fix the tests to use the new v1 endpoint locations.
Fix repository.delete_instance to properly filter the generated queries to avoid most subquery deletes, but still generate them when not explicitly filtered.
2015-07-17 11:50:41 -04:00
Jake Moshenko
acbcc2e206
Start of a v2 API.
2015-07-17 11:50:41 -04:00
Joseph Schorr
7a548ea101
Fix queries for repository list popularity and action count
...
Before this change, we used extremely inefficient outer joins as part of a single query of lookup, which was spiking our CPU usage to nearly 100% on the query. We now issue two separate queries for popularity and action account, by doing a lookup of the previously found IDs. Interestingly enough, because of the way the queries are now written, MySQL can actually do both queries *directly from the indicies*, which means they each occur in approx 20ms!
Verified by local tests, postgres tests, and testing on staging with monitoring of our CPU usage during lookup
2015-07-17 00:08:27 +03:00
Jimmy Zelinskie
2869e2a6ea
model: add missing params to validate_database_url
2015-07-15 17:39:26 -04:00
Jake Moshenko
eec7886e01
Add a server default for the broken migration.
2015-07-14 16:58:58 -04:00
Jimmy Zelinskie
bde781c98b
Merge pull request #205 from coreos-inc/delrobot
...
Fix deletion of robot accounts when attached to builds
2015-07-13 12:19:01 -04:00
Joseph Schorr
3a59c99b08
Add a secondary tab to Teams for managing org members
...
Also adds the ability to completely remove a user from an organization (repo permissions and teams), in a single click
Fixes #212
2015-07-02 17:06:36 +03:00
Joseph Schorr
b535e222b8
Have the fetch tag dialog show a warning for robot accounts without access
...
Before this change, we'd show the squash pulling command with the proper credentials, but it then 403s on the end user.
2015-07-01 19:37:52 +03:00
Joseph Schorr
3ba321934f
Fix deletion of robot accounts when attached to builds
...
Fixes #204
2015-06-30 22:56:44 +03:00
josephschorr
7aeaf2344e
Merge pull request #200 from coreos-inc/tagapilimit
...
Add pagination support to tag history API
2015-06-30 22:09:09 +03:00
Joseph Schorr
f7f10f4a6d
Add pagination support to tag history API
...
Fixes #198
2015-06-30 19:44:43 +03:00
Jake Moshenko
38a5963afe
Merge pull request #190 from coreos-inc/timezone
...
Fromtimestamp needs to be in UTC for JWT auth
2015-06-30 12:05:00 -04:00
Joseph Schorr
2b1bbcb579
Add a table view to the repos list page
...
Fixes #104
2015-06-29 21:12:53 +03:00
Jimmy Zelinskie
756d6784ca
Merge pull request #192 from coreos-inc/sqlssl
...
Allow SSL cert for the database to be configured
2015-06-29 13:33:31 -04:00
Joseph Schorr
dc5af7496c
Allow superusers to disable user accounts
2015-06-29 18:40:52 +03:00
Joseph Schorr
bb07d0965f
Allow SSL cert for the database to be configured
...
This change adds a field for the SSL cert for the database in the setup tool. Fixes #89
2015-06-29 08:08:10 +03:00
Joseph Schorr
477e244eff
Fromtimestamp needs to be in UTC for JWT auth
2015-06-28 11:37:09 +03:00
Joseph Schorr
e7915baf8c
Have LDAP return a better error message if it fails to connect
...
Currently, the error results in a 500 being raised when a user tries to login.
2015-06-23 17:41:53 -04:00
Joseph Schorr
07439328a4
Remove user_exists
endpoint from all auth systems
2015-06-23 17:33:51 -04:00
Jake Moshenko
b21a033ef3
Merge pull request #131 from coreos-inc/moveapp
...
Refactor JWT auth to not import app locally
2015-06-23 17:24:01 -04:00
Jake Moshenko
5f1d23c6e8
Use a UNION query instead of a multitude of left outer joins for performance reasons.
...
Fixes #159
2015-06-23 17:18:37 -04:00
Joseph Schorr
331c300893
Refactor JWT auth to not import app locally
2015-06-17 15:53:21 -04:00
Joseph Schorr
e7fa560787
Add support for custom fields in billing invoices
...
Customers (especially in Europe) need the ability to add Tax IDs, VAT IDs, and other custom fields to their invoices.
Fixes #106
2015-06-12 16:45:01 -04:00
Jake Moshenko
79f1181a63
Switch build-scheduled to an official build phase.
2015-06-10 16:19:51 -04:00
Jake Moshenko
e09d84b3c8
Merge pull request #55 from coreos-inc/oauthdeny
...
Fix OAuth redirect for denial action when generating for internal tokens
2015-06-05 14:00:16 -04:00
Joseph Schorr
8aac3fd86e
Add support for an external JWT-based authentication system
...
This authentication system hits two HTTP endpoints to check and verify the existence of users:
Existance endpoint:
GET http://endpoint/ with Authorization: Basic (username:) =>
Returns 200 if the username/email exists, 4** otherwise
Verification endpoint:
GET http://endpoint/ with Authorization: Basic (username:password) =>
Returns 200 and a signed JWT with the user's username and email address if the username+password validates, 4** otherwise with the body containing an optional error message
The JWT produced by the endpoint must be issued with an issuer matching that configured in the config.yaml, and the audience must be "quay.io/jwtauthn". The JWT is signed using a private key and then validated on the Quay.io side with the associated public key, found as "jwt-authn.cert" in the conf/stack directory.
2015-06-05 13:20:10 -04:00
Jake Moshenko
42da017d69
Merge pull request #48 from coreos-inc/nobots
...
Change API calls that expect non-robots to explicitly filter
2015-06-02 12:31:19 -04:00
Joseph Schorr
5516911de9
Fix OAuth redirect for denial action when generating for internal tokens
2015-06-02 12:25:59 -04:00
Joseph Schorr
2a56790d38
Switch to using a named LDAP tuple for more readable code
2015-06-01 14:02:05 -04:00
Joseph Schorr
1aff701bc7
Fix LDAP referral and multiple pair handling
...
Fixes two issues found with our LDAP handling code. First, we now follow referrals in both LDAP calls, as some LDAP systems will return a referral instead of the original record. Second, we now make sure to handle multiple search result pairs properly by further filtering based on the presence of the 'mail' attribute when we have multiple valid pairs. This CL also adds tests for all of the above cases.
2015-05-27 15:04:34 -04:00
Joseph Schorr
fdd43e2490
Change API calls that expect non-robots to explicitly filter
...
Before this change, we'd filter in the UI but calls to the API could allow robots accounts where we only expect real users
2015-05-26 17:47:33 -04:00
Joseph Schorr
b0d763b5ff
Fix encrypted password generator to use the LDAP username, not the Quay username.
...
Currently, we use the Quay username via `verify_user` when we go to create the encrypted password. This is only correct if Quay has not generated its own different username for the LDAP user, and fails if it has. We therefore add a new method `confirm_existing_user`, which looks up the federated login for the LDAP user and then runs the auth flow using that username.
2015-05-20 16:37:09 -04:00
Joseph Schorr
d5e70c6e2a
Explicitly enable LDAP referrals
...
Note: The mock LDAP system doesn't support referrals, so we can't add a unit test for this.
2015-05-20 14:53:31 -04:00
Jimmy Zelinskie
3daa0e6a52
migration: use SA boolean value for server_default
2015-05-15 19:04:54 -04:00
Jimmy Zelinskie
e12d5af52c
migration: server_default for boolean field
2015-05-15 17:04:50 -04:00
Joseph Schorr
6a52ffa942
Add missing default value on the new reversion column
2015-05-13 13:49:21 -04:00
Joseph Schorr
07b4fb9105
LDAP sometimes has multiple records for a user
2015-05-12 12:02:09 -04:00
Joseph Schorr
efab02ae47
LDAP improvements:
...
- Better logging
- Better error messages
- Add unit tests
- Clean up the setup tool for LDAP
2015-05-11 21:23:18 -04:00
Joseph Schorr
3e1abba284
Add ability for super users to rename and delete organizations
2015-05-11 18:03:25 -04:00
Joseph Schorr
1c41d34b7c
Add ability for superusers to change user emails
2015-05-11 14:38:10 -04:00
Joseph Schorr
1c83def15b
LDAP should only show logs when asked.
2015-05-11 13:01:49 -04:00
Joseph Schorr
f858caf6cd
Only return the team and repo permissions when listing robots when we absolutely need them.
2015-05-08 16:43:07 -04:00
Joseph Schorr
561f2c7db0
Remove unnecessary sort
2015-05-08 15:10:31 -04:00
Joseph Schorr
3b93854c0b
MySQL returns a decimal type for the tuple, which we need to convert to an int
2015-05-08 14:55:54 -04:00
Joseph Schorr
36f33e8fd7
Merge branch 'master' of github.com:coreos-inc/quay
2015-05-08 14:19:39 -04:00
Joseph Schorr
f9c1f123c2
Add better debugging to LDAP
2015-05-08 14:19:32 -04:00
Joseph Schorr
8ed8367404
PR changes in response to comments
2015-05-08 13:38:34 -04:00
Joseph Schorr
c767aafcd6
Make the repository API faster by only checking the log entries table once for each kind of entry, rather than twice. We make use of a special subquery-like syntax, which allows us to count those entries that are both 30 days only and 1 day old in the same query. This was tested successfully on MySQL, Postgres and Sqlite.
2015-05-07 22:49:11 -04:00
Joseph Schorr
3627de103c
Minimize the queries used when retrieve builds. Previously, we'd call out to SQL extra times per build.
2015-05-07 21:11:15 -04:00
Joseph Schorr
7b35555776
Make sure to test for unicode usernames, since the collate on the username field is latin1
2015-05-07 18:13:45 -04:00
Joseph Schorr
729249f42b
Need to add the tables to the .select() to prevent N+1, not merely to the join
2015-05-07 16:59:13 -04:00
Joseph Schorr
b50600cde8
Make sure to join the Repo table on its namespace user and its visibility so we don't issue additional queries from the starred repos API call
2015-05-07 16:18:17 -04:00
Joseph Schorr
3cec22defe
Make sorting by repo ID, rather than description
2015-05-07 13:10:26 -04:00
Joseph Schorr
73193e2ab5
Remove sort by description, which is slow (and fairly useless anyway)
2015-05-06 20:55:31 -04:00
Joseph Schorr
c89760278f
Add migration to add gitlab trigger type
2015-05-03 10:49:13 -07:00
Joseph Schorr
a59100b231
Add the missing index on the peewee side. We already have the associated migration.
2015-04-30 15:56:40 -04:00
Joseph Schorr
17bc9b81bf
Add bitbucket trigger type
2015-04-30 15:55:57 -04:00
Joseph Schorr
b96e35b28c
Merge master into bitbucket
2015-04-30 15:52:08 -04:00
Joseph Schorr
31260d50f5
Rename the new images method to a slightly better name
2015-04-24 16:37:37 -04:00
Joseph Schorr
e70343d849
Faster cache lookup by removing a join with the ImagePlacementTable, removing the extra loop to add the locations and filtering the images looked up by the base image
2015-04-24 16:22:19 -04:00
Jimmy Zelinskie
fd65ca5916
migration: add custom-git service to database
2015-04-24 16:11:58 -04:00
Joseph Schorr
c480fb2105
Work in progress: bitbucket support
2015-04-24 15:13:08 -04:00
Jimmy Zelinskie
d25cc4db9c
migrations: fix percona migration test
2015-04-21 18:01:40 -04:00
Jimmy Zelinskie
93cd459460
Merge branch 'master' into git
2015-04-20 10:58:49 -04:00
Joseph Schorr
43ff6839b8
Hide hidden tags in the tags timeline
2015-04-19 18:12:06 -04:00
Joseph Schorr
ed342ae831
Add migration for properly creating the repository_id+datetime index
2015-04-19 16:03:06 -04:00
Joseph Schorr
d1e2d072ea
Add unit tests and a stronger restriction on the revert API call
2015-04-19 15:43:16 -04:00
Joseph Schorr
a8f8c317f9
Fix branch in alembic migrations
2015-04-19 15:20:01 -04:00
Joseph Schorr
2815ea2723
Merge branch 'master' of github.com:coreos-inc/quay
2015-04-19 15:16:39 -04:00
Joseph Schorr
8714eb207a
Merge branch 'actioncount'
2015-04-19 15:12:44 -04:00
Jimmy Zelinskie
4133924813
migrations: rebase on top of master's changes
2015-04-17 13:19:04 -04:00
Jimmy Zelinskie
ba2cb08904
Merge branch 'master' into git
2015-04-16 17:38:35 -04:00
Joseph Schorr
f19d2f684e
Add ability to revert tags via time machine
2015-04-16 17:18:00 -04:00
Joseph Schorr
f8c80f7d11
Add a history view to the tags page. Next step will add the ability to revert back in time
2015-04-15 15:21:09 -04:00
Joseph Schorr
3f1e8f3c27
Add a RepositoryActionCount table so we can use it (instead of LogEntry) when scoring repo search results
2015-04-13 13:31:07 -04:00
Joseph Schorr
1df025b57e
Change search to use a set of queries for repo lookup rather than a single monolithic query, in the hopes that this will make things significantly faster and actually useable. The individual queries have been tested by hand on MySQL, but the real test will be staging
2015-04-10 15:27:37 -04:00
Joseph Schorr
0be0aed17d
Move the repo sorting by pull count into the main matching query, to both make it more accurate and make the search faster
2015-04-09 14:41:59 -04:00
Joseph Schorr
396cba64e6
Fix search to return better results by searching for robots and namespaces in different queries.
2015-04-09 12:57:20 -04:00
Joseph Schorr
762fdaf319
Fix DB migration tests
2015-04-08 17:41:53 -04:00
Joseph Schorr
4f4bb05621
Fix search SQL issues
2015-04-08 17:41:08 -04:00
Joseph Schorr
19e25ac340
Merge branch 'master' into bing
2015-04-08 15:23:36 -04:00
Joseph Schorr
d09f2f6e22
Get the new context-sensitive new menu working
2015-04-07 18:33:43 -04:00
Joseph Schorr
40a6892a49
Add search tests
2015-04-07 14:05:12 -04:00
Joseph Schorr
1b56567268
Make sure also include teams from organizations that the user admins
2015-04-07 13:45:49 -04:00
Joseph Schorr
951b0cbab8
Start on new interactive search
2015-04-06 19:17:18 -04:00
Joseph Schorr
4cb7921c3a
Make sure to show public repos on the user and orgs pages
2015-04-03 14:55:09 -04:00
Joseph Schorr
094f91fb8b
Fix the tutorial's user events
2015-04-03 12:13:33 -04:00
Joseph Schorr
5cd500257d
Merge branch 'master' into orgview
2015-04-01 13:56:49 -04:00
Joseph Schorr
fde9666647
Add the team membership to the robots view
2015-04-01 13:56:30 -04:00
Joseph Schorr
1f5e6df678
- Fix tests
...
- Add new endpoints for retrieving the repo permissions for a robot account
- Have the robots list return the number of repositories for which there are permissions
- Other UI fixes
2015-03-31 18:50:43 -04:00
Joseph Schorr
27a9b84587
Switch avatars to be built out of CSS and only overlayed with the gravatar when a non-default exists
2015-03-30 17:55:04 -04:00
Jimmy Zelinskie
da15eda2bf
migration: make auth_token for builds nullable
2015-03-27 11:20:30 -04:00
Joseph Schorr
7b1d35737e
Merge branch 'master' of github.com:coreos-inc/quay
2015-03-26 17:45:49 -04:00
Joseph Schorr
a7b6cb5c23
Fix handling of byte strings and large ints
2015-03-26 17:45:43 -04:00
Jimmy Zelinskie
4d4947edaf
Merge branch 'master' of github.com:coreos-inc/quay
2015-03-26 16:31:20 -04:00
Jimmy Zelinskie
bcc7a9580b
models: change UUID of user on password change
...
This prevents old cookies from continuing to work after a password has
been changed.
2015-03-26 16:30:41 -04:00
Joseph Schorr
c4a2574b0d
Clarify unencrypted password error message
2015-03-26 16:23:28 -04:00
Joseph Schorr
f8afd8b5ce
Make sure to parse the big int into a byte string
2015-03-26 16:13:35 -04:00
Joseph Schorr
4d1792db1c
getrandbits creates an int, not a float
2015-03-26 15:47:44 -04:00
Joseph Schorr
aaf1b23e98
Address CL concerns and switch to a real encryption system
2015-03-26 15:10:58 -04:00
Joseph Schorr
d23bb6616d
Fix error message to exactly match current output
2015-03-26 13:22:16 -04:00
Joseph Schorr
e4b659f107
Add support for encrypted client tokens via basic auth (for the docker CLI) and a feature flag to disable normal passwords
2015-03-25 18:43:12 -04:00
Jake Moshenko
3d44416016
Add the migration for the unique index which helps prevent tag deadlocks.
2015-03-24 18:00:04 -04:00
Jake Moshenko
201943ed1c
Fix deadlocks with tags and garbage collection.
2015-03-24 18:00:04 -04:00
Jimmy Zelinskie
9c55aca011
migration: make resource_key nullable
2015-03-23 15:46:35 -04:00
Jimmy Zelinskie
93a9e9d01a
migration: add private key to build triggers
2015-03-19 14:30:25 -04:00
Joseph Schorr
ab2331a486
Performance improvements for the repo API and the new repo UI
2015-03-18 14:47:53 -04:00
Joseph Schorr
fe8d006855
Fix subquery on MySQL
2015-03-17 13:54:51 -04:00
Joseph Schorr
44ff85d044
Remove migration (temporarily), fix a broken test, and make the aggregate size calculation use the entire image ancestry (for now).
2015-03-17 12:13:01 -04:00
Joseph Schorr
b8d88c0f4e
Add aggregate size column and a migration to backfill it
2015-03-16 18:03:17 -04:00
Joseph Schorr
333e0acd6d
Add the builds tab
2015-03-13 15:34:28 -07:00
Joseph Schorr
002dc083f2
Get the main repo page design working
2015-03-10 17:22:46 -07:00
Joseph Schorr
afc8e95e19
Start on new tag view
2015-03-09 22:03:39 -07:00
Joseph Schorr
929bad57f5
Fix revision
2015-03-05 15:06:46 -05:00
Joseph Schorr
86447c0a99
Merge branch 'master' into pagesnew
2015-03-05 14:22:10 -05:00
Joseph Schorr
edafa26e6e
Fix email confirmation for users created through the superuser API
2015-03-03 14:26:48 -05:00
Joseph Schorr
a36266f758
Add LDAP tracing
2015-02-27 17:01:46 -05:00
Jake Moshenko
6c4e78ec0c
Fix the deadlock in tags.
2015-02-25 17:49:46 -05:00
Jimmy Zelinskie
fb0d3d69c2
changes to reflect PR comments (not finished)
2015-02-24 17:50:54 -05:00
Jake Moshenko
d8a34427ec
Try to get around the tag deadlock by using a select for update.
2015-02-24 17:18:42 -05:00
Jake Moshenko
246ff556b9
Fix some other list reifications to use nested subqueries for performance and query size safety reasons.
2015-02-23 15:07:38 -05:00
Joseph Schorr
4020cc1102
Reset the number of invalid login attempts when the user change's their password
2015-02-23 13:56:42 -05:00
Joseph Schorr
a0706d20ae
Merge branch 'master' of github.com:coreos-inc/quay
2015-02-23 13:38:04 -05:00
Joseph Schorr
5f605b7cc8
Fix queue handling to remove the dependency from repobuild, and have a cancel method
2015-02-23 13:38:01 -05:00
Jake Moshenko
75d41ca371
Switch to an inner inner subquery to make mysql arbitrarily happy.
2015-02-23 12:36:14 -05:00
Jimmy Zelinskie
7fbbf14e49
migrations: resolve branched migration tree
...
This moves the migration adding stars to HEAD.
2015-02-19 17:10:27 -05:00
Jimmy Zelinskie
917dd6b674
Merge branch 'master' into star
2015-02-18 17:36:58 -05:00
Jake Moshenko
41108a0856
Allow tags to be marked as hidden. Create a hidden tag on every image during a push to prevent them from getting GCed.
2015-02-18 17:05:16 -05:00
Jake Moshenko
59b794dd61
Move the creation of images to when the JSON is uploaded.
2015-02-18 17:04:25 -05:00
Joseph Schorr
89eb5bdcc5
Merge branch 'quark'
2015-02-18 15:59:36 -05:00
Joseph Schorr
a5ff765f3b
Validate that we have a valid JSON body
2015-02-18 15:57:05 -05:00
Jake Moshenko
2dd03f1bed
Merge remote-tracking branch 'origin/master' into rockyhorror
...
Conflicts:
test/data/test.db
2015-02-18 10:56:01 -05:00
Jake Moshenko
2cd5bdb563
Address concerns from pull request.
2015-02-18 10:43:08 -05:00
Joseph Schorr
ff7e97faf7
Merge branch 'master' into quark
2015-02-17 19:16:12 -05:00
Joseph Schorr
524705b88c
Get dashboard working and upgrade bootstrap. Note: the bootstrap fixes will be coming in the followup CL
2015-02-17 19:15:54 -05:00
Joseph Schorr
83e05d2342
Add tracking of the kind of temporary access tokens, so we can display if a pull/push by token is for a build worker
2015-02-17 12:35:16 -05:00
Joseph Schorr
3e04e3cfd7
Merge branch 'master' of github.com:coreos-inc/quay
2015-02-13 15:54:09 -05:00
Joseph Schorr
81ce4c771e
Add ability to cancel builds that are in the waiting state
2015-02-13 15:54:01 -05:00
Jake Moshenko
579d900ef2
Add server defaults to the new fields which are not nullable.
2015-02-13 14:45:42 -05:00
Jake Moshenko
2ce6e76d9d
Add the required migration for time machine tag lifetimes.
2015-02-13 14:41:08 -05:00
Jake Moshenko
5aedd1fabc
Merge remote-tracking branch 'origin/master' into rockyhorror
...
Conflicts:
test/data/test.db
2015-02-13 13:17:10 -05:00
Jake Moshenko
2743fe6f4d
Trap migration exceptions for the migration that we expect to fail in prod.
2015-02-13 13:11:50 -05:00
Jake Moshenko
cd245e62bf
Remove the access_token_id column from log entries.
2015-02-13 11:56:59 -05:00
Jake Moshenko
4a921a49b9
Merge remote-tracking branch 'origin/master' into rockyhorror
2015-02-12 16:24:30 -05:00
Joseph Schorr
f84d1bad45
Handle internal errors in a better fashion: If a build would be marked as internal error, only do so if there are retries remaining. Otherwise, we mark it as failed (since it won't be rebuilt anyway)
2015-02-12 16:19:44 -05:00
Jake Moshenko
b154e7acef
Merge remote-tracking branch 'origin/master' into rockyhorror
2015-02-12 15:27:28 -05:00
Jake Moshenko
990739b1e5
Add the APIs required to change the time machine policy for users and organizations.
2015-02-12 14:37:11 -05:00
Jake Moshenko
872539bdbf
Switch to a per-namespace configurable expiration policy for time machine, and switch the tag gc to respect it.
2015-02-12 14:11:56 -05:00
Joseph Schorr
f107b50a46
Merge branch 'master' into ackbar
2015-02-12 12:04:45 -05:00
Jake Moshenko
f32bd748e4
Use a consistent concept of tag liveness everywhere. Fix the tests.
2015-02-11 15:02:50 -05:00
Jake Moshenko
90c0a9c1e0
First stab at time machine using fixed two week expiration policy.
2015-02-11 14:15:40 -05:00
Joseph Schorr
045614c6c8
Merge branch 'master' into ackbar
2015-02-09 17:16:42 -05:00
Joseph Schorr
3d31c64da2
Make sure to select the latest image in the repository with the matching comment
2015-02-09 16:45:06 -05:00
Joseph Schorr
48949627e0
Merge master in delta
2015-02-09 12:07:43 -05:00
Joseph Schorr
e1c5ccb7d6
Fixes
2015-02-05 17:37:58 -05:00
Joseph Schorr
cf774e23df
Merge branch 'master' into v2
2015-02-05 15:37:14 -05:00
Joseph Schorr
a12bfa7623
Add migration for the new tables for signatures
2015-02-05 15:30:45 -05:00
Joseph Schorr
9dfe523615
Merge master changes
2015-02-05 13:11:16 -05:00
Joseph Schorr
bfb0784abc
Add signing to the ACI converter
2015-02-04 15:29:24 -05:00
Jake Moshenko
ce7033489b
Hopefully fix the deadlock in the queue.
2015-02-03 14:50:01 -05:00
Jake Moshenko
64750e31fc
Add the ability to select for update within transactions to fix some write after read hazards. Fix a bug in extend_processing.
2015-01-30 16:32:13 -05:00
Joseph Schorr
3872d29de9
Add a transaction around the extend_processing call
2015-01-29 18:40:41 -05:00
Joseph Schorr
30b895b795
Merge branch 'grunt-js-folder' of https://github.com/coreos-inc/quay into ackbar
2015-01-23 17:26:14 -05:00
Joseph Schorr
c8229b9c8a
Implement new step-by-step setup
2015-01-23 17:19:15 -05:00
Jake Moshenko
44f7ab53a2
Merge remote-tracking branch 'origin/master' into ephemeral
2015-01-21 13:39:27 -05:00
Joseph Schorr
92d32bc636
Make the DB health check first attempt a simple DB connection. If the database is in the middle of a failover, this will fail after 3 seconds (the connection timeout specified), rather than hanging and causing the ELB health checks to timeout and fail.
2015-01-20 14:46:22 -05:00
Joseph Schorr
15a69ac872
Change robot deletions to set the performer to null, rather than attempting to delete the rows from the large log entries table
2015-01-14 12:56:06 -05:00
Joseph Schorr
bfd273d16f
- Make validation a bit nicer:
...
- Add timeout to the DB validation
- Make DB validation exception handling a bit nicer
- Move the DB validation error message
- Fix bug around RADOS config default for Is Secure
- Allow hiding of the validation box
2015-01-08 15:27:49 -05:00
Joseph Schorr
47fb10b79f
Merge branch 'master' into ackbar
2015-01-08 13:57:39 -05:00
Jimmy Zelinskie
b18986c72b
Continuation of baa4918d89
.
2015-01-06 15:11:37 -05:00
Jimmy Zelinskie
baa4918d89
Explicitly declare length of large MySQL indices.
...
Most MySQL installs silently truncate indices at a specific number of bytes.
This value is 767 bytes unless either the innodb_large_prefix option is turned
on or the innodb_page_size has been changed. This change explicitly limits the
size of large indices to 767 characters due to the latin1 charset being 1 byte
per char.
2015-01-06 14:55:31 -05:00
Joseph Schorr
1bf25f25c1
WIP
2015-01-04 14:38:41 -05:00
Jake Moshenko
cc70225043
Generalize the ephemeral build managers so that any manager may manage a builder spawned by any other manager.
2014-12-31 11:33:56 -05:00
Jimmy Zelinskie
e8cd24781a
remove 'include_starred' option on repo listings
2014-12-29 14:11:46 -05:00
Jimmy Zelinskie
84f6cc7385
add percona to migration tests
2014-12-23 18:34:02 -05:00
Jimmy Zelinskie
977782d047
Add MariaDB to migration.sh
2014-12-22 15:34:17 -05:00
Jimmy Zelinskie
eeeb2e620c
move slackwebhook migration from tools to util
...
tools isn't shipped inside of the container because it contains private
keys
2014-12-18 13:22:13 -05:00
Joseph Schorr
107847a4bb
Fix Slack notification setup to support the new slack web hook format and convert all existing data to use the new format (so we only have one code path)
2014-12-16 14:34:43 +02:00
Jimmy Zelinskie
5a484cfe11
Initial redesigned UI for repo listings w/ stars.
2014-12-11 15:07:41 -05:00
Joseph Schorr
6601e83285
When speaking to version 0.2-beta of the build worker, properly lookup the cached commands and see if we have a matching image/tag in the repository
2014-12-11 18:03:40 +02:00
Joseph Schorr
dbac8c7e3d
Fix build code:
...
- Fix issue with the queue_item in extend processing
- Add the new compiled docker binary with the lxc volume fix
2014-12-04 17:49:39 +01:00
Jimmy Zelinskie
97b605ca8d
Fix starring after rebase.
2014-12-03 12:20:51 -08:00
Jimmy Zelinskie
bd9f529e38
Add missing Star import.
2014-12-02 17:31:55 -08:00
Jimmy Zelinskie
08d84298e2
Fix indentation in star migration.
2014-12-02 17:31:21 -08:00
Jimmy Zelinskie
eb956e5b7d
initial work on adding models for starring repos.
...
I'm sick of using `git stash`.
2014-12-02 17:31:21 -08:00
Jimmy Zelinskie
f3259c862b
Merge branch 'koh'
...
Conflicts:
auth/scopes.py
requirements-nover.txt
requirements.txt
static/css/quay.css
static/directives/namespace-selector.html
static/js/app.js
static/partials/manage-application.html
templates/oauthorize.html
2014-12-01 12:30:09 -08:00
Joseph Schorr
72d613614d
Merge branch 'bagger'
2014-12-01 12:48:59 -05:00
Joseph Schorr
7bf96c506f
Merge branch 'bees' into koh
2014-11-24 19:25:53 -05:00
Joseph Schorr
e9cac407df
Add a configurable avatar system and add an internal avatar system for enterprise
2014-11-24 19:25:13 -05:00
Jimmy Zelinskie
716d7a737b
Strip whitespace from ALL the things.
2014-11-24 16:07:38 -05:00
Joseph Schorr
b8e873b00b
Add support to the build system for tracking if/when the build manager crashes and make sure builds are restarted within a few minutes
2014-11-21 14:27:06 -05:00
Jimmy Zelinskie
c918d15979
Add a migration for backfilling user UUIDs.
2014-11-20 18:28:54 -05:00
Jake Moshenko
2b8c246476
Temporarily put user rename behind a feature flag. Switch queue names back to using the username for namespace while we figure out a real migration strategy.
2014-11-20 15:36:39 -05:00
Jimmy Zelinskie
e6742b00b0
Remove backfill from UUID migration
2014-11-20 14:04:12 -05:00
Joseph Schorr
8fab3b6d34
Make sure to switch the context back to LogEntry and make sure to only include the performer if actually present
2014-11-20 12:04:53 -05:00
Joseph Schorr
cc7bccea4c
Add the logentrykind and performer information to log listing. This makes the system much faster, since new queries are not needed for every log entry.
2014-11-20 12:03:16 -05:00
Joseph Schorr
9b31b9805a
Fix performance problem with looking up org members and add some tests
2014-11-20 11:33:42 -05:00
Jake Moshenko
ac0d0ce36a
Change the order of migrations to allow migration code to work with the updated model.
2014-11-19 15:58:47 -05:00
Jake Moshenko
e863b96166
Tweak the uuid backfill to leave the uuid column nullable.
2014-11-19 15:32:30 -05:00
Jimmy Zelinskie
7c8a438b58
Rebase migration on top of master and add backfill.
2014-11-19 14:07:54 -05:00
Jimmy Zelinskie
606ad21bec
Apply reviewed changes.
...
Adds a length to the UUID field, renames QuayDeferredPermissionUser
parameter id->uuid, adds transactions to backfill script.
2014-11-19 13:28:16 -05:00
Jimmy Zelinskie
9d677b8eb3
Add UUID to User model and use in cookie.
2014-11-19 13:28:16 -05:00
Jake Moshenko
17fc72d262
Switch postgres to a non-transactional DDL to allow us to use peewee to modify data in migrations: enterprise customers are running postgres migrations offline already. Move the image backfill script back to a migration since it will now work. Unify the interface to sending a DB URI to env.py for the migration script.
2014-11-18 14:07:33 -05:00
Jake Moshenko
a7bae6c1d9
Fix robot renaming. Allow for trigger URLs to contain or omit the repository in the path. Fix calls to get_trigger to remove the namespace and repository.
2014-11-18 10:24:48 -05:00
Jake Moshenko
f4681f2c18
Merge branch 'master' into nomenclature
...
Conflicts:
test/data/test.db
2014-11-17 17:59:59 -05:00
Joseph Schorr
58ca76239b
Add ability to one-click generate an authorization access token in the applications panel
2014-11-17 14:54:07 -05:00
Joseph Schorr
178c5a7ac0
Add an index to the docker_image_id for faster lookup
2014-11-13 12:51:50 -05:00
Joseph Schorr
d73747ce1d
- Fix some other group_by clauses
...
- Fix garbage_collect for an empty list (fixes a test)
- Add a script which runs the full test suite against mysql and postgres (note: QueueTest's are broken for MySQL, but they obviously work in production, so they need to be fixed)
2014-11-13 12:51:37 -05:00
Joseph Schorr
a2e2dcb010
Fix group_by clauses for postgres
2014-11-12 20:32:06 -05:00
Joseph Schorr
079289c53c
HACK: Don't join on the visibility table because it is horrendously slow for some odd reason.
2014-11-12 14:51:24 -05:00
Joseph Schorr
eddcc02ea6
Make repository deletes much faster by adding custom deletion code and have additional tests to verify the deletion code paths
2014-11-10 23:05:20 -05:00
Joseph Schorr
e7cbda86f7
Merge branch 'perf'
2014-11-10 21:52:27 -05:00
Joseph Schorr
091f821a6a
- Rename get_repo_image to get_repo_image_extended and get_repo_image_directly to get_repo_image
...
- Remove the configure call from CloseForLongOperation
- Other small fixes
2014-11-10 13:44:36 -05:00
Joseph Schorr
72fedef097
Move the robot deletion code into a delete_instance method on the User object
2014-11-10 13:18:17 -05:00
Jake Moshenko
ca435fc7a6
Rename robots when we rename a user. Do not use the namespace from the path to check permissions from the incoming webhooks since the namespace may have changed and we cannot recreate them in remote services easily.
2014-11-09 17:50:57 -05:00
Joseph Schorr
17f605a9ef
Select only a single token.
2014-11-09 15:50:50 -05:00
Joseph Schorr
691be49817
Fix issues with the perf updated code
2014-11-07 14:36:32 -05:00
Joseph Schorr
158acd4f41
- Turn on foreign key constraint checking in the tests
...
- Change all ForeignKeyField's that refer to users to use our custom class, and mark those that allow robots
- Change robot delete to only execute the subset of queries necessary to actually delete robots
2014-11-07 12:05:21 -05:00
Joseph Schorr
d5bbb57481
Change registry code to disconnect from the DB before long I/O operations
2014-11-06 18:00:52 -05:00
Joseph Schorr
23d9bd2b42
Change verbs to use short lived database connections
2014-11-06 17:50:48 -05:00
Joseph Schorr
c569299e5c
Database optimizations around image creation and logs lookup
2014-11-06 14:48:16 -05:00
Joseph Schorr
a35bc11912
Add perf comments
2014-11-05 12:27:38 -05:00
Joseph Schorr
29c30b336e
Move the uncompressed image size migration call outside of alembic, since it will sometimes deadlock with certain kinds of DBs (because alembic is running things inside a transaction)
2014-11-03 21:01:41 -05:00
Joseph Schorr
9aa72c5cc2
Fix migration issues:
...
- MySQL 5.5 doesn't support the now() call as a default
- Postgres migration isn't auto-committed, so we have to check if the table exists first
2014-11-03 15:25:55 -05:00
Joseph Schorr
4eedd54b66
- Make usage language more accurate by stating "repositories"
...
- Have usage counter be based on a 4 weeks TTL
- Add a simple usage counter breakage test
2014-10-30 13:26:02 -04:00
Joseph Schorr
c1398c6d2b
- Add a log entry for repo verb handling and make the container usage calculation take it into account
...
- Move all the repo push/pull/verb logging into a central track_and_log method
- Readd images accidentally deleted in the last CL
- Make the uncompressed size migration script better handle exceptions
2014-10-29 15:42:44 -04:00
Joseph Schorr
e9c2e54dbc
Change delete to once again not perform everything under a transaction. A recent revision accidentally moved everything back under it.
2014-10-29 11:59:02 -04:00
Joseph Schorr
109850b428
Add a basic usage counter for enterprise
2014-10-28 16:33:13 -04:00
Joseph Schorr
c06f57a6e7
Make sure builders close the db handle when no work comes in and make the metrics transaction smaller in scope
2014-10-24 11:40:02 -04:00
Jake Moshenko
1461310ab8
Merge remote-tracking branch 'origin/master' into nomenclature
...
Conflicts:
endpoints/common.py
endpoints/notificationhelper.py
test/data/test.db
workers/dockerfilebuild.py
2014-10-23 13:25:37 -04:00
Joseph Schorr
5db9cd948b
Add better (jinja-based) messaging to the notifications and add some fixes for the email templates
2014-10-22 19:01:56 -04:00
Joseph Schorr
28a463f998
Make the 'created' field optional, in case the user does not specify it via the API
2014-10-20 12:07:49 -04:00
Jake Moshenko
dc5ee43a3a
MySQL still doesn't allow us to have empty IN clauses.
2014-10-17 17:49:18 -04:00
Jake Moshenko
6ca0115b5e
MySQL doesnt allow us to use the table we are modifying in a subquery.
2014-10-17 17:01:55 -04:00
Jake Moshenko
efe8825a15
We can't even use an empty in query for images.
2014-10-17 16:37:05 -04:00
Jake Moshenko
acd5a1e07e
Only garbage collect storage when it might result in something. Return the count of the removed images to fulfill the contract expected by some callers.
2014-10-17 16:13:32 -04:00
Jake Moshenko
18a944427f
Merge branch 'master' of bitbucket.org:yackob03/quay
2014-10-17 15:26:58 -04:00
Jake Moshenko
c093e5a326
Add a whitelist of candidate storages which will speed up the orphan queries and limit the damage of GC run amok.
2014-10-17 15:26:51 -04:00
Joseph Schorr
26a9bb50d5
Fix spelling mistake
2014-10-17 14:44:17 -04:00
Jake Moshenko
d8149295ab
Update the GC code to do everything with subqueries, making each GC run a bounded finite number of queries with a fixed length.
2014-10-17 14:33:04 -04:00
Jake Moshenko
380eb49e58
Split out the GC code into smaller transactions. Remove from the actual storage component after the fact, outside of the transactions.
2014-10-17 11:42:09 -04:00
Jake Moshenko
fa6a06502d
Change the default redis host to localhost. Fix some whitespace issues in the userevents module.
2014-10-14 14:37:02 -04:00
Jake Moshenko
328db8b660
Split the app into separate backends, which can use different worker types and different timeouts.
2014-10-14 13:58:08 -04:00
Joseph Schorr
bb9502ee77
Make sure to raise the exception for WSGI and make the delete method's intent more clear
2014-10-08 13:51:50 -04:00
Joseph Schorr
d16fdde528
Fix bug in dockerloadformat and make sure we handle exceptions properly in the verb call
2014-10-08 13:43:12 -04:00
Jake Moshenko
c34a8b6727
Fix the squashed image migration to insert the proper image transformation type.
2014-10-08 11:47:02 -04:00
Jake Moshenko
1a8287af55
Missed a table import.
2014-10-07 17:33:20 -04:00
Jake Moshenko
bbcdf18fdd
Update the migration to reflect the state of the database.
2014-10-07 16:19:05 -04:00
Jake Moshenko
1968c67cba
Merge remote-tracking branch 'origin/laffa' into nomenclature
2014-10-07 16:09:45 -04:00
Jake Moshenko
626d984fb2
Fixes to the migration generator script.
2014-10-07 16:09:30 -04:00
Jake Moshenko
fe59ad4fb5
Rectify our prod database, previous migrations, and database definition files.
2014-10-07 16:03:57 -04:00
Jake Moshenko
c77a3dc6ea
Update the migration for translating queue names to take into account previous migrations.
2014-10-07 15:57:17 -04:00
Jake Moshenko
a0d94f9d59
Merge remote-tracking branch 'origin/laffa' into nomenclature
...
Conflicts:
test/data/test.db
2014-10-07 15:56:28 -04:00
Jake Moshenko
ee9973a395
Merge remote-tracking branch 'origin/master' into nomenclature
2014-10-07 15:43:59 -04:00
Joseph Schorr
f4daa5e97b
- Update the migrations tool to verify migrations work up and down for both MySQL and PostgresSQL.
...
- Add migrations for the squashed image tables and for backfilling the uncompressed sizes
- Make sure gzip stream uses a max length when determining the uncompressed size
2014-10-07 15:29:56 -04:00
Joseph Schorr
f38ce51943
Merge master into laffa
2014-10-07 14:03:17 -04:00
Jake Moshenko
4ad592e7ce
Add an index to the image storage uuid to improve performance.
2014-10-06 18:44:37 -04:00
Jake Moshenko
c4266140e2
Fix all of the upgrades and downgrades to work on both mysql and postgres.
2014-10-06 16:41:49 -04:00
Jake Moshenko
ed8bcff39e
Merge remote-tracking branch 'origin/master' into nomenclature
...
Conflicts:
test/data/test.db
workers/dockerfilebuild.py
2014-10-06 10:29:39 -04:00
Jake Moshenko
5c18ffe67d
Allow the namespace column to be null, and also non-unique. Fix the uncompressed size clobbering the size on the wire field. Add metadata constraints so that foreign key constraints get predictable names. Fix all downgrade migrations.
2014-10-02 10:46:20 -04:00
Jake Moshenko
e8b3d1cc4a
Phase 4 of the namespace to user migration: actually remove the column from the db and remove the dependence on serialized namespaces in the workers and queues
2014-10-01 14:23:46 -04:00
Joseph Schorr
2c5cc7990f
Allow for additional REDIS config such as password and port
2014-10-01 14:16:42 -04:00
Joseph Schorr
6c520b8b0b
Fixed broken query
2014-09-30 13:19:32 -04:00
Joseph Schorr
5f776771d1
Merge branch 'master' of https://bitbucket.org/yackob03/quay
2014-09-29 17:01:07 -04:00
Joseph Schorr
474add0fb1
Have the layer PUT method calculate the uncompressed size in realtime, as trusting the JSON is fraught with complications
2014-09-29 17:00:47 -04:00
Jake Moshenko
fbc7ee3bce
Phase 3 of the namespace migration, remove write references to the namespace string column.
2014-09-29 15:44:25 -04:00
Jake Moshenko
ae9026ee91
MySQL will not allow us to revoke the possibility for the namespace_user column to be nullable.
2014-09-26 14:50:26 -04:00
Jake Moshenko
21d8c41320
Add the existing types to the alter column to make MySQL happy.
2014-09-26 12:58:03 -04:00
Jake Moshenko
e9745dbc96
Fix the SQL migration.
2014-09-26 11:17:42 -04:00
Jake Moshenko
03190efde3
Phase 2 of migrating repo namespaces to referencing user objects, backfilling the rows without a value for namespace_user, and changing all accesses to go through the namespace_user object. All tests are passing, manual testing still required.
2014-09-24 18:01:35 -04:00
Jake Moshenko
98d8e0fe37
Fixes for the uncompressed size backfill script.
2014-09-23 16:06:59 -04:00
Joseph Schorr
ec484e3efc
Move size and checksum updates into the metadata call
2014-09-23 15:49:28 -04:00
Jake Moshenko
74e35f917e
Switch socket_timeout to socket_connect_timeout for redis.
2014-09-23 14:59:00 -04:00
Jake Moshenko
798c13aec8
Fix the migration to use the proper column name that peewee expects.
2014-09-23 14:20:15 -04:00
Jake Moshenko
b6d635b648
Fix the migration to use the proper base.
2014-09-23 14:04:20 -04:00
Joseph Schorr
86dfca2e3e
Add uncompressed size field to the image storage and add a backfill script (which is not yet automatically called)
2014-09-23 14:01:27 -04:00
Joseph Schorr
87bc37f6c8
Merge branch 'sunday'
2014-09-23 11:29:03 -04:00
Joseph Schorr
ba0963a81c
Update the worker code to better handle exceptions, fix the utcdate issue and make sure we send the proper retry. Also updates notification workers to send JobExceptions rather than returning true or false
2014-09-23 11:01:50 -04:00
Joseph Schorr
f3b03ebc34
Add a feature flag for disabling all emails
2014-09-22 19:11:48 -04:00
Jake Moshenko
c68d49dfce
Fix the get_matching_users query to work with peewee 2.3+
2014-09-22 18:42:48 -04:00
Jake Moshenko
a6225ad34a
Set an aggressive timeout on the redis connections to allow the unit tests to run when communication with redis is disabled.
2014-09-22 18:42:22 -04:00
Jake Moshenko
3259cda000
The new strategy is to do a three phase migration. This is the first phase: getting the namespace user in the db and written for all new repositories.
2014-09-22 17:27:02 -04:00
Joseph Schorr
297c8ad29c
Add migration to backfill uncompressed image sizes on the storage
2014-09-22 15:04:28 -04:00
Joseph Schorr
1658475ac1
Previous revision should not have the image storage col in it
2014-09-22 14:39:44 -04:00
Joseph Schorr
f16878cce9
Add migration for synthetic image tables
2014-09-22 14:38:42 -04:00
Joseph Schorr
70e0aba257
Add a script for generating schema migrations. Should be run from the root quay directory.
2014-09-22 14:36:52 -04:00
Joseph Schorr
f23038c6ee
Update the worker code to better handle exceptions, fix the utcdate issue and make sure we send the proper retry. Also updates notification workers to send JobExceptions rather than returning true or false
2014-09-22 12:52:57 -04:00
Jake Moshenko
8626d1cd70
Initial changes to move repositories from using a namespace string to referencing a user object. Also stores the user id in the cookie rather than the username, to allow users to be renamed. This commit must not be used unmodified because the database migration is too aggressive for live migration.
2014-09-19 10:17:23 -04:00
Jake Moshenko
8c00eabedd
Merge remote-tracking branch 'origin/better-emails'
2014-09-19 10:04:02 -04:00
Joseph Schorr
6b05b55225
Add unicode password support
2014-09-18 17:36:26 -04:00
Jake Moshenko
11bb8e6448
Actually store the generated image storage in the database, and allow it to be garbage collected when the parent image storage is collected.
2014-09-18 17:26:40 -04:00
Joseph Schorr
b212dbb2ab
Merge branch 'master' into better-emails
2014-09-18 13:20:32 -04:00