vbatts/quay
Archived
1
0
Fork 0
Code Issues Pull requests Releases Wiki Activity
3945 commits 2 branches 62 tags 205 MiB
Commit graph

729 commits

Author SHA1 Message Date
josephschorr
ee53c04a45 Merge pull request #309 from coreos-inc/fasterqueue 
Improve the performance of queue candidate queries.
 2015-08-04 18:24:28 -04:00
Jake Moshenko
ed62339f89 Improve the performance of queue candidate queries. 2015-08-04 18:20:54 -04:00
Joseph Schorr
9f2d6282bd Add missing index on retries_remaining 2015-08-04 18:01:28 -04:00
josephschorr
f772bd0c9e Merge pull request #300 from coreos-inc/toomanyutils 
Refactor the util directory to use subpackages.
 2015-08-03 16:18:55 -04:00
Jake Moshenko
18100be481 Refactor the util directory to use subpackages. 2015-08-03 16:04:19 -04:00
Jimmy Zelinskie
8e6a0fbbee Merge pull request #294 from coreos-inc/logsload 
Switch to using an aggregated logs query and infinite scrolling
 2015-08-03 14:52:04 -04:00
Joseph Schorr
3d6c92901c Switch to using an aggregated logs query and infinite scrolling 
This should allow users to work with large logs set.

Fixes #294
 2015-07-31 16:38:02 -04:00
Jake Moshenko
e133ea0962 Try not to throw any sets of data away when computing images to garbage collect. 2015-07-31 16:12:57 -04:00
Joseph Schorr
0fdc8b0f1f Fix spelling of ancestors 2015-07-28 15:30:04 -04:00
Joseph Schorr
ba7686af99 Switch back to the read-then-write tag deletion code 
We changed to this originally to avoid locks
 2015-07-28 15:30:04 -04:00
Joseph Schorr
70de107268 Make GC of repositories fully async for whitelisted namespaces 
This change adds a worker to conduct GC on repositories with garbage every 10s.

Fixes #144
 2015-07-28 15:30:04 -04:00
Joseph Schorr
acd86008c8 Switch tag deletion to use a single query 2015-07-28 15:30:04 -04:00
Joseph Schorr
378c83598d Fix subquery issues in MySQL 2015-07-28 15:28:00 -04:00
Joseph Schorr
66b3d45fbc Remove legacy.py that was misadded 2015-07-27 15:53:25 -04:00
Joseph Schorr
c3f269ee23 Add migration for BitBucket web hooks 
This needs to added only *after* we roll out #255
 2015-07-23 14:45:12 -04:00
Joseph Schorr
ac1b46e7ec Add missing migration 2015-07-22 16:19:10 -04:00
Joseph Schorr
687bab1c05 Support invite codes for verification of email 
Also changes the system so we don't apply the invite until it is called explicitly from the frontend

Fixes #241
 2015-07-22 13:41:27 -04:00
Jake Moshenko
5d86fa80e7 Merge pull request #197 from coreos-inc/keystone 
Add Keystone Auth
 2015-07-22 13:38:47 -04:00
Joseph Schorr
38a6b3621c Automatically link the superuser account to federated service for auth 
When the user commits the configuration, if they have chosen a non-DB auth system, we now auto-link the superuser account to that auth system, to ensure they can login again after restart.
 2015-07-22 13:37:23 -04:00
Joseph Schorr
a0c4e72f13 Clean up the repository list API and loads stars with it 
We load stars with the same list API now so that we get the extra metadata needed in the repo list (popularity and last modified)
 2015-07-22 13:05:02 -04:00
Joseph Schorr
7e4b23916a Small SQL query fix 
Fixes #248
 2015-07-20 14:17:26 -04:00
Joseph Schorr
33b54218cc Refactor the users class into their own files, add a common base class for federated users and add a verify_credentials method which only does the verification, without the linking. We use this in the superuser verification pass 2015-07-20 11:39:59 -04:00
Joseph Schorr
1245385808 Fix typo 2015-07-20 10:55:21 -04:00
Joseph Schorr
066637f496 Basic Keystone Auth support 
Note: This has been verified as working by the end customer
 2015-07-20 10:55:21 -04:00
Jake Moshenko
bc29561f8f Fix and templatize the logic for external JWT AuthN and registry v2 Auth. 
Make it explicit that the registry-v2 stuff is not ready for prime time.
 2015-07-17 11:56:15 -04:00
Jake Moshenko
3efaa255e8 Accidental refactor, split out legacy.py into separate sumodules and update all call sites. 2015-07-17 11:56:15 -04:00
Jake Moshenko
bea8b9ac53 More changes for registry-v2 in python. 
Implement the minimal changes to the local filesystem storage driver and feed them through the distributed storage driver.
Create a digest package which contains digest_tools and checksums.
Fix the tests to use the new v1 endpoint locations.
Fix repository.delete_instance to properly filter the generated queries to avoid most subquery deletes, but still generate them when not explicitly filtered.
 2015-07-17 11:50:41 -04:00
Jake Moshenko
acbcc2e206 Start of a v2 API. 2015-07-17 11:50:41 -04:00
Joseph Schorr
7a548ea101 Fix queries for repository list popularity and action count 
Before this change, we used extremely inefficient outer joins as part of a single query of lookup, which was spiking our CPU usage to nearly 100% on the query. We now issue two separate queries for popularity and action account, by doing a lookup of the previously found IDs. Interestingly enough, because of the way the queries are now written, MySQL can actually do both queries *directly from the indicies*, which means they each occur in approx 20ms!

Verified by local tests, postgres tests, and testing on staging with monitoring of our CPU usage during lookup
 2015-07-17 00:08:27 +03:00
Jimmy Zelinskie
2869e2a6ea model: add missing params to validate_database_url 2015-07-15 17:39:26 -04:00
Jake Moshenko
eec7886e01 Add a server default for the broken migration. 2015-07-14 16:58:58 -04:00
Jimmy Zelinskie
bde781c98b Merge pull request #205 from coreos-inc/delrobot 
Fix deletion of robot accounts when attached to builds
 2015-07-13 12:19:01 -04:00
Joseph Schorr
3a59c99b08 Add a secondary tab to Teams for managing org members 
Also adds the ability to completely remove a user from an organization (repo permissions and teams), in a single click

Fixes #212
 2015-07-02 17:06:36 +03:00
Joseph Schorr
b535e222b8 Have the fetch tag dialog show a warning for robot accounts without access 
Before this change, we'd show the squash pulling command with the proper credentials, but it then 403s on the end user.
 2015-07-01 19:37:52 +03:00
Joseph Schorr
3ba321934f Fix deletion of robot accounts when attached to builds 
Fixes #204
 2015-06-30 22:56:44 +03:00
josephschorr
7aeaf2344e Merge pull request #200 from coreos-inc/tagapilimit 
Add pagination support to tag history API
 2015-06-30 22:09:09 +03:00
Joseph Schorr
f7f10f4a6d Add pagination support to tag history API 
Fixes #198
 2015-06-30 19:44:43 +03:00
Jake Moshenko
38a5963afe Merge pull request #190 from coreos-inc/timezone 
Fromtimestamp needs to be in UTC for JWT auth
 2015-06-30 12:05:00 -04:00
Joseph Schorr
2b1bbcb579 Add a table view to the repos list page 
Fixes #104
 2015-06-29 21:12:53 +03:00
Jimmy Zelinskie
756d6784ca Merge pull request #192 from coreos-inc/sqlssl 
Allow SSL cert for the database to be configured
 2015-06-29 13:33:31 -04:00
Joseph Schorr
dc5af7496c Allow superusers to disable user accounts 2015-06-29 18:40:52 +03:00
Joseph Schorr
bb07d0965f Allow SSL cert for the database to be configured 
This change adds a field for the SSL cert for the database in the setup tool. Fixes #89
 2015-06-29 08:08:10 +03:00
Joseph Schorr
477e244eff Fromtimestamp needs to be in UTC for JWT auth 2015-06-28 11:37:09 +03:00
Joseph Schorr
e7915baf8c Have LDAP return a better error message if it fails to connect 
Currently, the error results in a 500 being raised when a user tries to login.
 2015-06-23 17:41:53 -04:00
Joseph Schorr
07439328a4 Remove user_exists endpoint from all auth systems 2015-06-23 17:33:51 -04:00
Jake Moshenko
b21a033ef3 Merge pull request #131 from coreos-inc/moveapp 
Refactor JWT auth to not import app locally
 2015-06-23 17:24:01 -04:00
Jake Moshenko
5f1d23c6e8 Use a UNION query instead of a multitude of left outer joins for performance reasons. 
Fixes #159
 2015-06-23 17:18:37 -04:00
Joseph Schorr
331c300893 Refactor JWT auth to not import app locally 2015-06-17 15:53:21 -04:00
Joseph Schorr
e7fa560787 Add support for custom fields in billing invoices 
Customers (especially in Europe) need the ability to add Tax IDs, VAT IDs, and other custom fields to their invoices.

Fixes #106
 2015-06-12 16:45:01 -04:00
Jake Moshenko
79f1181a63 Switch build-scheduled to an official build phase. 2015-06-10 16:19:51 -04:00
Jake Moshenko
e09d84b3c8 Merge pull request #55 from coreos-inc/oauthdeny 
Fix OAuth redirect for denial action when generating for internal tokens
 2015-06-05 14:00:16 -04:00
Joseph Schorr
8aac3fd86e Add support for an external JWT-based authentication system 
This authentication system hits two HTTP endpoints to check and verify the existence of users:

Existance endpoint:
GET http://endpoint/ with Authorization: Basic (username:) =>
    Returns 200 if the username/email exists, 4** otherwise

Verification endpoint:
GET http://endpoint/ with Authorization: Basic (username:password) =>
    Returns 200 and a signed JWT with the user's username and email address if the username+password validates, 4** otherwise with the body containing an optional error message

The JWT produced by the endpoint must be issued with an issuer matching that configured in the config.yaml, and the audience must be "quay.io/jwtauthn". The JWT is signed using a private key and then validated on the Quay.io side with the associated public key, found as "jwt-authn.cert" in the conf/stack directory.
 2015-06-05 13:20:10 -04:00
Jake Moshenko
42da017d69 Merge pull request #48 from coreos-inc/nobots 
Change API calls that expect non-robots to explicitly filter
 2015-06-02 12:31:19 -04:00
Joseph Schorr
5516911de9 Fix OAuth redirect for denial action when generating for internal tokens 2015-06-02 12:25:59 -04:00
Joseph Schorr
2a56790d38 Switch to using a named LDAP tuple for more readable code 2015-06-01 14:02:05 -04:00
Joseph Schorr
1aff701bc7 Fix LDAP referral and multiple pair handling 
Fixes two issues found with our LDAP handling code. First, we now follow referrals in both LDAP calls, as some LDAP systems will return a referral instead of the original record. Second, we now make sure to handle multiple search result pairs properly by further filtering based on the presence of the 'mail' attribute when we have multiple valid pairs. This CL also adds tests for all of the above cases.
 2015-05-27 15:04:34 -04:00
Joseph Schorr
fdd43e2490 Change API calls that expect non-robots to explicitly filter 
Before this change, we'd filter in the UI but calls to the API could allow robots accounts where we only expect real users
 2015-05-26 17:47:33 -04:00
Joseph Schorr
b0d763b5ff Fix encrypted password generator to use the LDAP username, not the Quay username. 
Currently, we use the Quay username via `verify_user` when we go to create the encrypted password. This is only correct if Quay has not generated its own different username for the LDAP user, and fails if it has. We therefore add a new method `confirm_existing_user`, which looks up the federated login for the LDAP user and then runs the auth flow using that username.
 2015-05-20 16:37:09 -04:00
Joseph Schorr
d5e70c6e2a Explicitly enable LDAP referrals 
Note: The mock LDAP system doesn't support referrals, so we can't add a unit test for this.
 2015-05-20 14:53:31 -04:00
Jimmy Zelinskie
3daa0e6a52 migration: use SA boolean value for server_default 2015-05-15 19:04:54 -04:00
Jimmy Zelinskie
e12d5af52c migration: server_default for boolean field 2015-05-15 17:04:50 -04:00
Joseph Schorr
6a52ffa942 Add missing default value on the new reversion column 2015-05-13 13:49:21 -04:00
Joseph Schorr
07b4fb9105 LDAP sometimes has multiple records for a user 2015-05-12 12:02:09 -04:00
Joseph Schorr
efab02ae47 LDAP improvements: 
- Better logging
  - Better error messages
  - Add unit tests
  - Clean up the setup tool for LDAP
 2015-05-11 21:23:18 -04:00
Joseph Schorr
3e1abba284 Add ability for super users to rename and delete organizations 2015-05-11 18:03:25 -04:00
Joseph Schorr
1c41d34b7c Add ability for superusers to change user emails 2015-05-11 14:38:10 -04:00
Joseph Schorr
1c83def15b LDAP should only show logs when asked. 2015-05-11 13:01:49 -04:00
Joseph Schorr
f858caf6cd Only return the team and repo permissions when listing robots when we absolutely need them. 2015-05-08 16:43:07 -04:00
Joseph Schorr
561f2c7db0 Remove unnecessary sort 2015-05-08 15:10:31 -04:00
Joseph Schorr
3b93854c0b MySQL returns a decimal type for the tuple, which we need to convert to an int 2015-05-08 14:55:54 -04:00
Joseph Schorr
36f33e8fd7 Merge branch 'master' of github.com:coreos-inc/quay 2015-05-08 14:19:39 -04:00
Joseph Schorr
f9c1f123c2 Add better debugging to LDAP 2015-05-08 14:19:32 -04:00
Joseph Schorr
8ed8367404 PR changes in response to comments 2015-05-08 13:38:34 -04:00
Joseph Schorr
c767aafcd6 Make the repository API faster by only checking the log entries table once for each kind of entry, rather than twice. We make use of a special subquery-like syntax, which allows us to count those entries that are both 30 days only and 1 day old in the same query. This was tested successfully on MySQL, Postgres and Sqlite. 2015-05-07 22:49:11 -04:00
Joseph Schorr
3627de103c Minimize the queries used when retrieve builds. Previously, we'd call out to SQL extra times per build. 2015-05-07 21:11:15 -04:00
Joseph Schorr
7b35555776 Make sure to test for unicode usernames, since the collate on the username field is latin1 2015-05-07 18:13:45 -04:00
Joseph Schorr
729249f42b Need to add the tables to the .select() to prevent N+1, not merely to the join 2015-05-07 16:59:13 -04:00
Joseph Schorr
b50600cde8 Make sure to join the Repo table on its namespace user and its visibility so we don't issue additional queries from the starred repos API call 2015-05-07 16:18:17 -04:00
Joseph Schorr
3cec22defe Make sorting by repo ID, rather than description 2015-05-07 13:10:26 -04:00
Joseph Schorr
73193e2ab5 Remove sort by description, which is slow (and fairly useless anyway) 2015-05-06 20:55:31 -04:00
Joseph Schorr
c89760278f Add migration to add gitlab trigger type 2015-05-03 10:49:13 -07:00
Joseph Schorr
a59100b231 Add the missing index on the peewee side. We already have the associated migration. 2015-04-30 15:56:40 -04:00
Joseph Schorr
17bc9b81bf Add bitbucket trigger type 2015-04-30 15:55:57 -04:00
Joseph Schorr
b96e35b28c Merge master into bitbucket 2015-04-30 15:52:08 -04:00
Joseph Schorr
31260d50f5 Rename the new images method to a slightly better name 2015-04-24 16:37:37 -04:00
Joseph Schorr
e70343d849 Faster cache lookup by removing a join with the ImagePlacementTable, removing the extra loop to add the locations and filtering the images looked up by the base image 2015-04-24 16:22:19 -04:00
Jimmy Zelinskie
fd65ca5916 migration: add custom-git service to database 2015-04-24 16:11:58 -04:00
Joseph Schorr
c480fb2105 Work in progress: bitbucket support 2015-04-24 15:13:08 -04:00
Jimmy Zelinskie
d25cc4db9c migrations: fix percona migration test 2015-04-21 18:01:40 -04:00
Jimmy Zelinskie
93cd459460 Merge branch 'master' into git 2015-04-20 10:58:49 -04:00
Joseph Schorr
43ff6839b8 Hide hidden tags in the tags timeline 2015-04-19 18:12:06 -04:00
Joseph Schorr
ed342ae831 Add migration for properly creating the repository_id+datetime index 2015-04-19 16:03:06 -04:00
Joseph Schorr
d1e2d072ea Add unit tests and a stronger restriction on the revert API call 2015-04-19 15:43:16 -04:00
Joseph Schorr
a8f8c317f9 Fix branch in alembic migrations 2015-04-19 15:20:01 -04:00
Joseph Schorr
2815ea2723 Merge branch 'master' of github.com:coreos-inc/quay 2015-04-19 15:16:39 -04:00
Joseph Schorr
8714eb207a Merge branch 'actioncount' 2015-04-19 15:12:44 -04:00
Jimmy Zelinskie
4133924813 migrations: rebase on top of master's changes 2015-04-17 13:19:04 -04:00
Jimmy Zelinskie
ba2cb08904 Merge branch 'master' into git 2015-04-16 17:38:35 -04:00
Joseph Schorr
f19d2f684e Add ability to revert tags via time machine 2015-04-16 17:18:00 -04:00
Joseph Schorr
f8c80f7d11 Add a history view to the tags page. Next step will add the ability to revert back in time 2015-04-15 15:21:09 -04:00