vbatts/quay
Archived
1
0
Fork 0
Code Issues Pull requests Releases Wiki Activity
8869 commits 2 branches 62 tags 205 MiB
Commit graph

1252 commits

Author SHA1 Message Date
Joseph Schorr
555041876d Make public catalog only enabled via a feature flag 2017-06-02 15:30:51 -07:00
Kenny Lee Sin Cheong
203c0b76e0 Raise an APIRequestFailure exception when security scanner is unavailable 
Put worker to sleep for the duration of the default indexing interval
when an APIRequestFailure occurs, when the API request fails due to a
connection error, timeout, or other ambiguous errors, from
analyze_layer or get_layer_data .
 2017-05-24 11:04:44 -04:00
Charlton Austin
a71f60a9c1 Merge pull request #2652 from charltonaustin/failing_repository_notifications_to_be_disabled_after_n_failures_in_a_row_144646649 
Failing repository notifications to be disabled after n failures in a row 144646649
 2017-05-22 09:30:53 -04:00
Charlton Austin
993f2a174c feat(full-stack): disable notifications after 3 failures 
This stops notifications from firing over and over again if they are repeatedly failing.

[TESTING -> locally with docker compose, DATABASE MIGRATION -> there is a single migration]

Issue: https://www.pivotaltracker.com/story/show/b144646649n

- [ ] It works!
- [ ] Comments provide sufficient explanations for the next contributor
- [ ] Tests cover changes and corner cases
- [ ] Follows Quay syntax patterns and format
 2017-05-19 16:58:46 -04:00
Joseph Schorr
065e327190 Make sure catalog always returns public repositories 2017-05-19 16:24:08 -04:00
Jimmy Zelinskie
e2c25ce9bc registry tests: assert audit log metadata 2017-05-16 17:05:31 -04:00
Evan Cordell
85d382cd84 Only show signing UI when namespace is explicitly whitelisted 2017-05-08 13:50:07 -04:00
Joseph Schorr
30a681343f Make sure to escape LDAP queries 
Fixes an issue in team sync around group names that contain *s

Fixes https://www.pivotaltracker.com/story/show/144628235
 2017-05-01 14:00:54 -04:00
Joseph Schorr
de6352ef73 Make sure we don't allow anonymous binding in LDAP auth 
We already prevented it, but let's make sure we never allow it by adding some tests
 2017-04-28 17:09:47 -04:00
josephschorr
8b148bf1d4 Merge pull request #2576 from coreos-inc/full-db-tests-tox 
Reenable full database testing locally and in concourse
 2017-04-27 18:09:15 -04:00
EvB
b216290e31 test(endpoints/web): test buildlogs endpoint 2017-04-26 18:16:26 -04:00
josephschorr
5c4f7d50c6 Merge pull request #2580 from coreos-inc/team-sync-email-fix 
Fix handling of team sync when a user already exists with the email address
 2017-04-26 14:19:23 -04:00
EvB
5e995fae20 refactor(archivedlogs): move archivelog handler to endpoints 2017-04-26 11:41:55 -04:00
Joseph Schorr
36f2272fe2 Fix handling of team sync when a user already exists with the email address 2017-04-25 17:42:35 -04:00
Joseph Schorr
d895b4d5ff Fix appr tests to use the shared test fixtures 2017-04-24 23:08:28 -04:00
Joseph Schorr
d7f3ef96ce Small fixes found by running full db tests 2017-04-24 16:45:15 -04:00
Joseph Schorr
a1a4b68306 Change fulldbtests to use py.test 2017-04-24 16:45:15 -04:00
Joseph Schorr
6ba7ed4cd6 Prep test fixtures for supporting non-SQLite database 2017-04-24 16:45:14 -04:00
Joseph Schorr
3dcbe3c631 If enabled, allow users and orgs to set their time machine expiration 
Fixes https://www.pivotaltracker.com/story/show/142881203
 2017-04-21 11:32:45 -04:00
Joseph Schorr
95e9cdaccc Disable tag and build operations in repos when trust is enabled 2017-04-17 14:34:43 -04:00
Evan Cordell
2661db7485 Add flag to enable trust per repo (#2541) 
* Add flag to enable trust per repo

* Add api for enabling/disabling trust

* Add new LogEntryKind for changing repo trust settings
Also add tests for repo trust api

* Add `set_trust` method to repository

* Expose new logkind to UI

* Fix registry tests

* Rebase migrations and regen test.db

* Raise downstreamissue if trust metadata can't be removed

* Refactor change_repo_trust

* Add show_if to change_repo_trust endpoint
 2017-04-15 08:26:33 -04:00
Alec Merdler
581d7c67a7 Fix Trigger Setup Robot Permissions for Private Base (#2543) 2017-04-14 12:10:00 -07:00
Joseph Schorr
ab2f044331 Switch get repo API to use a single list tags query 
Should make things faster since the join occurs on the database side
 2017-04-13 18:06:58 -04:00
josephschorr
928b9915ed Merge pull request #2441 from coreos-inc/repo-score-denormalization 
Add a RepositorySearchScore table and calculation to the RAC worker
 2017-04-10 16:31:09 -04:00
Joseph Schorr
df3f47c79a Add a RepositorySearchScore table and calculation to the RAC worker 
This will be used in a followup PR to order search results instead of the RAC join. Currently, the join with the RAC table in search results in a lookup of ~600K rows, which causes searching to take ~6s. This PR denormalizes the data we need, as well as allowing us to score based on a wider band (6 months vs the current 1 week).
 2017-04-10 14:29:02 -04:00
josephschorr
2bc619137a Merge pull request #2512 from ecordell/tufmetadata 
Add tufmetadata endpoint
 2017-04-07 17:16:11 -04:00
Jake Moshenko
a8ec7865a7 Merge pull request #2511 from jakedt/fixwarnings 
Fixwarnings
 2017-04-06 16:12:19 -04:00
Jake Moshenko
c7241911a5 Fix old-style flask imports to silence deprecation warnings. 2017-04-06 13:15:48 -04:00
Evan Cordell
1a78722521 Update tests 2017-04-05 13:57:16 -04:00
Joseph Schorr
f9e6110f73 Add basic user interface for application repos 
Adds support for creating app repos, viewing app repos and seeing the list of app repos in the Quay UI.
 2017-04-05 11:30:09 -04:00
Evan Cordell
9515f18fb6 Add tufmetadata endpoint 2017-04-05 10:03:27 -04:00
Joseph Schorr
b26bd3c9c5 Regenerate test.db after merge 2017-04-03 11:36:42 -04:00
Joseph Schorr
8c07f733eb Add pagination tests for LDAP 2017-04-03 11:36:41 -04:00
Joseph Schorr
d7825c6720 Add group iteration and syncing support to Keystone auth 2017-04-03 11:31:30 -04:00
Joseph Schorr
04225f2d25 Add feature flag for team syncing 2017-04-03 11:31:29 -04:00
Joseph Schorr
96b9d6b0cd Add end-to-end test for team sync 2017-04-03 11:31:29 -04:00
Joseph Schorr
b683088f87 Update tests for teams API 2017-04-03 11:31:29 -04:00
Joseph Schorr
a17b637032 Fix ordering in LDAP test 2017-04-03 11:31:28 -04:00
Joseph Schorr
bb20422260 Fix pagination disabling in LDAP with mockldap 
Since mockldap doesn't support pagination, just disable it globally
 2017-04-03 11:31:28 -04:00
Joseph Schorr
ecfac81721 Add check_group_lookup_args and service_metadata to auth providers 2017-04-03 11:31:28 -04:00
Joseph Schorr
f5a854c189 Add TeamSync database and API support 
Teams can now have a TeamSync entry in the database, indicating how they are synced via an external group. If found, then the user membership of the team cannot be changed via the API.
 2017-04-03 11:31:28 -04:00
Joseph Schorr
d718829f5d Initial LDAP group member iteration support 
Add interface for group member iteration on internal auth providers and implement support in the LDAP interface.
 2017-04-03 11:31:28 -04:00
Charlton Austin
ca99535774 Merge pull request #2449 from charltonaustin/phase_two_config 
feat(build runner): added in context, dockerfile_location
 2017-03-28 14:14:36 -04:00
Charlton Austin
e6d201e0b0 feat(build runner): added in context, dockerfile_location 
this is a new feature meant to allow people to use any file as
  a dockerfile and any folder as a context directory
 2017-03-28 13:55:31 -04:00
Jimmy Zelinskie
65a17dc155 Merge pull request #2473 from coreos-inc/certs-fixes 
Fixes and improvements around custom certificate handling
 2017-03-27 15:08:36 -04:00
Evan Cordell
4c94d2c760 Fix xAuth test 2017-03-27 11:37:17 -04:00
Evan Cordell
6ad107709c Change build_context_and_subject to take kwargs 2017-03-27 11:37:17 -04:00
Evan Cordell
43dd974dca Determine which TUF root to show based on actual access, not requested 
access
 2017-03-27 11:37:17 -04:00
Joseph Schorr
e509eb4cba Better custom cert handling in the superuser tool 
We now only allow certificates ending in .crt to be uploaded and we automatically install the certificate once it has been validated
 2017-03-24 17:15:26 -04:00
Joseph Schorr
651666b60b Refactor our auth handling code to be cleaner 
Breaks out the validation code from the auth context modification calls, makes decorators easier to define and adds testing for each individual piece. Will be the basis of better error messaging in the following change.
 2017-03-23 15:42:45 -04:00
Joseph Schorr
abf179eb09 Move fixtures under test, since they are shared globally 2017-03-23 15:42:45 -04:00
Joseph Schorr
e204f7784c Make app registry off by default 2017-03-23 12:01:59 -04:00
Joseph Schorr
bdda74d6df Make sure GC checks new Blob table as well before deleting CAS storage 2017-03-22 23:53:21 -04:00
Jimmy Zelinskie
3ccf3c5f33 Merge pull request #2447 from jzelinskie/cnr-step2 
CNR Step 2
 2017-03-22 18:45:51 -04:00
Joseph Schorr
df1e7f90e0 Add verb security tests and fix small issues 2017-03-22 18:29:53 -04:00
Joseph Schorr
dcb970b783 Add registry app repository failure test 2017-03-22 17:26:59 -04:00
Joseph Schorr
30b532254c Disallow non-apps-supported APIs for application repositories 2017-03-22 15:51:19 -04:00
Jimmy Zelinskie
dae2cc8c3a initdb: add beta class data 2017-03-22 13:58:50 -04:00
Joseph Schorr
a047257656 Transitive query checks need to be for updates, not just deletes 
MySQL doesn't support transitive updates either, so we need to extend the testing to prevent the recent breakage from occurring again
 2017-03-20 21:06:37 -04:00
Joseph Schorr
6476488221 Skip bitbucket pushes without any commits 
Fixes https://sentry.io/coreos/backend-production/issues/178220183/
 2017-03-20 18:23:21 -04:00
josephschorr
4bee4dbfff Merge pull request #2443 from coreos-inc/build-webhook-tests 
Add tests for build web hooks endpoint
 2017-03-20 16:26:57 -04:00
josephschorr
27aa12de7a Merge pull request #2439 from coreos-inc/remove_redis_log_expiration 
Switch from expire to delete redis log_entries
 2017-03-20 13:41:57 -04:00
Joseph Schorr
6f567e0850 Add tests for build web hooks endpoint 2017-03-20 13:22:59 -04:00
Jimmy Zelinskie
f94d5c7684 initdb: skip data checks for beta classes & regen 2017-03-17 11:35:16 -04:00
Antoine Legrand
ec847ce613 Switch from expire to delete redis log_entries 2017-03-17 15:35:47 +01:00
Alec Merdler
cd61482ff1 Merge pull request #2426 from alecmerdler/frontend-refactoring 
More Front-end TypeScript Refactoring
 2017-03-16 14:19:13 -07:00
Joseph Schorr
e90cab4d77 Change revert tag into restore tag and add manifest support 2017-03-14 11:34:42 -04:00
Joseph Schorr
af743b156b Show manifest digests in place of V1 ids in the tag view when possible 2017-03-14 11:34:41 -04:00
josephschorr
cbac673d58 Merge pull request #2404 from coreos-inc/cas-gc-fix 
Fix GC handling around CAS paths
 2017-03-10 17:34:21 -05:00
josephschorr
432b2d3fe8 Merge pull request #2392 from coreos-inc/search-optimization 
Optimize repository search by changing our lookup strategy
 2017-03-10 15:44:26 -05:00
Joseph Schorr
48db77b521 Fix bug in QSS notifications 2017-03-10 11:25:55 -05:00
Joseph Schorr
b5bb76cdea Optimize repository search by changing our lookup strategy 
Previous to this change, repositories were looked up unfiltered in six different queries, and then filtered using the permissions model, which issued a query per repository found, making search incredibly slow. Instead, we now lookup a chunk of repositories unfiltered and then filter them via a single query to the database. By layering the filtering on top of the lookup, each as queries, we can minimize the number of queries necessary, without (at the same time) using a super expensive join.

Other changes:
- Remove the 5 page pre-lookup on V1 search and simply return that there is one more page available, until there isn't. While technically not correct, it is much more efficient, and no one should be using pagination with V1 search anyway.
- Remove the lookup for repos without entries in the RAC table. Instead, we now add a new RAC entry when the repository is created for *the day before*, with count 0, so that it is immediately searchable
- Remove lookup of results with a matching namespace; these aren't very relevant anyway, and it overly complicates sorting
 2017-03-09 19:47:55 -05:00
alecmerdler
4e913f106d refactored DockerfileServiceImpl to return promise instead of callbacks 2017-03-09 01:26:19 -08:00
Joseph Schorr
0ab6388e30 Add support for null ref, as that can be the value if a default branch is not chosen 2017-03-07 20:39:42 -05:00
alecmerdler
80b3666eb7 refactoring DockerfileService 2017-03-07 11:25:18 -08:00
Joseph Schorr
69e550d125 Fix GC handling around CAS paths 
Adds code to ensure we never GC CAS paths that are shared amongst multiple ImageStorage rows, as well as an associated pair of tests to catch the positive and negative cases.
 2017-03-07 13:48:07 -05:00
Jimmy Zelinskie
b9ac2b7b3b workers.securityworker: simplify min id 2017-03-03 14:51:18 -05:00
Jimmy Zelinskie
4ed0cdda14 securityscanner: add a min image id option 
This will enable us to force some instances of the securityworker to
scan only new images.
 2017-03-03 13:55:25 -05:00
josephschorr
aa2f88d321 Merge pull request #2337 from coreos-inc/new-trigger-ux 
Implement new create and manager trigger UI
 2017-03-02 18:15:32 -05:00
Joseph Schorr
9e6c368f7a Make QSS multiple notification messaging nicer 2017-03-01 16:11:11 -05:00
Joseph Schorr
eff1827d9d Batch QSS notifications after initial scan 2017-03-01 15:42:49 -05:00
alecmerdler
8fcd76c0be removed old templates 2017-02-28 16:51:44 -05:00
alecmerdler
ea9d47ba75 ignore invalid linear workflow sections that are after the current section 2017-02-28 16:51:44 -05:00
alecmerdler
ff07533d80 added tests for linear workflow components 2017-02-28 16:51:44 -05:00
alecmerdler
e59d394491 refactoring linear workflow directives 2017-02-28 16:51:43 -05:00
alecmerdler
b1516193a5 converted ManageTriggerGithostComponent to TypeScript 2017-02-28 16:51:43 -05:00
alecmerdler
14222be9fe working on ManageTriggerGithostComponent 2017-02-28 16:51:43 -05:00
alecmerdler
00b1f0e3cc starting ManageTriggerCustomGitComponent 2017-02-28 16:51:43 -05:00
alecmerdler
389a4cb1c4 fixed tests 2017-02-28 16:51:42 -05:00
Joseph Schorr
8e863b8cf5 Implement new create and manager trigger UI 
Implements the new trigger setup user interface, which is now a linear workflow found on its own page, rather than a tiny modal dialog

Fixes #1187
 2017-02-28 16:51:42 -05:00
Joseph Schorr
f8d74305e1 Remove old validator tests 2017-02-24 12:23:18 -05:00
Joseph Schorr
c0f7530b29 Pull out JWT auth validation into validator class 
Also fixes a small bug in validation (yay tests!)
 2017-02-24 12:23:16 -05:00
Joseph Schorr
b2afe68632 Pull out redis validation into validator class 2017-02-24 12:23:15 -05:00
Joseph Schorr
f933b3e295 Pull out database validation into validator class 2017-02-24 12:23:14 -05:00
Joseph Schorr
94be8731f3 Change Docker Version tests to pytest 2017-02-22 15:45:06 -05:00
josephschorr
f7a7d30ec2 Merge pull request #2366 from coreos-inc/alert-spam-fixes 
Small fixes for alert spam
 2017-02-22 14:18:18 -05:00
Joseph Schorr
7cc7e54945 Remove unicode before sending it to path parser 
Fixes https://sentry.io/coreos/backend-production/issues/175929456/
 2017-02-22 13:21:12 -05:00
Jake Moshenko
b03e03c389 Read the number of unscanned clair images from the block allocator 2017-02-21 19:13:51 -05:00
Joseph Schorr
198bdf88bc Move OAuth login into its own endpoints module 2017-02-16 16:27:54 -05:00
Joseph Schorr
6736e69ebd Add end-to-end OIDC binding test 2017-02-16 16:27:53 -05:00
josephschorr
c8e5eb5ad1 Merge pull request #2359 from coreos-inc/fix-gitlab-tag 
Fix handling of gitlab web hooks when tagging
 2017-02-16 15:56:56 -05:00
Joseph Schorr
a34d18b9ea Fix handling of gitlab web hooks when tagging 
Gitlab doesn't send any commit information for tagging events (because... reasons), and so we have to perform the lookup ourselves to have full metadata.

Fixes #1467
 2017-02-16 15:40:37 -05:00
josephschorr
38e079ced2 Merge pull request #2344 from coreos-inc/v1-search-fix 
Implement the full spec for the old Docker V1 registry search API
 2017-02-16 15:08:33 -05:00
Joseph Schorr
a0bc0e9488 Implement the full spec for the old Docker V1 registry search API 
This API is still (apparently) being used by the Docker CLI for `docker search` (why?!) and we therefore have customers expecting this to work the same way as the DockerHub.
 2017-02-16 14:45:33 -05:00
josephschorr
8f01cb959a Merge pull request #2354 from coreos-inc/license-sorting 
Change entitlement sorting to sort *valid* entitlements by reverse expiration time
 2017-02-15 16:24:51 -05:00
Joseph Schorr
d506279892 Change entitlement sorting to sort *valid* entitlements by reverse expiration time 
With this change, if all entitlements are valid, we sort to show the entitlement that will expire the farthest in the future, as that defines the point at which the user must act before the license becomes invalid.
 2017-02-15 14:31:24 -05:00
Jimmy Zelinskie
ab941607d4 test.test_api_usage: add AppConfigChange ctxmgr 2017-02-14 15:27:22 -05:00
Jimmy Zelinskie
1d6339e644 test.test_api_usage: fix secscan tests 2017-02-14 15:21:18 -05:00
Jimmy Zelinskie
8a1b48dd8c move ConfigProvider ctxmgr back to su tests 2017-02-14 14:36:36 -05:00
Joseph Schorr
2f4487c184 Fix flaky OAuth tests under tor 
The `> 0` check fails if the code was found first in the query string, which can occasionally happen under tox due to the `PYTHONHASHSEED` var changing. We simply change to use a proper parse and check to avoid this issue entirely.
 2017-02-14 13:51:58 -05:00
Joseph Schorr
8d96d8b682 Add tests for missing logs APIs 2017-02-08 16:52:17 -08:00
Jimmy Zelinskie
c2c6bc1e90 test: add qss read failover case 2017-02-03 19:20:13 -05:00
Jimmy Zelinskie
dd033e4feb test: move ConfigForTesting 2017-02-03 19:20:13 -05:00
Charlton Austin
5a06530b43 Merge pull request #2314 from charltonaustin/move_tests_over_to_pytest_no_story 
update(security_test.py): moving tests to new framework
 2017-02-03 16:21:03 -05:00
Joseph Schorr
cf539487a1 Add API endpoint for retrieving security status by *manifest*, rather than Docker V1 image ID 2017-02-02 17:51:18 -05:00
Charlton Austin
85bcb63439 update(security_test.py): moving tests to new framework 
We should be moving tests over to pytest

[none]
 2017-02-02 13:40:00 -05:00
josephschorr
01ec22b362 Merge pull request #2300 from coreos-inc/openid-connect 
OpenID Connect support and OAuth login refactoring
 2017-01-31 18:14:44 -05:00
Joseph Schorr
973a110ac7 Full text search for repository name and description 
Adds support for searching full text against the name and description of a repository

[Delivers #134867401]
 2017-01-31 11:38:31 -05:00
Joseph Schorr
f5dbc350f8 Fix missed tests and revert conftest change (breaks docker build) 2017-01-30 17:28:25 -05:00
Joseph Schorr
d63cca025a DNS name check got reversed; breaks wildcards 2017-01-29 11:51:37 -05:00
Charlton Austin
dae93dce78 feature(superuser panel): ability to view logs 
users would like the ability to view build logs in the superuser panel

[None]
 2017-01-26 13:54:03 -05:00
Joseph Schorr
a9791ea419 Have external login always make an API request to get the authorization URL 
This makes the OIDC lookup lazy, ensuring that the rest of the registry and app continues working even if one OIDC provider goes down.
 2017-01-23 19:06:19 -05:00
Joseph Schorr
fda203e4d7 Add proper and tested OIDC support on the server 
Note that this will still not work on the client side; the followup CL for the client side is right after this one.
 2017-01-23 17:53:34 -05:00
Charlton Austin
7854bf6b3a Making test independent of message ordering. 2017-01-23 14:32:34 -05:00
Joseph Schorr
19f7acf575 Lay foundation for truly dynamic external logins 
Moves all the external login services into a set of classes that share as much code as possible. These services are then registered on both the client and server, allowing us in the followup change to dynamically register new handlers
 2017-01-20 15:21:08 -05:00
Joseph Schorr
4755d08677 Refactor and rename the standard OAuth services 2017-01-19 15:23:15 -05:00
Joseph Schorr
7c7a07fb5a Allow namespaces to be between 2 and 255 characters in length 
[Delivers #137924329]
 2017-01-19 13:10:26 -05:00
josephschorr
e2748fccd9 Merge pull request #2282 from coreos-inc/motd-updates 
Severity and Markdown support in MOTD
 2017-01-18 17:41:27 -05:00
Joseph Schorr
3106504f39 Severity and Markdown support in MOTD 
[Delivers #133555165]
 2017-01-18 16:55:32 -05:00
Joseph Schorr
b3a74b94b3 Fix flaky port selection in registry tests 
Depends on https://github.com/jarus/flask-testing/pull/98

[Fixes #136705135]
 2017-01-18 15:06:11 -05:00
Joseph Schorr
462f47924e More detailed namespace validation 
Fixes namespace validation to use the proper regex for checking length, as well as showing the proper messaging if the entered namespace is invalid

[Delivers #137830461]
 2017-01-17 17:31:59 -05:00
josephschorr
aafcb592a6 Merge pull request #2257 from coreos-inc/clair-gc-take2 
feat(gc): Garbage collection for security scanning
 2017-01-17 14:49:36 -05:00
Joseph Schorr
8c4e86f48b Change queue to use state-field for claiming items 
Before this change, the queue code would check that none of the fields on the item to be claimed had changed between the time when the item was selected and the item is claimed. While this is a safe approach, it also causes quite a bit of lock contention in MySQL, because InnoDB will take a lock on *any* rows examined by the `where` clause of the `update`, even if they will ultimately thrown out due to other clauses (See: http://dev.mysql.com/doc/refman/5.7/en/innodb-locks-set.html: "A ..., an UPDATE, ... generally set record locks on every index record that is scanned in the processing of the SQL statement. It does not matter whether there are WHERE conditions in the statement that would exclude the row. InnoDB does not remember the exact WHERE condition, but only knows which index ranges were scanned").

As a result, we want to minimize the number of fields accessed in the `where` clause on an update to the QueueItem row. To do so, we introduce a new `state_id` column, which is updated on *every change* to the QueueItem rows with a unique, random value. We can then have the queue item claiming code simply check that the `state_id` column has not changed between the retrieval and claiming steps. This minimizes the number of columns being checked to two (`id` and `state_id`), and thus, should significantly reduce lock contention. Note that we can not (yet) reduce to just a single `state_id` column (which should work in theory), because we need to maintain backwards compatibility with existing items in the QueueItem table, which will be given empty `state_id` values when the migration in this change runs.

Also adds a number of tests for other queue operations that we want to make sure operate correctly following this change.

[Delivers #133632501]
 2017-01-17 13:29:26 -05:00
Joseph Schorr
939c122f70 Complete item queue test 2017-01-17 13:26:09 -05:00
Joseph Schorr
dcfd379b17 Queue cancelation test 2017-01-17 13:26:09 -05:00
Charlton Austin
8ca8c17e27 Merge pull request #2225 from charltonaustin/adding_in_new_indices 
Adding in new indices for queueitem table.
 2017-01-17 11:46:51 -05:00
josephschorr
9b65b37011 Merge pull request #2245 from coreos-inc/recaptcha 
Add support for recaptcha during the create account flow
 2017-01-17 11:34:23 -05:00
josephschorr
eb2cafacd4 Merge pull request #2249 from coreos-inc/notifier-fixes 
Security notification pagination fix
 2017-01-17 11:33:25 -05:00
Charlton Austin
ca832df975 Adding in new indices for queueitem table. 2017-01-17 10:04:31 -05:00
Joseph Schorr
7e0fbeb625 Custom SSL certificates config panel 
Adds a new panel to the superuser config tool, for managing custom SSL certificates in the config bundle

[Delivers #135586525]
 2017-01-13 14:34:35 -05:00
Joseph Schorr
3a24871422 Add SSL certificate utility and tests 2017-01-10 17:06:13 -05:00
Joseph Schorr
3eb17b7caa Add support for recaptcha during the create account flow 
If the feature is enabled and recaptcha keys are given in config, then a recaptcha box is displayed in the UI when creating a user and a recaptcha response code *must* be sent with the create API call for it to succeed.
 2017-01-09 11:08:21 -05:00
Joseph Schorr
ce21788da8 test(queue): delete_namespaced_items 
Add queue tests for delete_namespaced_items
 2017-01-09 11:05:39 -05:00
Joseph Schorr
d609e6a1c4 Security scanner garbage collection support 
Adds support for calling GC in the security scanner for any layers+storage removed by GC on the Quay side
 2016-12-22 14:55:26 -05:00
Joseph Schorr
5225642850 Garbage collection image+storage callback support 
Add support to GC to invoke a callback with the image+storages removed. Only images whose storage was also removed will be sent to the callback. This will be used by security scanning for its own GC in the followup change.
 2016-12-22 14:27:42 -05:00
Joseph Schorr
ef80471a39 fix(136521333): Handle None email_or_id in avatar code 
Fixes https://www.pivotaltracker.com/story/show/136521333
 2016-12-21 15:00:55 -05:00
josephschorr
732ab67b57 Merge pull request #2252 from coreos-inc/parallel-tests 
Fix pytests and enable parallel registry tests
 2016-12-20 16:56:52 -05:00
Joseph Schorr
481cebe46b Fix pytests and enable parallel registry tests 2016-12-20 15:42:04 -05:00
Joseph Schorr
001691e579 Fix whitespace 2016-12-20 13:25:23 -05:00
Joseph Schorr
5b3212ea0e Change security notification code to use the new stream diff reporters 
This ensures that even if security scanner pagination sends Old and New layer IDs on different pages, they will properly be handled across the entire notification.

Fixes https://www.pivotaltracker.com/story/show/136133657
 2016-12-20 12:50:19 -05:00
Joseph Schorr
ced0149520 Implement helper classes for tracking streaming diffs, both indexed and non-indexed 
These classes will be used to handle the Layer ID paginated diffs from Clair.
 2016-12-20 12:50:18 -05:00
Joseph Schorr
e2efb6c458 Add default and configurable LDAP timeouts 
Fixes https://www.pivotaltracker.com/story/show/135885019
 2016-12-19 11:53:06 -05:00
josephschorr
e58e04b0e9 Merge pull request #2242 from coreos-inc/clair-exceptions 
Security scanner flow changes and auto-retry
 2016-12-16 15:54:52 -05:00
Joseph Schorr
405eca074c Security scanner flow changes and auto-retry 
Changes the security scanner code to raise exceptions now for non-successful operations. One of the new exceptions raised is MissingParentLayerException, which, when raised, will cause the security worker to perform a full rescan of all parent images for the current layer, before trying once more to scan the current layer. This should allow the system to be "self-healing" in the case where the security scanner engine somehow loses or corrupts a parent layer.
 2016-12-16 15:38:09 -05:00
josephschorr
f72185f527 Merge pull request #2240 from coreos-inc/wrong-email-invite-accept 
Fix attempts to confirm team invite for mismatched email address
 2016-12-16 14:30:37 -05:00
josephschorr
9fa16679f8 Merge pull request #2238 from coreos-inc/fake-clair 
Add a fake security scanner class for easier testing
 2016-12-15 20:51:24 -05:00
Joseph Schorr
785c74de52 Fix attempts to confirm team invite for mismatched email address 
Currently, if a user tries to confirm an invite sent to them on an account with a mismatching email address, we simply redirect to the org (where they get a 403). This change ensures they get the proper error response message, and restyles the error page to be nicer.

Fixes #2227
Fixes https://www.pivotaltracker.com/story/show/136088507
 2016-12-15 17:15:11 -05:00
Joseph Schorr
15041ac5ed Add a fake security scanner class for easier testing 
The FakeSecurityScanner mocks out all calls that Quay is expected to make to the security scanner API, and returns faked data that can be adjusted by the calling test case
 2016-12-14 17:11:45 -05:00
EvB
0a5d4990e6 test(endpoints/api): ensure empty 202 resp 2016-12-14 16:32:06 -05:00
Joseph Schorr
6871eb95b1 Send notifications for previously unscannable layers in QSS 
Following this change, if an image was previously indexed unsuccessfully, then we will send notifications once successfully indexed
 2016-12-14 11:25:45 -05:00
Joseph Schorr
a9a75cd4cf Add a test for selecting images to be scanned 2016-12-14 00:07:48 -05:00
Joseph Schorr
624b2a8385 Have security scanner analyze only send notifications for *new* layers 
Following this change, anytime a layer is indexed by the security scanner, we only send notifications out if the layer previously had a security_indexed_engine value of `-1`, thus ensuring it has *never* been indexed previously. This will allow us to change to version of the security scanner upwards, and have all the images be re-indexed, without firing off notifications in a spammy manner.
 2016-12-13 23:17:11 -05:00
Evan Cordell
5686c80af1 Revert "Add GC of layers in Clair" 
This reverts 49872838ab
 2016-12-13 18:40:58 -05:00
Joseph Schorr
1302fd2fbd Switch csrf token check to use compare_digest to prevent timing attacks 
Also adds some additional tests for CSRF tokens
 2016-12-08 23:46:31 -05:00
Joseph Schorr
dbdcb802b1 Add end-to-end OAuth login and attach tests 2016-12-08 18:35:42 -05:00
josephschorr
410b9d74fc Merge pull request #2214 from coreos-inc/clair-gc 
Add GC of layers in Clair
 2016-12-07 17:58:21 -05:00
josephschorr
111b7b0788 Merge pull request #2206 from coreos-inc/ldap-user-search-fix 
Fix external auth returns for query_user calls
 2016-12-07 17:53:04 -05:00
Jimmy Zelinskie
00eafff747 Merge pull request #2204 from jzelinskie/429builds 
add rate limiting to build queues
 2016-12-07 15:03:31 -05:00
Joseph Schorr
3203fd6de1 Fix external auth returns for query_user calls 
Adds the missing field on the query_user calls, updates the external auth tests to ensure it is returned properly, and adds new end-to-end tests which call the external auth engines via the *API*, to ensure this doesn't break again
 2016-12-07 14:28:42 -05:00
Jimmy Zelinskie
ebbe58d311 replace prefix w/ canonical name list 2016-12-07 12:56:56 -05:00
Joseph Schorr
49872838ab Add GC of layers in Clair 
Fixes https://www.pivotaltracker.com/story/show/135583207
 2016-12-06 19:52:56 -05:00
Jimmy Zelinskie
eb69abff8b build rate limiting: tests 2016-12-06 16:30:12 -05:00
Jake Moshenko
21e3001446 Add a bulk insert for queue and notifications. 
Use it for Clair spawned notifications.
 2016-12-06 14:00:16 -05:00
Joseph Schorr
97d150e281 Have QSS only add security scanner notifications once 2016-12-05 19:08:20 -05:00
Joseph Schorr
a565251b58 Remove check that breaks under full db tests 2016-12-02 17:46:01 -05:00
Charlton Austin
0a6322015c Fix the queue item delete. 2016-12-02 15:30:35 -05:00
Antoine Legrand
784c5f4fc7 Merge pull request #2160 from ant31/use_pytest 
Add pytest, tox  and code-coverage to run tests
 2016-12-02 15:53:40 +01:00
Joseph Schorr
fdff0bee4e Add configurable Docker host in full db tests 2016-12-01 15:45:08 -05:00
josephschorr
64c954dc58 Merge pull request #2182 from coreos-inc/fix-full-db-tests 
Fix full database test script to not fail randomly
 2016-12-01 14:33:22 -05:00
Charlton Austin
1f03fcb146 Adding in notification type for notification kind. 2016-12-01 12:26:18 -05:00
Joseph Schorr
e6ee538e15 Fix full database test script to not fail randomly 
- Switches database schema creation to alembic, which solves the MySQL issue (and makes sure we test migrations as well)
- Adds a few time.sleep(1) to work around MySQL's second-precision issue when adding items to queues and then immediately retrieving them
- Disables the storage proxy tests when running against non-SQLite databases, as it causes failures with the multiple process and multiple transactions
- Changes initdb to support only populating the database, as well as fixing a few small items around the test data when working with non-SQLite data
 2016-11-30 18:24:08 -05:00
Charlton Austin
2c637fe5ce Merge pull request #2173 from charltonaustin/adding_in_build_cancel_notifications 
Adding in cancel notifications
 2016-11-30 15:03:17 -05:00
Charlton Austin
4103a0b75f Adding in cancel notifications 2016-11-30 14:38:34 -05:00
Joseph Schorr
236655adb4 Fix config validator for storage and add a test suite 
Note that the test suite doesn't fully verify that each validation succeeds; rather, it ensures that the proper system (storage, security scanning, etc) is called with the configuration and returns at all (usually with an expected error). This should prevent us from forgetting to update these code paths when we change config-based systems. Longer term, we might want to have these tests stand up fake/mock versions of the endpoint services as well, for end-to-end testing.
 2016-11-30 11:58:41 -05:00
Joseph Schorr
402ad25690 Change team invitation acceptance to join all invited teams under the org 
Fixes #1989
 2016-11-28 18:39:28 -05:00
Evan Cordell
b4ace1dd29 registry auth tests: test more access types 2016-11-28 14:02:08 -05:00
Evan Cordell
9e96e6870f Add support for * (admin) permission to registry auth v2 endpoint 2016-11-28 14:02:08 -05:00
ant31
2eaa8a4a1b Add pytest and tox to run tests 2016-11-28 13:13:07 +01:00
Jimmy Zelinskie
498d7fc15e Merge pull request #2143 from jakedt/makebuildmanasyncagain 
Make buildman async again
 2016-11-21 15:08:06 -05:00
Charlton Austin
2fe74e4057 Adding in UI for cancel anytime. 2016-11-21 10:58:32 -05:00
Jake Moshenko
f0ef4347e5 Make the redis client use AsyncWrapper and coroutines 
Change all log messages to be synchronous
 2016-11-18 15:59:14 -05:00
Jake Moshenko
5935e93eb8 Linter fixes. 2016-11-18 15:56:08 -05:00
Joseph Schorr
0b549125d9 Fix 500 on get label endpoint and add a test 
Fixes #2133
 2016-11-17 14:55:14 -05:00
Joseph Schorr
69e2cfad70 Fix github trigger when submitting a webhook without a head_commit 
Fixes #2125
 2016-11-16 14:14:17 -05:00
josephschorr
1346b7fb63 Merge pull request #2105 from coreos-inc/frack-swift 
Fix swift exception reporting on deletion and add async chunk cleanup
 2016-11-15 17:59:48 -05:00
Joseph Schorr
5f99448adc Add a chunk cleanup queue for async GC of empty chunks 
Instead of having the Swift storage engine try to delete the empty chunk(s) synchronously, we simply queue them and have a worker come along after 30s to delete the empty chunks. This has a few key benefits: it is async (doesn't slow down the push code), helps deal with Swift's eventual consistency (less retries necessary) and is generic for other storage engines if/when they need this as well
 2016-11-15 15:07:41 -05:00
Joseph Schorr
1a61ef4e04 Report the user's name and company to Marketo 
Also fixes the API to report the other changes (username and email) as well
 2016-11-14 17:34:50 -05:00
Joseph Schorr
59cb6bd216 Make sure to not log exceptions if Swift deletes fail 2016-11-11 14:17:32 -05:00
Joseph Schorr
3d221bcdd7 Add tests for the Swift storage layer using a fake swift engine 2016-11-10 15:43:03 -05:00
josephschorr
45b1148118 Merge pull request #2086 from coreos-inc/user-info 
Add collection of user metadata: name and company
 2016-11-09 13:15:07 -05:00
Joseph Schorr
bf2804bd4d Add a test for deleting a user with a user prompt 2016-11-08 18:27:12 -05:00
Joseph Schorr
0f2eb61f4a Add collection of user metadata: name and company 2016-11-08 16:15:02 -05:00
josephschorr
233b2be5c2 Merge pull request #2066 from coreos-inc/select-username 
Add support for temp usernames and an interstitial to confirm username
 2016-11-03 16:22:16 -04:00
Joseph Schorr
1e3b354201 Add support for temp usernames and an interstitial to confirm username 
When a user now logs in for the first time for any external auth (LDAP, JWT, Keystone, Github, Google, Dex), they will be presented with a confirmation screen that affords them the opportunity to change their Quay-assigned username.

Addresses most of the user issues around #74
 2016-11-03 15:59:14 -04:00
Joseph Schorr
3fd92aef35 Fix entity search API to not IndexError 2016-11-02 16:22:35 -04:00
josephschorr
840ea4e768 Merge pull request #2047 from coreos-inc/external-auth-email-optional 
Make email addresses optional in external auth if email feature is turned off
 2016-10-31 14:16:33 -04:00
Joseph Schorr
d7f56350a4 Make email addresses optional in external auth if email feature is turned off 
Before this change, external auth such as Keystone would fail if a user without an email address tried to login, even if the email feature was disabled.
 2016-10-31 13:50:24 -04:00
Joseph Schorr
bab17932ac Fix namespace lookup in V1 registry search 
Fixes #2053
 2016-10-31 13:24:40 -04:00
josephschorr
934cdecbd6 Merge pull request #1905 from coreos-inc/external-auth-search 
Add support for entity search against external auth users not yet linked
 2016-10-27 16:06:42 -04:00
Joseph Schorr
b3d1d7227c Add support to Keystone Auth for external user linking 
Also adds Keystone V3 support
 2016-10-27 15:42:03 -04:00
Joseph Schorr
fbb524e34e Add support to ExternalJWT Auth for external user linking 2016-10-27 15:42:03 -04:00
Joseph Schorr
f9ee8d2bef Add support to LDAP for external user linking 2016-10-27 15:42:03 -04:00
Joseph Schorr
d145222812 Add support for linking to external users in entity search 2016-10-27 15:42:03 -04:00
Jimmy Zelinskie
30821569a4 key server: fix tests by exporting jwk_with_kid 2016-10-25 16:14:18 -04:00
josephschorr
500a218768 Merge pull request #1875 from coreos-inc/max-chunk-size 
Make sure we don't generate chunk sizes larger than 5 GB.
 2016-10-25 14:12:31 -04:00
Joseph Schorr
bfe2646a50 Make sure we don't generate chunk sizes larger than 5 GB. 
Amazon S3 does not allow for chunk sizes larger than 5 GB; we currently don't handle that case at all, which is why large uploads are failing. This change ensures that if a storage engine specifies a *maximum* chunk size, we write multiple chunks no larger than that size.
 2016-10-25 13:57:49 -04:00
Jake Moshenko
6f815907a4 Merge pull request #2030 from jakedt/twooh 
Prepare the changelog for v2.0.0
 2016-10-24 16:30:49 -04:00
Charlton Austin
dc35769396 Merge pull request #2022 from charltonaustin/refactor_for_cancel_anytime 
Making some refactors to make it easier to cancel the build at any time.
 2016-10-24 16:17:55 -04:00
Jake Moshenko
45bacbabaa s/Regions/Deployments 2016-10-24 16:04:04 -04:00
Charlton Austin
1cde22e76c Making some refactors to make it easier to cancel the build at any time. 2016-10-24 15:59:33 -04:00
Joseph Schorr
19393a8619 Add a test for deleting a user with federated login 2016-10-21 17:55:22 -04:00
Joseph Schorr
5ed13da2e6 Add missing security test for delete org 2016-10-21 17:37:49 -04:00
Joseph Schorr
73eb66eac5 Add support for deleting namespaces (users, organizations) 
Fixes #102
Fixes #105
 2016-10-21 15:41:09 -04:00
Joseph Schorr
864c44501e Fix global messages by removing "extra" method 
I think this happened due to a bad merge.
 2016-10-20 13:53:51 -04:00
josephschorr
67dde6e154 Merge pull request #1852 from coreos-inc/underscore_orgs 
Better handling of namespace validation to fix a number of issues
 2016-10-20 13:36:32 -04:00
Joseph Schorr
3a68740ff7 Better handling of namespace validation to fix a number of issues 
- Fixes a bug which allows for underscores at the beginning of namespaces: Fixes #1849
- Allows dots and dashes for newer Docker clients: Fixes #1188
- Has the UI display better messaging associated with namespace entry
 2016-10-20 13:32:22 -04:00
Joseph Schorr
2eabf1a291 Fix tests and test provider for real license format 2016-10-18 23:44:08 -04:00
Joseph Schorr
67f828279d Switch the license validator to use config_provider and have a test license 
Fixes the broken tests currently which try (and fail) to read the license file
 2016-10-18 11:44:13 -04:00
Joseph Schorr
7a6fb7554d Only attempt to load the license for the setup tool once there is a valid user 
Prevents the 401 session expired box from appearing
 2016-10-17 21:57:17 -04:00
Joseph Schorr
ee96693252 Add superuser config section for updating license 2016-10-17 21:44:25 -04:00
Jimmy Zelinskie
0c5400b7d1 enforce license across registry blueprints 2016-10-17 21:43:45 -04:00
Joseph Schorr
8fe29c5b89 Add license upload step to the setup flow 
Fixes #853
 2016-10-17 21:43:15 -04:00
Charlton Austin
f45aac063e Merge pull request #2005 from charltonaustin/fix_spacing_for_motd 
Moving the messages endpoint to something more generic, and making th…
 2016-10-17 17:21:03 -04:00
Charlton Austin
8e5dc8d3db Moving the messages endpoint to something more generic, and making the get visible all the time. 2016-10-17 16:23:48 -04:00
Joseph Schorr
3439f814b6 Fix quoting of scopes in WWW-Authenticate header 
Fixes part of #2002
 2016-10-17 14:32:43 -04:00
Joseph Schorr
18097a1bd6 Fix Link headers for pagination to match RFC 
Fixes part of #2002
 2016-10-17 13:57:05 -04:00
josephschorr
b4dd04cca4 Merge pull request #1996 from coreos-inc/aci-con-test-fix 
Add a sleep to fix ACI conversion tests
 2016-10-14 16:25:57 -04:00
Joseph Schorr
886489c666 Fix NPE raised if a vulnerability notification doesn't have a level filter 
Fixes #1990
 2016-10-14 14:23:50 -04:00
Joseph Schorr
c3ce491f02 Add a sleep to fix ACI conversion tests 
This ensures that the cache is cleared before the second request is made
 2016-10-14 13:36:47 -04:00
Charlton Austin
97d644d95d Adding in the delete api and the delete and create UI. 2016-10-13 10:40:52 -04:00
charltonaustin
5546fa6214 Adding in messages table 2016-10-11 10:55:12 -04:00
Charlton Austin
be916fb6ed Merge pull request #1966 from charltonaustin/j_code_review_comments 
Adding in security tests and docs.
 2016-10-11 09:50:47 -04:00
charltonaustin
5a4b702888 Adding in security tests and docs. 2016-10-11 09:30:37 -04:00
Jake Moshenko
7a3ee86e53 Merge pull request #1957 from jakedt/absolutecorruption 
Always use absolute URLs in Location headers.
 2016-10-10 18:25:29 -04:00
Jake Moshenko
df1f35e9f9 Always use absolute URLs in Location headers. 
This works around docker/docker#15048
 2016-10-10 16:30:24 -04:00
josephschorr
7fc33a9a57 Merge pull request #1965 from coreos-inc/condense-slack-notifications 
Less verbose notifications for QSS
 2016-10-10 15:38:12 -04:00
Joseph Schorr
ebf4120326 Less verbose notifications for QSS 
Fixes #1914
 2016-10-10 15:18:49 -04:00
charltonaustin
fa10d799b2 Adding in one more unit test. 2016-10-10 14:00:20 -04:00
charltonaustin
14eb3005b6 Some fixes for code review. 2016-10-10 12:55:00 -04:00
charltonaustin
1e733ddffb Adding in a new message data model and the corresponding methods to in the API. 2016-10-07 15:56:58 -04:00