Commit graph

942 commits

Author SHA1 Message Date
Joseph Schorr
8d96d8b682 Add tests for missing logs APIs 2017-02-08 16:52:17 -08:00
Jimmy Zelinskie
c2c6bc1e90 test: add qss read failover case 2017-02-03 19:20:13 -05:00
Jimmy Zelinskie
dd033e4feb test: move ConfigForTesting 2017-02-03 19:20:13 -05:00
Charlton Austin
5a06530b43 Merge pull request #2314 from charltonaustin/move_tests_over_to_pytest_no_story
update(security_test.py): moving tests to new framework
2017-02-03 16:21:03 -05:00
Joseph Schorr
cf539487a1 Add API endpoint for retrieving security status by *manifest*, rather than Docker V1 image ID 2017-02-02 17:51:18 -05:00
Charlton Austin
85bcb63439 update(security_test.py): moving tests to new framework
We should be moving tests over to pytest

[none]
2017-02-02 13:40:00 -05:00
josephschorr
01ec22b362 Merge pull request #2300 from coreos-inc/openid-connect
OpenID Connect support and OAuth login refactoring
2017-01-31 18:14:44 -05:00
Joseph Schorr
973a110ac7 Full text search for repository name and description
Adds support for searching full text against the name and description of a repository

[Delivers #134867401]
2017-01-31 11:38:31 -05:00
Joseph Schorr
f5dbc350f8 Fix missed tests and revert conftest change (breaks docker build) 2017-01-30 17:28:25 -05:00
Joseph Schorr
d63cca025a DNS name check got reversed; breaks wildcards 2017-01-29 11:51:37 -05:00
Charlton Austin
dae93dce78 feature(superuser panel): ability to view logs
users would like the ability to view build logs in the superuser panel

[None]
2017-01-26 13:54:03 -05:00
Joseph Schorr
a9791ea419 Have external login always make an API request to get the authorization URL
This makes the OIDC lookup lazy, ensuring that the rest of the registry and app continues working even if one OIDC provider goes down.
2017-01-23 19:06:19 -05:00
Joseph Schorr
fda203e4d7 Add proper and tested OIDC support on the server
Note that this will still not work on the client side; the followup CL for the client side is right after this one.
2017-01-23 17:53:34 -05:00
Charlton Austin
7854bf6b3a Making test independent of message ordering. 2017-01-23 14:32:34 -05:00
Joseph Schorr
19f7acf575 Lay foundation for truly dynamic external logins
Moves all the external login services into a set of classes that share as much code as possible. These services are then registered on both the client and server, allowing us in the followup change to dynamically register new handlers
2017-01-20 15:21:08 -05:00
Joseph Schorr
4755d08677 Refactor and rename the standard OAuth services 2017-01-19 15:23:15 -05:00
Joseph Schorr
7c7a07fb5a Allow namespaces to be between 2 and 255 characters in length
[Delivers #137924329]
2017-01-19 13:10:26 -05:00
josephschorr
e2748fccd9 Merge pull request #2282 from coreos-inc/motd-updates
Severity and Markdown support in MOTD
2017-01-18 17:41:27 -05:00
Joseph Schorr
3106504f39 Severity and Markdown support in MOTD
[Delivers #133555165]
2017-01-18 16:55:32 -05:00
Joseph Schorr
b3a74b94b3 Fix flaky port selection in registry tests
Depends on https://github.com/jarus/flask-testing/pull/98

[Fixes #136705135]
2017-01-18 15:06:11 -05:00
Joseph Schorr
462f47924e More detailed namespace validation
Fixes namespace validation to use the proper regex for checking length, as well as showing the proper messaging if the entered namespace is invalid

[Delivers #137830461]
2017-01-17 17:31:59 -05:00
josephschorr
aafcb592a6 Merge pull request #2257 from coreos-inc/clair-gc-take2
feat(gc): Garbage collection for security scanning
2017-01-17 14:49:36 -05:00
Joseph Schorr
8c4e86f48b Change queue to use state-field for claiming items
Before this change, the queue code would check that none of the fields on the item to be claimed had changed between the time when the item was selected and the item is claimed. While this is a safe approach, it also causes quite a bit of lock contention in MySQL, because InnoDB will take a lock on *any* rows examined by the `where` clause of the `update`, even if they will ultimately thrown out due to other clauses (See: http://dev.mysql.com/doc/refman/5.7/en/innodb-locks-set.html: "A ..., an UPDATE, ... generally set record locks on every index record that is scanned in the processing of the SQL statement. It does not matter whether there are WHERE conditions in the statement that would exclude the row. InnoDB does not remember the exact WHERE condition, but only knows which index ranges were scanned").

As a result, we want to minimize the number of fields accessed in the `where` clause on an update to the QueueItem row. To do so, we introduce a new `state_id` column, which is updated on *every change* to the QueueItem rows with a unique, random value. We can then have the queue item claiming code simply check that the `state_id` column has not changed between the retrieval and claiming steps. This minimizes the number of columns being checked to two (`id` and `state_id`), and thus, should significantly reduce lock contention. Note that we can not (yet) reduce to just a single `state_id` column (which should work in theory), because we need to maintain backwards compatibility with existing items in the QueueItem table, which will be given empty `state_id` values when the migration in this change runs.

Also adds a number of tests for other queue operations that we want to make sure operate correctly following this change.

[Delivers #133632501]
2017-01-17 13:29:26 -05:00
Joseph Schorr
939c122f70 Complete item queue test 2017-01-17 13:26:09 -05:00
Joseph Schorr
dcfd379b17 Queue cancelation test 2017-01-17 13:26:09 -05:00
Charlton Austin
8ca8c17e27 Merge pull request #2225 from charltonaustin/adding_in_new_indices
Adding in new indices for queueitem table.
2017-01-17 11:46:51 -05:00
josephschorr
9b65b37011 Merge pull request #2245 from coreos-inc/recaptcha
Add support for recaptcha during the create account flow
2017-01-17 11:34:23 -05:00
josephschorr
eb2cafacd4 Merge pull request #2249 from coreos-inc/notifier-fixes
Security notification pagination fix
2017-01-17 11:33:25 -05:00
Charlton Austin
ca832df975 Adding in new indices for queueitem table. 2017-01-17 10:04:31 -05:00
Joseph Schorr
7e0fbeb625 Custom SSL certificates config panel
Adds a new panel to the superuser config tool, for managing custom SSL certificates in the config bundle

[Delivers #135586525]
2017-01-13 14:34:35 -05:00
Joseph Schorr
3a24871422 Add SSL certificate utility and tests 2017-01-10 17:06:13 -05:00
Joseph Schorr
3eb17b7caa Add support for recaptcha during the create account flow
If the feature is enabled and recaptcha keys are given in config, then a recaptcha box is displayed in the UI when creating a user and a recaptcha response code *must* be sent with the create API call for it to succeed.
2017-01-09 11:08:21 -05:00
Joseph Schorr
ce21788da8 test(queue): delete_namespaced_items
Add queue tests for delete_namespaced_items
2017-01-09 11:05:39 -05:00
Joseph Schorr
d609e6a1c4 Security scanner garbage collection support
Adds support for calling GC in the security scanner for any layers+storage removed by GC on the Quay side
2016-12-22 14:55:26 -05:00
Joseph Schorr
5225642850 Garbage collection image+storage callback support
Add support to GC to invoke a callback with the image+storages removed. Only images whose storage was also removed will be sent to the callback. This will be used by security scanning for its own GC in the followup change.
2016-12-22 14:27:42 -05:00
Joseph Schorr
ef80471a39 fix(136521333): Handle None email_or_id in avatar code
Fixes https://www.pivotaltracker.com/story/show/136521333
2016-12-21 15:00:55 -05:00
josephschorr
732ab67b57 Merge pull request #2252 from coreos-inc/parallel-tests
Fix pytests and enable parallel registry tests
2016-12-20 16:56:52 -05:00
Joseph Schorr
481cebe46b Fix pytests and enable parallel registry tests 2016-12-20 15:42:04 -05:00
Joseph Schorr
001691e579 Fix whitespace 2016-12-20 13:25:23 -05:00
Joseph Schorr
5b3212ea0e Change security notification code to use the new stream diff reporters
This ensures that even if security scanner pagination sends Old and New layer IDs on different pages, they will properly be handled across the entire notification.

Fixes https://www.pivotaltracker.com/story/show/136133657
2016-12-20 12:50:19 -05:00
Joseph Schorr
ced0149520 Implement helper classes for tracking streaming diffs, both indexed and non-indexed
These classes will be used to handle the Layer ID paginated diffs from Clair.
2016-12-20 12:50:18 -05:00
Joseph Schorr
e2efb6c458 Add default and configurable LDAP timeouts
Fixes https://www.pivotaltracker.com/story/show/135885019
2016-12-19 11:53:06 -05:00
josephschorr
e58e04b0e9 Merge pull request #2242 from coreos-inc/clair-exceptions
Security scanner flow changes and auto-retry
2016-12-16 15:54:52 -05:00
Joseph Schorr
405eca074c Security scanner flow changes and auto-retry
Changes the security scanner code to raise exceptions now for non-successful operations. One of the new exceptions raised is MissingParentLayerException, which, when raised, will cause the security worker to perform a full rescan of all parent images for the current layer, before trying once more to scan the current layer. This should allow the system to be "self-healing" in the case where the security scanner engine somehow loses or corrupts a parent layer.
2016-12-16 15:38:09 -05:00
josephschorr
f72185f527 Merge pull request #2240 from coreos-inc/wrong-email-invite-accept
Fix attempts to confirm team invite for mismatched email address
2016-12-16 14:30:37 -05:00
josephschorr
9fa16679f8 Merge pull request #2238 from coreos-inc/fake-clair
Add a fake security scanner class for easier testing
2016-12-15 20:51:24 -05:00
Joseph Schorr
785c74de52 Fix attempts to confirm team invite for mismatched email address
Currently, if a user tries to confirm an invite sent to them on an account with a mismatching email address, we simply redirect to the org (where they get a 403). This change ensures they get the proper error response message, and restyles the error page to be nicer.

Fixes #2227
Fixes https://www.pivotaltracker.com/story/show/136088507
2016-12-15 17:15:11 -05:00
Joseph Schorr
15041ac5ed Add a fake security scanner class for easier testing
The FakeSecurityScanner mocks out all calls that Quay is expected to make to the security scanner API, and returns faked data that can be adjusted by the calling test case
2016-12-14 17:11:45 -05:00
EvB
0a5d4990e6 test(endpoints/api): ensure empty 202 resp 2016-12-14 16:32:06 -05:00
Joseph Schorr
6871eb95b1 Send notifications for previously unscannable layers in QSS
Following this change, if an image was previously indexed unsuccessfully, then we will send notifications once successfully indexed
2016-12-14 11:25:45 -05:00