Commit graph

6575 commits

Author SHA1 Message Date
josephschorr
0e04d1ce80 Merge pull request #2304 from coreos-inc/small-bug-fix
Security scanner ordered tuplize bug fix
2017-01-24 14:59:31 -05:00
Joseph Schorr
7c1bb886db Security scanner ordered tuplize bug fix
If only the old list is present, we still need to tuplize the entries.

Fixes https://sentry.io/coreos/backend-production/issues/207196561/
2017-01-24 13:16:44 -05:00
Charlton Austin
d0c6c47fb1 Merge pull request #2290 from charltonaustin/add_metrics_for_imagestoragereplication
Adding a metrics queue for imagestoragereplication
2017-01-24 13:06:06 -05:00
Erica
82898d05f3 Merge pull request #2302 from coreos-inc/FIX-more-fe-bugs
fix(update-user.js): add another existence check
2017-01-24 12:17:14 -05:00
EvB
f049ddefde fix(update-user.js): add another existence check 2017-01-24 12:16:03 -05:00
Charlton Austin
0eb7612cbb Merge pull request #2296 from charltonaustin/fix_failing_tests_138172105
Making test independent of message ordering.
2017-01-24 12:09:46 -05:00
Joseph Schorr
a9791ea419 Have external login always make an API request to get the authorization URL
This makes the OIDC lookup lazy, ensuring that the rest of the registry and app continues working even if one OIDC provider goes down.
2017-01-23 19:06:19 -05:00
Jimmy Zelinskie
b338adea1d Merge pull request #2299 from jzelinskie/readmefix
README: remove last py 2.7.11 reference
2017-01-23 18:04:28 -05:00
Jimmy Zelinskie
bfd820eb7a README: remove last py 2.7.11 reference 2017-01-23 17:57:11 -05:00
Joseph Schorr
fda203e4d7 Add proper and tested OIDC support on the server
Note that this will still not work on the client side; the followup CL for the client side is right after this one.
2017-01-23 17:53:34 -05:00
Jimmy Zelinskie
32d92f4c4e Merge pull request #2297 from jzelinskie/readmeupdate
README: update mac instructions
2017-01-23 15:46:41 -05:00
Jimmy Zelinskie
125c03ab68 README: update mac instructions
also some misc cleanup
2017-01-23 15:44:53 -05:00
Charlton Austin
7854bf6b3a Making test independent of message ordering. 2017-01-23 14:32:34 -05:00
Joseph Schorr
19f7acf575 Lay foundation for truly dynamic external logins
Moves all the external login services into a set of classes that share as much code as possible. These services are then registered on both the client and server, allowing us in the followup change to dynamically register new handlers
2017-01-20 15:21:08 -05:00
Evan Cordell
28813159e5 fix(userevent): ignore subscribe notifications in userevents
[Fixes #138007389]
2017-01-20 13:38:02 -05:00
Charlton Austin
5ce3e7cdcd Merge pull request #2289 from charltonaustin/add_in_documentation_for_migrations
Adding in a description on how to run migrations against remote machi…
2017-01-19 16:18:00 -05:00
Charlton Austin
72597ac455 Adding a metrics queue for imagestoragereplication 2017-01-19 15:47:55 -05:00
Joseph Schorr
4755d08677 Refactor and rename the standard OAuth services 2017-01-19 15:23:15 -05:00
Joseph Schorr
bee2551dc2 Temporarily remove Dex login support
This will be added back in later in this PR as part of proper generic OIDC support
2017-01-19 14:51:12 -05:00
josephschorr
c116ef2987 Merge pull request #2291 from coreos-inc/namespace-adjustment
Allow namespaces to be between 2 and 255 characters in length
2017-01-19 13:43:50 -05:00
Charlton Austin
8f97e065b0 Adding in a description on how to run migrations against remote machines. 2017-01-19 13:38:06 -05:00
Joseph Schorr
7c7a07fb5a Allow namespaces to be between 2 and 255 characters in length
[Delivers #137924329]
2017-01-19 13:10:26 -05:00
josephschorr
6b0691577e Merge pull request #2287 from coreos-inc/backfill-state-id
Switch QueueItem state_id to be unique after a backfill
2017-01-18 17:44:17 -05:00
Joseph Schorr
71ec23b550 Switch QueueItem state_id to be unique after a backfill 2017-01-18 17:43:41 -05:00
josephschorr
e2748fccd9 Merge pull request #2282 from coreos-inc/motd-updates
Severity and Markdown support in MOTD
2017-01-18 17:41:27 -05:00
Joseph Schorr
3106504f39 Severity and Markdown support in MOTD
[Delivers #133555165]
2017-01-18 16:55:32 -05:00
josephschorr
d36545af83 Merge pull request #2288 from coreos-inc/stupid-grunt
Reorder grunt JS file exceptions to have it properly exclude test files
2017-01-18 15:36:54 -05:00
Joseph Schorr
933215b9f1 Reorder grunt JS file exceptions to have it properly exclude test files 2017-01-18 15:35:20 -05:00
josephschorr
db96778fa6 Merge pull request #2261 from coreos-inc/fix-flaky-port
fix(registry_test): Fix flaky port selection in registry tests
2017-01-18 15:21:27 -05:00
josephschorr
8b7178bdd6 Merge pull request #2286 from coreos-inc/remove-unique
Remove unique from queue item state_id
2017-01-18 15:08:51 -05:00
Joseph Schorr
b3a74b94b3 Fix flaky port selection in registry tests
Depends on https://github.com/jarus/flask-testing/pull/98

[Fixes #136705135]
2017-01-18 15:06:11 -05:00
Joseph Schorr
af23d2bedd Remove unique from queue item state_id 2017-01-18 15:04:26 -05:00
josephschorr
9667eab12c Merge pull request #2285 from coreos-inc/fix-tox
Fix tox definition file for registry tests
2017-01-18 14:33:55 -05:00
Joseph Schorr
0228d22b3d Fix tox definition file for registry tests 2017-01-18 14:31:43 -05:00
josephschorr
64a6f10431 Merge pull request #2284 from coreos-inc/better-tracking
Small fixes around tracking in the registry
2017-01-18 12:56:01 -05:00
Joseph Schorr
3cf8f6c28a Cleanup user event reporting and lower its timeout 2017-01-18 11:27:00 -05:00
Joseph Schorr
669a3070bd Only parse request URL in track_and_log when necessary 2017-01-18 11:23:23 -05:00
Joseph Schorr
89229a8f2c Don't publish registry events to Redis for robots
The tutorial can only be used by users, so no need to publish for robots, which can cause issues in pulling for builders and other prod mechanisms if Redis is being finicky
2017-01-18 11:22:07 -05:00
josephschorr
1bbb69367a Merge pull request #2283 from coreos-inc/namespace-messaging
More detailed namespace validation
2017-01-17 18:33:32 -05:00
Joseph Schorr
462f47924e More detailed namespace validation
Fixes namespace validation to use the proper regex for checking length, as well as showing the proper messaging if the entered namespace is invalid

[Delivers #137830461]
2017-01-17 17:31:59 -05:00
josephschorr
aafcb592a6 Merge pull request #2257 from coreos-inc/clair-gc-take2
feat(gc): Garbage collection for security scanning
2017-01-17 14:49:36 -05:00
josephschorr
0f203b01d3 Merge pull request #2276 from coreos-inc/queue-improvements
Various improvements to the queue code
2017-01-17 14:04:14 -05:00
Joseph Schorr
8c4e86f48b Change queue to use state-field for claiming items
Before this change, the queue code would check that none of the fields on the item to be claimed had changed between the time when the item was selected and the item is claimed. While this is a safe approach, it also causes quite a bit of lock contention in MySQL, because InnoDB will take a lock on *any* rows examined by the `where` clause of the `update`, even if they will ultimately thrown out due to other clauses (See: http://dev.mysql.com/doc/refman/5.7/en/innodb-locks-set.html: "A ..., an UPDATE, ... generally set record locks on every index record that is scanned in the processing of the SQL statement. It does not matter whether there are WHERE conditions in the statement that would exclude the row. InnoDB does not remember the exact WHERE condition, but only knows which index ranges were scanned").

As a result, we want to minimize the number of fields accessed in the `where` clause on an update to the QueueItem row. To do so, we introduce a new `state_id` column, which is updated on *every change* to the QueueItem rows with a unique, random value. We can then have the queue item claiming code simply check that the `state_id` column has not changed between the retrieval and claiming steps. This minimizes the number of columns being checked to two (`id` and `state_id`), and thus, should significantly reduce lock contention. Note that we can not (yet) reduce to just a single `state_id` column (which should work in theory), because we need to maintain backwards compatibility with existing items in the QueueItem table, which will be given empty `state_id` values when the migration in this change runs.

Also adds a number of tests for other queue operations that we want to make sure operate correctly following this change.

[Delivers #133632501]
2017-01-17 13:29:26 -05:00
Joseph Schorr
19cb64df5d Remove unused class 2017-01-17 13:26:09 -05:00
Joseph Schorr
7f63cbd14f Remove FOR UPDATE in Queue cancel and complete
We have no need for them anymore and it should reduce lock contention a bit

Fixes #776
2017-01-17 13:26:09 -05:00
Joseph Schorr
939c122f70 Complete item queue test 2017-01-17 13:26:09 -05:00
Joseph Schorr
dcfd379b17 Queue cancelation test 2017-01-17 13:26:09 -05:00
Charlton Austin
8ca8c17e27 Merge pull request #2225 from charltonaustin/adding_in_new_indices
Adding in new indices for queueitem table.
2017-01-17 11:46:51 -05:00
josephschorr
9b65b37011 Merge pull request #2245 from coreos-inc/recaptcha
Add support for recaptcha during the create account flow
2017-01-17 11:34:23 -05:00
josephschorr
eb2cafacd4 Merge pull request #2249 from coreos-inc/notifier-fixes
Security notification pagination fix
2017-01-17 11:33:25 -05:00