Charlton Austin
f701677a8e
refactor(data): add in new config for builder
...
we are doing phase one of the four phase migration on the builder config
2017-03-20 13:03:41 -04:00
Jimmy Zelinskie
0ea600628b
Merge pull request #2436 from jzelinskie/cnr-step1
...
CNR - Step 1
2017-03-17 15:37:29 -04:00
Jimmy Zelinskie
ad029fb331
data.migrations: don't use UTF-8 for unique fields
...
Unique indexes must have less than 767 bytes and UTF-8 encoding with 255
chars is beyond this maximum. Since this is an internal identifier, we
can be confident that we will not require UTF-8 for it in the future.
2017-03-17 15:21:24 -04:00
Jimmy Zelinskie
c915a40531
data.database: rm tag_kind from Tag indexes
...
These shouldn't be necessary.
2017-03-17 11:35:16 -04:00
Jimmy Zelinskie
0e32e77e99
data.database: document all CNR/OCI models
2017-03-17 11:35:16 -04:00
Jimmy Zelinskie
2a117f2d24
data.migrations: change CNR mimetypes to v0
...
Our initial CNR support is of a pre-v1 implementation of the
specification.
2017-03-17 11:33:16 -04:00
Jimmy Zelinskie
1e9ce85af6
data.database/migrations: remove repo_id from db
...
This also manually organizes and removes broken parts of the migration.
2017-03-17 11:33:16 -04:00
Antoine Legrand
8f323154ce
data.migrations: add OCI/CNR models
2017-03-17 11:33:16 -04:00
Antoine Legrand
c61024586d
data.database: add CNR/OCI models
2017-03-17 11:33:16 -04:00
Antoine Legrand
718aeeead8
Fix search group_by clause for PG
2017-03-17 16:30:24 +01:00
Antoine Legrand
ec847ce613
Switch from expire to delete redis log_entries
2017-03-17 15:35:47 +01:00
Joseph Schorr
e25c989fef
Add a cleanup worker for blob uploads
2017-03-16 13:36:59 -04:00
Joseph Schorr
e90cab4d77
Change revert tag into restore tag and add manifest support
2017-03-14 11:34:42 -04:00
Joseph Schorr
af743b156b
Show manifest digests in place of V1 ids in the tag view when possible
2017-03-14 11:34:41 -04:00
Jimmy Zelinskie
123d003d4e
Merge pull request #2424 from jzelinskie/qss-image
...
workers.securityworker: revert to image querying
2017-03-10 17:38:02 -05:00
Jimmy Zelinskie
a780136337
workers.securityworker: revert to image querying
2017-03-10 17:37:40 -05:00
josephschorr
cbac673d58
Merge pull request #2404 from coreos-inc/cas-gc-fix
...
Fix GC handling around CAS paths
2017-03-10 17:34:21 -05:00
josephschorr
432b2d3fe8
Merge pull request #2392 from coreos-inc/search-optimization
...
Optimize repository search by changing our lookup strategy
2017-03-10 15:44:26 -05:00
Jimmy Zelinskie
53eb579459
data.model.tag: find min *alive* tag
2017-03-10 13:15:35 -05:00
Joseph Schorr
b5bb76cdea
Optimize repository search by changing our lookup strategy
...
Previous to this change, repositories were looked up unfiltered in six different queries, and then filtered using the permissions model, which issued a query per repository found, making search incredibly slow. Instead, we now lookup a chunk of repositories unfiltered and then filter them via a single query to the database. By layering the filtering on top of the lookup, each as queries, we can minimize the number of queries necessary, without (at the same time) using a super expensive join.
Other changes:
- Remove the 5 page pre-lookup on V1 search and simply return that there is one more page available, until there isn't. While technically not correct, it is much more efficient, and no one should be using pagination with V1 search anyway.
- Remove the lookup for repos without entries in the RAC table. Instead, we now add a new RAC entry when the repository is created for *the day before*, with count 0, so that it is immediately searchable
- Remove lookup of results with a matching namespace; these aren't very relevant anyway, and it overly complicates sorting
2017-03-09 19:47:55 -05:00
Joseph Schorr
62312e6461
Add warning when CAS paths are skipped and ensure we are under a transaction
2017-03-08 17:01:07 -05:00
Joseph Schorr
69e550d125
Fix GC handling around CAS paths
...
Adds code to ensure we never GC CAS paths that are shared amongst multiple ImageStorage rows, as well as an associated pair of tests to catch the positive and negative cases.
2017-03-07 13:48:07 -05:00
Jimmy Zelinskie
40636d4103
find work based on tag IDs rather than image IDs
2017-03-06 17:09:57 -05:00
Jimmy Zelinskie
2cead05f53
data.model.tag: filter hidden for scan eligibility
2017-03-06 15:44:01 -05:00
Jimmy Zelinskie
904b902295
workers.securityworker: find eligible tag images
2017-03-06 14:37:34 -05:00
Jimmy Zelinskie
b9ac2b7b3b
workers.securityworker: simplify min id
2017-03-03 14:51:18 -05:00
Jimmy Zelinskie
4ed0cdda14
securityscanner: add a min image id option
...
This will enable us to force some instances of the securityworker to
scan only new images.
2017-03-03 13:55:25 -05:00
Joseph Schorr
8e863b8cf5
Implement new create and manager trigger UI
...
Implements the new trigger setup user interface, which is now a linear workflow found on its own page, rather than a tiny modal dialog
Fixes #1187
2017-02-28 16:51:42 -05:00
Joseph Schorr
8ec6221ca2
Fix health check
2017-02-24 12:23:18 -05:00
Joseph Schorr
c0f7530b29
Pull out JWT auth validation into validator class
...
Also fixes a small bug in validation (yay tests!)
2017-02-24 12:23:16 -05:00
josephschorr
f7a7d30ec2
Merge pull request #2366 from coreos-inc/alert-spam-fixes
...
Small fixes for alert spam
2017-02-22 14:18:18 -05:00
Joseph Schorr
478b1642b2
Eat AttributeError in peewee close database call
...
Fixes https://sentry.io/coreos/backend-production/issues/104257892/
2017-02-22 13:21:12 -05:00
Joseph Schorr
d29d2da1ca
Handle IntegrityError in tag update code
...
Fixes https://sentry.io/coreos/backend-production/issues/173470565/events/4938537230/
2017-02-22 13:20:04 -05:00
Joseph Schorr
ef9cb3757d
Check for missing repository on GC call
...
Fixes https://sentry.io/coreos/backend-production/issues/192273882/
2017-02-22 13:18:23 -05:00
Joseph Schorr
89b7c13da5
Catch team member invite missing exception
...
Fixes https://sentry.io/coreos/backend-production/issues/195926082/
2017-02-22 13:18:22 -05:00
Jake Moshenko
27f5f14f90
Linter fixes
2017-02-22 11:45:38 -05:00
Jake Moshenko
add6b654ae
Move the total image count stat back to the prom stat worker
2017-02-22 11:45:38 -05:00
Jimmy Zelinskie
3d21af59fd
data.model.image: fake QSS progress metric
2017-02-21 17:48:40 -05:00
Joseph Schorr
eece782038
Prevent peewee from loading the visibility every time
...
By calling `visibility` instead of `visibility_id`, peewee was issuing a SQL Select statement for the repository, which removes the benefit of the optimization
2017-02-17 12:09:48 -05:00
Joseph Schorr
421c5d6012
Fix bug where the login service ID doesn't exist
2017-02-16 16:27:53 -05:00
josephschorr
2a7d1fbe57
Merge pull request #2358 from coreos-inc/better-logging
...
Log more information to the action logs and display the namespaces for superusers
2017-02-14 16:38:35 -05:00
Charlton Austin
3fd8c8a60d
feature(app.py): adding queue_metrics to queues
...
publishing queue metrics for SRE
[none]
2017-02-14 16:01:28 -05:00
Joseph Schorr
11c931f781
Log more information to the action logs and display the namespaces for superusers
...
This helps superusers understand better what, exactly, is going on in the registry
2017-02-14 14:55:24 -05:00
Charlton Austin
85bcb63439
update(security_test.py): moving tests to new framework
...
We should be moving tests over to pytest
[none]
2017-02-02 13:40:00 -05:00
Joseph Schorr
b407f88a26
Remove unnecessary CloudWatch metrics
...
They are spamming the API and costing us a lot of money
2017-02-01 13:08:21 -05:00
josephschorr
01ec22b362
Merge pull request #2300 from coreos-inc/openid-connect
...
OpenID Connect support and OAuth login refactoring
2017-01-31 18:14:44 -05:00
Joseph Schorr
3324743bff
Fix db migration revision
2017-01-31 11:38:31 -05:00
Joseph Schorr
973a110ac7
Full text search for repository name and description
...
Adds support for searching full text against the name and description of a repository
[Delivers #134867401 ]
2017-01-31 11:38:31 -05:00
Joseph Schorr
d65d32b284
Convert model to use moved prefix_search method
2017-01-31 11:38:31 -05:00
Joseph Schorr
d89c79b92d
Full text support in peewee
...
Adds support for full text search in peewee with the creation of two new field types: `FullIndexedCharField` and `FullIndexedTextField`.
Note that this change depends upon https://github.com/zzzeek/sqlalchemy/pull/339
[Delivers #137453279 ]
[Delivers #137453317 ]
2017-01-31 11:38:31 -05:00
Joseph Schorr
fda203e4d7
Add proper and tested OIDC support on the server
...
Note that this will still not work on the client side; the followup CL for the client side is right after this one.
2017-01-23 17:53:34 -05:00
Evan Cordell
28813159e5
fix(userevent): ignore subscribe notifications in userevents
...
[Fixes #138007389 ]
2017-01-20 13:38:02 -05:00
Joseph Schorr
71ec23b550
Switch QueueItem state_id to be unique after a backfill
2017-01-18 17:43:41 -05:00
josephschorr
e2748fccd9
Merge pull request #2282 from coreos-inc/motd-updates
...
Severity and Markdown support in MOTD
2017-01-18 17:41:27 -05:00
Joseph Schorr
3106504f39
Severity and Markdown support in MOTD
...
[Delivers #133555165 ]
2017-01-18 16:55:32 -05:00
Joseph Schorr
af23d2bedd
Remove unique from queue item state_id
2017-01-18 15:04:26 -05:00
Joseph Schorr
3cf8f6c28a
Cleanup user event reporting and lower its timeout
2017-01-18 11:27:00 -05:00
Joseph Schorr
462f47924e
More detailed namespace validation
...
Fixes namespace validation to use the proper regex for checking length, as well as showing the proper messaging if the entered namespace is invalid
[Delivers #137830461 ]
2017-01-17 17:31:59 -05:00
josephschorr
aafcb592a6
Merge pull request #2257 from coreos-inc/clair-gc-take2
...
feat(gc): Garbage collection for security scanning
2017-01-17 14:49:36 -05:00
Joseph Schorr
8c4e86f48b
Change queue to use state-field for claiming items
...
Before this change, the queue code would check that none of the fields on the item to be claimed had changed between the time when the item was selected and the item is claimed. While this is a safe approach, it also causes quite a bit of lock contention in MySQL, because InnoDB will take a lock on *any* rows examined by the `where` clause of the `update`, even if they will ultimately thrown out due to other clauses (See: http://dev.mysql.com/doc/refman/5.7/en/innodb-locks-set.html : "A ..., an UPDATE, ... generally set record locks on every index record that is scanned in the processing of the SQL statement. It does not matter whether there are WHERE conditions in the statement that would exclude the row. InnoDB does not remember the exact WHERE condition, but only knows which index ranges were scanned").
As a result, we want to minimize the number of fields accessed in the `where` clause on an update to the QueueItem row. To do so, we introduce a new `state_id` column, which is updated on *every change* to the QueueItem rows with a unique, random value. We can then have the queue item claiming code simply check that the `state_id` column has not changed between the retrieval and claiming steps. This minimizes the number of columns being checked to two (`id` and `state_id`), and thus, should significantly reduce lock contention. Note that we can not (yet) reduce to just a single `state_id` column (which should work in theory), because we need to maintain backwards compatibility with existing items in the QueueItem table, which will be given empty `state_id` values when the migration in this change runs.
Also adds a number of tests for other queue operations that we want to make sure operate correctly following this change.
[Delivers #133632501 ]
2017-01-17 13:29:26 -05:00
Joseph Schorr
19cb64df5d
Remove unused class
2017-01-17 13:26:09 -05:00
Joseph Schorr
7f63cbd14f
Remove FOR UPDATE
in Queue cancel and complete
...
We have no need for them anymore and it should reduce lock contention a bit
Fixes #776
2017-01-17 13:26:09 -05:00
Charlton Austin
ca832df975
Adding in new indices for queueitem table.
2017-01-17 10:04:31 -05:00
Joseph Schorr
1cbacbbb63
Add tool for handling abusing users
2017-01-13 14:42:03 -05:00
Joseph Schorr
5225642850
Garbage collection image+storage callback support
...
Add support to GC to invoke a callback with the image+storages removed. Only images whose storage was also removed will be sent to the callback. This will be used by security scanning for its own GC in the followup change.
2016-12-22 14:27:42 -05:00
Joseph Schorr
e2efb6c458
Add default and configurable LDAP timeouts
...
Fixes https://www.pivotaltracker.com/story/show/135885019
2016-12-19 11:53:06 -05:00
Joseph Schorr
58b7481a63
Make sure robot accounts always show up first in entity search
...
Fixes https://www.pivotaltracker.com/story/show/136277321
Fixes #2241
2016-12-16 15:04:30 -05:00
Joseph Schorr
785c74de52
Fix attempts to confirm team invite for mismatched email address
...
Currently, if a user tries to confirm an invite sent to them on an account with a mismatching email address, we simply redirect to the org (where they get a 403). This change ensures they get the proper error response message, and restyles the error page to be nicer.
Fixes #2227
Fixes https://www.pivotaltracker.com/story/show/136088507
2016-12-15 17:15:11 -05:00
Joseph Schorr
624b2a8385
Have security scanner analyze only send notifications for *new* layers
...
Following this change, anytime a layer is indexed by the security scanner, we only send notifications out if the layer previously had a security_indexed_engine value of `-1`, thus ensuring it has *never* been indexed previously. This will allow us to change to version of the security scanner upwards, and have all the images be re-indexed, without firing off notifications in a spammy manner.
2016-12-13 23:17:11 -05:00
Evan Cordell
5686c80af1
Revert "Add GC of layers in Clair"
...
This reverts 49872838ab
2016-12-13 18:40:58 -05:00
Charlton Austin
0b8c2ef92f
Removing an unused import.
2016-12-08 13:53:52 -05:00
josephschorr
410b9d74fc
Merge pull request #2214 from coreos-inc/clair-gc
...
Add GC of layers in Clair
2016-12-07 17:58:21 -05:00
josephschorr
543d86ae10
Merge pull request #2221 from coreos-inc/fix-error-pages
...
Have all error pages be rendered by Angular
2016-12-07 17:53:14 -05:00
josephschorr
111b7b0788
Merge pull request #2206 from coreos-inc/ldap-user-search-fix
...
Fix external auth returns for query_user calls
2016-12-07 17:53:04 -05:00
josephschorr
c5111d7930
Merge pull request #2144 from coreos-inc/buildlogs-improvements
...
Change the append build log method to execute the two calls via one pipelined connection
2016-12-07 17:52:22 -05:00
Joseph Schorr
c06bba38de
Have all error pages be rendered by Angular
...
Fixes #2198
Fixes https://www.pivotaltracker.com/story/show/135724483
2016-12-07 17:49:02 -05:00
Jimmy Zelinskie
00eafff747
Merge pull request #2204 from jzelinskie/429builds
...
add rate limiting to build queues
2016-12-07 15:03:31 -05:00
Joseph Schorr
3203fd6de1
Fix external auth returns for query_user calls
...
Adds the missing field on the query_user calls, updates the external auth tests to ensure it is returned properly, and adds new end-to-end tests which call the external auth engines via the *API*, to ensure this doesn't break again
2016-12-07 14:28:42 -05:00
Jimmy Zelinskie
ebbe58d311
replace prefix w/ canonical name list
2016-12-07 12:56:56 -05:00
Jimmy Zelinskie
c41de8ded6
build queue rate limiting: address PR comments
2016-12-06 20:40:54 -05:00
Joseph Schorr
49872838ab
Add GC of layers in Clair
...
Fixes https://www.pivotaltracker.com/story/show/135583207
2016-12-06 19:52:56 -05:00
Jimmy Zelinskie
eb69abff8b
build rate limiting: tests
2016-12-06 16:30:12 -05:00
Jimmy Zelinskie
57770493fa
build rate limiting: use a rate
2016-12-06 16:30:12 -05:00
Jimmy Zelinskie
7877c6ab94
add rate limiting to build queues
2016-12-06 16:30:12 -05:00
Charlton Austin
0aa6e6cd58
Merge pull request #2203 from charltonaustin/fix_build_component_cleanup
...
Adding in a cancel method to the build component so we can properly c…
2016-12-06 14:13:10 -05:00
Jake Moshenko
21e3001446
Add a bulk insert for queue and notifications.
...
Use it for Clair spawned notifications.
2016-12-06 14:00:16 -05:00
Charlton Austin
c6be12e31e
Adding in a cancel method to the build component so we can properly clean up the job task.
2016-12-06 13:37:49 -05:00
Jimmy Zelinskie
3a7119d499
Merge pull request #2209 from coreos-inc/clair-notification-read
...
Clair notification read and queue fixes
2016-12-05 19:36:59 -05:00
Joseph Schorr
97d150e281
Have QSS only add security scanner notifications once
2016-12-05 19:08:20 -05:00
Jake Moshenko
7c490b46c8
Only save dirty fields on Queue queries.
2016-12-05 18:12:14 -05:00
Charlton Austin
0a6322015c
Fix the queue item delete.
2016-12-02 15:30:35 -05:00
Charlton Austin
7b3d8e3977
Merge pull request #2183 from charltonaustin/metrics_for_unscanned_images
...
Adding in some metrics around clair sec scan.
2016-12-02 11:50:29 -05:00
Charlton Austin
edd9dcd7f6
Adding in some metrics around clair sec scan.
2016-12-01 16:50:02 -05:00
Charlton Austin
1f03fcb146
Adding in notification type for notification kind.
2016-12-01 12:26:18 -05:00
Charlton Austin
2c637fe5ce
Merge pull request #2173 from charltonaustin/adding_in_build_cancel_notifications
...
Adding in cancel notifications
2016-11-30 15:03:17 -05:00
Charlton Austin
4103a0b75f
Adding in cancel notifications
2016-11-30 14:38:34 -05:00
Joseph Schorr
730a220eb0
Fix user lookup query under Postgres
...
Adds a missing group_by clause
2016-11-29 11:36:53 -05:00
Joseph Schorr
402ad25690
Change team invitation acceptance to join all invited teams under the org
...
Fixes #1989
2016-11-28 18:39:28 -05:00
Joseph Schorr
e29cb34336
Fix Set calls to gauges
...
Fixes #2150
The proper function is `Set` (not `set`), which was causing these gauges to not report to Prometheus
2016-11-21 15:27:17 -05:00
Charlton Austin
2fe74e4057
Adding in UI for cancel anytime.
2016-11-21 10:58:32 -05:00
Joseph Schorr
1b8820f2e7
Change the append build log method to execute the two calls via one pipelined connection
...
Should reduce the amount of packets used by the build manager
Reference: https://github.com/andymccurdy/redis-py#pipelines
2016-11-18 11:47:16 -05:00
Charlton Austin
fd7c566d31
Adding in cancel for a build that is building.
2016-11-16 17:40:24 -05:00
Joseph Schorr
1a61ef4e04
Report the user's name and company to Marketo
...
Also fixes the API to report the other changes (username and email) as well
2016-11-14 17:34:50 -05:00
Jake Moshenko
10255d4052
Merge pull request #2109 from jakedt/collapsemigrations
...
Collapse all migrations prior to 2.0.0 into one.
2016-11-10 17:35:07 -05:00
Jake Moshenko
b5834a8a66
Collapse all migrations prior to 2.0.0 into one.
2016-11-10 17:31:00 -05:00
Joseph Schorr
536809a992
Change LDAP errors into debug statements to reduce log clutter
...
Fixes #2083
2016-11-10 16:39:26 -05:00
Joseph Schorr
0f2eb61f4a
Add collection of user metadata: name and company
2016-11-08 16:15:02 -05:00
josephschorr
233b2be5c2
Merge pull request #2066 from coreos-inc/select-username
...
Add support for temp usernames and an interstitial to confirm username
2016-11-03 16:22:16 -04:00
Joseph Schorr
1e3b354201
Add support for temp usernames and an interstitial to confirm username
...
When a user now logs in for the first time for any external auth (LDAP, JWT, Keystone, Github, Google, Dex), they will be presented with a confirmation screen that affords them the opportunity to change their Quay-assigned username.
Addresses most of the user issues around #74
2016-11-03 15:59:14 -04:00
Joseph Schorr
3fd92aef35
Fix entity search API to not IndexError
2016-11-02 16:22:35 -04:00
Joseph Schorr
d7f56350a4
Make email addresses optional in external auth if email feature is turned off
...
Before this change, external auth such as Keystone would fail if a user without an email address tried to login, even if the email feature was disabled.
2016-10-31 13:50:24 -04:00
josephschorr
934cdecbd6
Merge pull request #1905 from coreos-inc/external-auth-search
...
Add support for entity search against external auth users not yet linked
2016-10-27 16:06:42 -04:00
Joseph Schorr
b3d1d7227c
Add support to Keystone Auth for external user linking
...
Also adds Keystone V3 support
2016-10-27 15:42:03 -04:00
Joseph Schorr
fbb524e34e
Add support to ExternalJWT Auth for external user linking
2016-10-27 15:42:03 -04:00
Joseph Schorr
f9ee8d2bef
Add support to LDAP for external user linking
2016-10-27 15:42:03 -04:00
Joseph Schorr
d145222812
Add support for linking to external users in entity search
2016-10-27 15:42:03 -04:00
Charlton Austin
2147005d2c
Adding a method of cancelling a build based on etcd message.
2016-10-25 12:50:58 -04:00
Charlton Austin
dc35769396
Merge pull request #2022 from charltonaustin/refactor_for_cancel_anytime
...
Making some refactors to make it easier to cancel the build at any time.
2016-10-24 16:17:55 -04:00
Charlton Austin
1cde22e76c
Making some refactors to make it easier to cancel the build at any time.
2016-10-24 15:59:33 -04:00
josephschorr
edc2bc8b93
Merge pull request #1698 from coreos-inc/delete-namespace
...
Add support for deleting namespaces (users, organizations)
2016-10-21 16:54:52 -04:00
Joseph Schorr
73eb66eac5
Add support for deleting namespaces (users, organizations)
...
Fixes #102
Fixes #105
2016-10-21 15:41:09 -04:00
josephschorr
4d89052bbf
Merge pull request #1764 from coreos-inc/db-timeout
...
Add a default database connect timeout
2016-10-20 15:16:53 -04:00
Joseph Schorr
b7fc7999c3
Delete old "license" checking code arounds user counts
...
This is legacy code that doesn't actually do anything of value
2016-10-20 14:58:35 -04:00
Jimmy Zelinskie
20ef43d5fb
workers.queuecleanup: remove direct peewee usage
2016-10-20 13:46:00 -04:00
Joseph Schorr
715fc27474
Add a default database connect timeout
...
Fixes #1760
2016-10-17 13:33:30 -04:00
Charlton Austin
97d644d95d
Adding in the delete api and the delete and create UI.
2016-10-13 10:40:52 -04:00
charltonaustin
5a4b702888
Adding in security tests and docs.
2016-10-11 09:30:37 -04:00
josephschorr
7fc33a9a57
Merge pull request #1965 from coreos-inc/condense-slack-notifications
...
Less verbose notifications for QSS
2016-10-10 15:38:12 -04:00
Joseph Schorr
ebf4120326
Less verbose notifications for QSS
...
Fixes #1914
2016-10-10 15:18:49 -04:00
charltonaustin
14eb3005b6
Some fixes for code review.
2016-10-10 12:55:00 -04:00
charltonaustin
4ae6e6efa9
Fixing some database integration errors
2016-10-10 10:51:30 -04:00
charltonaustin
1e733ddffb
Adding in a new message data model and the corresponding methods to in the API.
2016-10-07 15:56:58 -04:00
Joseph Schorr
0b7bb6d6c6
Fix issue in V1 registry code with accessing locations under HEAD
...
Fixes #1922
2016-10-03 17:09:12 +03:00
josephschorr
b4dd5ea4dd
Merge pull request #1867 from coreos-inc/keystone-timeout
...
Add configurable timeout and debug flags to Keystone users
2016-09-29 23:01:02 +02:00
Joseph Schorr
02b8afe127
Add labeling of built manifests with their build IDs
...
Also sends the digests to the notification
Fixes #593
2016-09-29 10:58:45 +02:00
Jimmy Zelinskie
44eca10c05
update interfaces to use ABC
2016-09-26 14:50:24 -04:00
Jimmy Zelinskie
a1a930b833
database: fix indices post-rebase
2016-09-26 14:49:58 -04:00
Jimmy Zelinskie
ca883e5662
port label support to refactored v2 registry
2016-09-26 14:49:58 -04:00
Joseph Schorr
3c8b87e086
Fix verbs in manifestlist
...
All registry_tests now pass
2016-09-26 14:49:58 -04:00
Jimmy Zelinskie
783c9e7a73
stop exporting experimental database models
2016-09-26 14:49:23 -04:00
Jimmy Zelinskie
c35413d4f6
add boilerplate for verbs data interface
2016-09-26 14:49:23 -04:00
Jimmy Zelinskie
2e5a94bc0b
create key server data interface
2016-09-26 14:49:23 -04:00
Jimmy Zelinskie
c06d395f96
create interfaces for v1 and v2 data model
2016-09-26 14:49:23 -04:00
Joseph Schorr
b775458d4b
lifetimes on Tags should now be in milliseconds
...
Fixes #1779
2016-09-26 14:49:04 -04:00
Joseph Schorr
db60df827d
Implement V2 interfaces and remaining V1 interfaces
...
Also adds some tests to registry tests for V1 stuff.
Note: All *registry* tests currently pass, but as verbs are not yet converted, the verb tests in registry_tests.py currently fail.
2016-09-26 14:49:04 -04:00
Jimmy Zelinskie
d67991987b
v1: refactor index
2016-09-26 14:48:42 -04:00
Jimmy Zelinskie
b68e1b5efc
add "get_" prefix to all db read funcs
2016-09-26 14:48:05 -04:00
Jimmy Zelinskie
32a6c22b43
mv data/types image
...
This change also merges formats into the new image module.
2016-09-26 14:48:05 -04:00
Jimmy Zelinskie
a516c08deb
v2: refactor auth to use data.types
2016-09-26 14:48:05 -04:00
Jimmy Zelinskie
3de6000428
v2: refactor blob.py to use data.types
2016-09-26 14:48:05 -04:00
Jimmy Zelinskie
e6c99bb471
re-ordered BlobUploading fields
2016-09-26 14:48:05 -04:00
Jimmy Zelinskie
3f722f880e
v2: add pagination decorator
2016-09-26 14:48:05 -04:00
Jimmy Zelinskie
5b630ebdb0
v2/manifest: refactor to use types
2016-09-26 14:48:05 -04:00
Joseph Schorr
ea18790dfe
Get V1 registry code working with new model methods
2016-09-26 14:47:06 -04:00
Joseph Schorr
94d71f2166
Fix model to actually initialize
2016-09-26 14:47:06 -04:00
Jimmy Zelinskie
8435c254c3
finish v1 registry refactor
2016-09-26 14:47:06 -04:00
Jimmy Zelinskie
c14437e54a
initial v1 refactor to use model methods
2016-09-26 14:47:06 -04:00
Jimmy Zelinskie
9cfd6ec452
database: initial manifestlist schema changes
2016-09-26 14:47:06 -04:00
Jimmy Zelinskie
e3a39d7bd6
fix indentation
2016-09-26 14:47:06 -04:00
josephschorr
ad4efba802
Merge pull request #1830 from coreos-inc/superuser-dashboard
...
Add prometheus stats to enable better dashboarding
2016-09-26 17:19:22 +02:00
Joseph Schorr
fd770422bb
Add configurable timeout and debug flags to Keystone users
...
Fixes #1855
2016-09-22 18:25:02 -04:00
Joseph Schorr
30af8aef1a
Add a worker for reporting global stats to Prometheus
...
Fixes #1789
2016-09-12 16:19:19 -04:00
Jake Moshenko
91963c17a0
Remove a join to slightly optimize the gc query.
2016-09-09 15:40:40 -04:00
Joseph Schorr
3d542b5e93
Handle KeyError nicer in _get_parent_image
...
Fixes #1810
2016-09-09 13:34:56 -04:00
Jake Moshenko
cf83c9a16a
Improve the garbage collection tests.
2016-09-07 13:25:19 -04:00
Jake Moshenko
584a5a7ddd
Reduce database bandwidth by tracking gc candidate images.
2016-09-07 13:25:19 -04:00
Jake Moshenko
0815f6b6c4
Fix indentation for DB queries.
2016-09-07 10:48:58 -04:00
Jake Moshenko
1d8b72235a
Add a helper method to Image to parse ancestor string.
2016-09-07 10:48:58 -04:00
josephschorr
cd8b45e25b
Merge pull request #1754 from coreos-inc/team-add-perms
...
Better UI and permissions handling for robots and teams
2016-09-06 17:21:19 -04:00
Joseph Schorr
b4939a3cd0
Fix filtering of repos only visible to org admins
2016-08-31 13:51:53 -04:00
Joseph Schorr
357005e33f
Raise a 409 if we try to insert a tag twice at the same time
...
Also fixes handling of labels for existing manifests
Fixes #1775
2016-08-29 16:03:35 -04:00
Joseph Schorr
1a2666be07
Fix deletion of labels and add tests
2016-08-26 16:07:49 -04:00
Joseph Schorr
608ffd9663
Basic labels support
...
Adds basic labels support to the registry code (V2), and the API. Note that this does not yet add any UI related support.
2016-08-26 15:24:26 -04:00
Joseph Schorr
391d70d9ec
Add repo permissions dialog for existing teams and robots
...
Fixes #1686
2016-08-22 14:43:12 -04:00
Joseph Schorr
6ebb417923
Redesign the teams page to use a table
...
Allows for faster loading and easier viewing of important information about teams
2016-08-22 14:42:54 -04:00
Jake Moshenko
d6a396be34
Fix all foreign key constraints to use naming convention.
2016-08-18 14:29:53 -04:00
Joseph Schorr
aeddc6af06
Handle GC constraint failures in a nicer way
...
Fixes #1739
2016-08-17 16:13:27 -04:00
josephschorr
2caa82d091
Merge pull request #1713 from coreos-inc/enable-iam
...
Enable IAM support for S3 storage
2016-08-16 16:13:29 -04:00
Joseph Schorr
7f5b536ddb
Fix pagination of repositories
...
Fixes #1725
2016-08-15 16:48:04 -04:00
Joseph Schorr
0f46230493
Add an index for lookup by account to log entries
...
Also fixes the query to require one less join
2016-08-12 17:39:31 -04:00
Joseph Schorr
855cc36057
Remove unneeded imports
2016-08-11 17:16:31 -04:00
Joseph Schorr
34d49e2d44
Fix duplicate derived storage cache creation issue
...
Fixes #1699
2016-08-10 16:18:52 -04:00
Joseph Schorr
4a2acac5dc
Fix pagination of public repos, make more efficient and add test
2016-08-10 15:08:06 -04:00
Joseph Schorr
bf8f621278
Temporarily remove the migration which drops the foreign keys on LogEntry, as it is invalid
2016-08-08 17:47:04 -04:00
josephschorr
1a137ee7b3
Merge pull request #1643 from coreos-inc/db-retry
...
Enable automatic retry for the database
2016-08-08 15:04:25 -04:00
Joseph Schorr
700e7b74e4
Enable automatic retry for the database
2016-08-08 15:02:42 -04:00
Jimmy Zelinskie
22a25ac2d3
Revert "Merge pull request #1678 from coreos-inc/delete-repo-fix"
...
This reverts commit df64caf133
, reversing
changes made to 0d1e453566
.
2016-08-08 12:38:15 -04:00
Jimmy Zelinskie
ce14b9dddf
modify log_action to internally resolve IDs
2016-08-08 12:38:15 -04:00
Jimmy Zelinskie
052c31752b
MIGRATION: drop foreign keys on logentry table
...
This migration generates the following for MySQL:
BEGIN;
-- Running upgrade 1093d8b212bb -> 6243159408b5
ALTER TABLE logentry DROP FOREIGN KEY fk_logentry_account_id_user;
ALTER TABLE logentry DROP FOREIGN KEY
fk_logentry_repository_id_repository;
ALTER TABLE logentry DROP FOREIGN KEY fk_logentry_performer_id_user;
UPDATE alembic_version SET version_num='6243159408b5' WHERE
alembic_version.version_num = '1093d8b212bb';
COMMIT;
2016-08-08 12:38:15 -04:00
Jimmy Zelinskie
e05bc8bf7d
migration.sh: default DOCKER_IP to localhost
2016-08-08 12:36:01 -04:00
josephschorr
6716a2562b
Merge pull request #1680 from coreos-inc/add-missing-index
...
Add various missing indexes
2016-08-08 12:34:58 -04:00
Joseph Schorr
80a37fd295
Add various missing indexes
...
Indexes added:
Image::repository - Needed for model.image.get_repository_images_without_placements
RepositoryTag::image - Needed for model.tag.get_tags_for_image
RepositoryTag::repository - Needed for repository deletion
RepositoryBuild::phase - Needed for model.build.list_repository_builds sorting
RepositoryBuild::started - Needed for model.build.list_repository_builds sorting
RepositoryBuild::repository+started+phase - Needed for model.build.list_repository_builds
RepositoryBuild::started+logs_archived+phase - Needed for model.build.get_archivable_build lookup
2016-08-08 12:34:45 -04:00
josephschorr
df64caf133
Merge pull request #1678 from coreos-inc/delete-repo-fix
...
Have repo deletion not lock all the things
2016-08-04 16:48:03 -04:00
Joseph Schorr
0b5cd95693
Have repo deletion not lock all the things
2016-08-04 16:45:59 -04:00
Joseph Schorr
b1b0da7afd
Fix off-by-one error in repo tags pagination
...
Fixes #1665
2016-08-02 14:17:33 -04:00
Jake Moshenko
05e2773fa7
Get rid of remaining slow query for garbage collection.
2016-08-01 18:22:38 -04:00
josephschorr
46a28617e8
Merge pull request #1651 from coreos-inc/fix-branches
...
Fix handling of multi-part branches in the build triggers
2016-07-26 16:00:21 -07:00
Joseph Schorr
9e4f8cac03
Optimize GC query for looking up deletable storages
2016-07-26 13:47:15 -07:00
Joseph Schorr
06d52f2c83
Fix handling of multi-part branches in the build triggers
...
Fixes #1360
2016-07-26 13:41:13 -07:00
Joseph Schorr
5de1e98d3c
Fix LDAP DN building for empty RDN list
2016-07-22 14:40:53 -04:00
Joseph Schorr
4d6f96cd6c
Add missing pass
keyword
2016-07-19 22:24:27 -04:00
Joseph Schorr
b8d2570725
Don't raise an error on duplicate placements
...
This can happen if two pushes are racing on the same storage.
2016-07-19 16:44:05 -04:00
Joseph Schorr
b0b7b63be9
Fix queue tests for MySQL
...
MySQL's date time's appear to have a 1 second threshold, so we need to make sure the queue items added for the tests are available as soon as they are added. Before this change, the available_after was set to `datetime.utcnow()`, and, if the `get` was called within 1 second, then its check would fail.
2016-07-15 13:27:50 -04:00
Joseph Schorr
4e1259b58a
Fix the Repository ID in pagination problem once and for all
...
But.... ONCE AND FOR ALL!
Note: Tested on SQLite, Postgres and MySQL
2016-07-14 17:09:52 -04:00
Jimmy Zelinskie
64d0c5b675
data.queue: fix race condition
...
It's possible that multiple consumers will acquire a queue item if they
race on an expired item. To mitigate this, we check that the
processing_expires time hasn't been changed since we last read.
2016-07-14 15:34:22 -04:00
Jimmy Zelinskie
609f4fccd8
data.queue: simplify put method
2016-07-14 15:34:22 -04:00
Joseph Schorr
c1e4bf79b7
Fix delete team error message for admin teams
2016-07-11 15:47:05 -04:00
Joseph Schorr
241ebaa084
Fix typo
2016-07-07 15:06:29 -04:00
Joseph Schorr
adaeeba5d0
Allow for multiple user RDNs in LDAP
...
Fixes #1600
2016-07-07 14:46:38 -04:00
Joseph Schorr
e252ee07cb
Fix popularity metrics on list repos API
2016-07-06 16:15:54 -04:00
Joseph Schorr
713ba3abaf
Further updates to the Prometheus client code
2016-07-01 14:16:51 -04:00
Jake Moshenko
668a8edc50
Refactor prometheus integration
...
Move prometheus to SaaS and make it a plugin
Move static callers to use metrics_queue plugin
Change local-docker to support different quay clone dirnames
Change prom_aggregator to use logrus
2016-07-01 14:16:50 -04:00
Matt Jibson
3d9acf2fff
Use prometheus as a metric backend
...
This entails writing a metric aggregation program since each worker has its
own memory, and thus own metrics because of python gunicorn. The python
client is a simple wrapper that makes web requests to it.
2016-07-01 14:16:50 -04:00
Joseph Schorr
117ccda1cf
Fix postgres error in SQL query
2016-07-01 13:04:20 -04:00
Joseph Schorr
1eec6f53b2
Fix SQL error with pagination around Repositories
...
Fixes #1591
2016-06-30 17:31:35 -04:00
Joseph Schorr
310ecd11cc
Handle user events Redis not working in tutorial
...
Also does some basic restyling
Fixes #1586
2016-06-28 17:04:31 -04:00
Joseph Schorr
853cca35f3
Change repo stats to use the RAC table and a nice UI
2016-06-22 15:06:53 -04:00
josephschorr
7173d53030
Merge pull request #1549 from coreos-inc/certs
...
Switch to install custom LDAP cert by name
2016-06-21 15:13:44 -04:00
Joseph Schorr
66ec1d81ce
Switch to install custom LDAP cert by name
2016-06-21 15:10:26 -04:00
josephschorr
9e6a264f5f
Merge pull request #1523 from coreos-inc/verb-tag-cache-fix
...
Add a uniqueness hash to derived image storage to break caching over …
2016-06-20 16:38:25 -04:00
Joseph Schorr
a43b741f1b
Add a uniqueness hash to derived image storage to break caching over tags
...
This allows converted ACIs and squashed images to be unique based on the specified tag.
Fixes #92
2016-06-20 16:34:52 -04:00
Joseph Schorr
3b994431eb
Auto expire the build status and logs in redis
2016-06-20 13:53:13 -04:00
Joseph Schorr
986d20bcad
Switch to generic RedisError
...
Fixes #1558
2016-06-20 11:20:17 -04:00
Jake Moshenko
a1cf12e460
Add a sitemap.txt for popular public repos
...
and reference it from the robots.txt
2016-06-17 14:34:20 -04:00
josephschorr
614b9124ae
Merge pull request #1512 from coreos-inc/optimize-queries
...
Optimize various queries
2016-06-16 14:22:59 -04:00
josephschorr
58bef472d9
Merge pull request #1526 from coreos-inc/superuser-grant
...
Add ability for super users to take ownership of namespaces
2016-06-13 16:23:10 -04:00
Joseph Schorr
20816804e5
Add ability for super users to take ownership of namespaces
...
Fixes #1395
2016-06-13 16:22:52 -04:00
josephschorr
bda5d7ae29
Merge pull request #1511 from coreos-inc/location-cache
...
Use a cache for ImageStorageLocation
2016-06-09 14:03:07 -04:00
Joseph Schorr
7aa6b812e2
Use a cache for ImageStorageLocation
...
No need to reload it from the DB or join as it is a static set only changed during migration
2016-06-09 14:02:42 -04:00
Joseph Schorr
8887f09ba8
Use the instance service key for registry JWT signing
2016-06-07 11:58:10 -04:00
Joseph Schorr
894b5fed6f
Remove TODO since we always need storage
2016-06-03 13:45:13 -04:00
Joseph Schorr
03fd2ea15a
Remove Image from _load_tag_manifests query
...
Doesn't appear used or necessary
2016-06-03 13:44:01 -04:00
Joseph Schorr
9a747ca6a0
Have get_parent_images not join on placements
...
The only case that needs the placements is in verbs, for which we use a new method
2016-06-03 13:33:15 -04:00
Joseph Schorr
8064419715
Remove Image join from get_active_tag
...
It isn't used anywhere in the query and appears to be completely unnecessary
2016-06-03 13:06:57 -04:00
Joseph Schorr
53538f9001
Optimize get_tag_image query
...
No caller uses the image placements or locations, so no need to load them.
2016-06-02 16:36:38 -04:00
josephschorr
cad8746f9d
Merge pull request #1502 from coreos-inc/image-replication
...
Enable storage replication for V2 and add backfill tool
2016-06-02 15:02:53 -04:00
Joseph Schorr
12924784ce
Enable storage replication for V2 and add backfill tool
...
Fixes #1501
2016-06-02 14:36:08 -04:00
josephschorr
a85c3ebff7
Merge pull request #1457 from coreos-inc/xauth
...
Add support for direct granting of OAuth tokens and add tests
2016-06-01 12:07:12 -04:00
josephschorr
1ddc73416c
Merge pull request #1500 from coreos-inc/better-errors
...
Better errors
2016-05-31 15:54:41 -04:00
Jimmy Zelinskie
1f488acf12
data.queue: move name matching clause
2016-05-31 15:44:11 -04:00
Joseph Schorr
04df2410ec
Add better errors if Redis is down
...
Fixes #1497
2016-05-31 15:24:36 -04:00
Jimmy Zelinskie
26300d3c8e
data.queue: lint
2016-05-27 14:51:19 -04:00
Jimmy Zelinskie
8a5aa65d74
data.queue: limiting before order by rand
2016-05-27 14:44:30 -04:00
Jimmy Zelinskie
44b56ae2cf
queue: explicitly declare ordering requirement
...
This change defaults the ordering requirement of queue items to be off
and only enables it for the build manager. This should make the queries
for getting queueitems significantly faster for every other use case.
2016-05-27 14:44:30 -04:00
Joseph Schorr
7933aecf25
Add support for direct granting of OAuth tokens and add tests
...
This allows a client (when authorized in a whitelist) to send direct credentials via a Basic auth header and therefore bypass the OAuth approval UI for that user.
2016-05-23 17:17:06 -04:00
Joseph Schorr
60bbca2185
Fix setup tool when binding to external auth
...
We now query the external auth provider for the external service's identifier before adding the linking row into the database. This fixes the case where the external service resolves a different identifier for the same username.
Fixes #1477
2016-05-23 17:11:36 -04:00
Joseph Schorr
043699cfb3
Always use log entry kind cache
...
Fixes #1445
2016-05-13 15:20:55 -04:00
Jimmy Zelinskie
972e4be811
log: cutoff at the max id past the cutoff_date
...
Previously we were using the min, which is always going to be equivalant
to the min id in the table.
2016-05-10 20:13:10 -07:00
Joseph Schorr
a736407611
Fix user:admin scope handling and add test
2016-05-09 11:16:01 +02:00
josephschorr
f55fd2049f
Merge pull request #1433 from coreos-inc/ldapoptions
...
Add additional options for LDAP
2016-05-04 14:06:29 -04:00
Joseph Schorr
42515ed9ec
Add additional options for LDAP
...
Fixes #1420
2016-05-04 13:59:20 -04:00
Joseph Schorr
6e2df3b339
Fix key server to not list expired keys
...
Fixes the key server to not list expire keys and by default not return expired or unapproved keys unless explicitly requested.
Fixes #1430
2016-05-03 17:58:47 -04:00
Jimmy Zelinskie
2aa88dcb80
only send notifications when superusers enabled
2016-04-29 15:42:25 -04:00
Jimmy Zelinskie
29e2d7c9d4
data.model.log: remove unused method
2016-04-29 14:22:53 -04:00
Jimmy Zelinskie
e47b29a974
migration: add missing delete from down migration
...
This also reorganizes the file a bit.
2016-04-29 14:10:33 -04:00
Jimmy Zelinskie
4a521f5844
database: revert logentry foreign key proxy
2016-04-29 14:10:33 -04:00
Evan Cordell
489752a0b7
Only refresh current instance service key
2016-04-29 14:10:33 -04:00
Evan Cordell
a6f6a114c2
service key worker to refresh automatic keys
2016-04-29 14:10:33 -04:00
Evan Cordell
2242c6773d
Add 'Automatic' ServiceKeyApprovalType
2016-04-29 14:10:33 -04:00
Joseph Schorr
6091db983b
Hide expired keys outside of their staleness window
2016-04-29 14:10:33 -04:00
Joseph Schorr
4f63a50a17
Change account-less logs to use a user and not null
...
This allows us to skip the migration
2016-04-29 14:09:37 -04:00
Jimmy Zelinskie
5cb6ba4d12
keyserver migration: fix constraint name
2016-04-29 14:09:37 -04:00
Joseph Schorr
28a80ef6a9
Make sure to verify service names on key creation
2016-04-29 14:09:37 -04:00
Joseph Schorr
522cf68c5d
Lots of smaller fixes:
...
- Add the rotation_duration to the keys API
- Have the key service UI use the new rotation_duration field
- Fix notification deletion lookup path
- Add proper support for the new notification in the UI
- Only delete expired keys after 7 days (configurable)
- Fix angular digest loop
- Fix unit tests
- Regenerate initdb
2016-04-29 14:05:16 -04:00
Jimmy Zelinskie
370ac3ecd0
service keys: add rotation_duration field
2016-04-29 14:05:16 -04:00
Jimmy Zelinskie
6577ac3e62
mv JWK-canonicalization util.security.fingerprint
2016-04-29 14:05:16 -04:00
Jimmy Zelinskie
4020ab9f55
service keys: delete notifications by prefix
2016-04-29 14:05:16 -04:00
Joseph Schorr
08017c5111
Further UI updates
2016-04-29 14:05:16 -04:00
Joseph Schorr
a4a01e76c0
Fix up the migration to include the additional changes needed
2016-04-29 14:05:16 -04:00
Jimmy Zelinskie
d19eb16b45
keyserver: add generate key function
...
The superuser API, initdb, and tests will all need this functionality.
2016-04-29 14:05:16 -04:00
Joseph Schorr
fb1dca4e94
Add API usage tests
2016-04-29 14:05:16 -04:00
Joseph Schorr
11ff3e9b59
keys ui WIP
2016-04-29 14:05:16 -04:00
Jimmy Zelinskie
885a41e6f5
key server: misc fixes to make jwtproxy work
2016-04-29 13:38:25 -04:00
Jimmy Zelinskie
5cdc7812dc
migration.sh: update to reflect timing
2016-04-29 13:38:25 -04:00
Jimmy Zelinskie
50ad1bb6b1
key server: misc cleanup to get it working
2016-04-29 13:38:25 -04:00
Jimmy Zelinskie
c79bb14049
service keys: fix stale query
2016-04-29 13:38:25 -04:00
Jimmy Zelinskie
86df0124c1
service keys: join with approvals
...
Also fixes a bug where we weren't reassigning the query after adding a
WHERE.
2016-04-29 13:38:25 -04:00
Jimmy Zelinskie
dff59b4a39
service key migration
2016-04-29 13:38:25 -04:00
Jimmy Zelinskie
c6b8b3ce8c
service_keys: s/get_keys/list_keys
2016-04-29 13:38:25 -04:00
Jimmy Zelinskie
0ec54fc70e
clear notifications on delete/replace service_key
2016-04-29 13:38:25 -04:00
Jimmy Zelinskie
42b5196b21
add notification path and use for service keys
2016-04-29 13:38:25 -04:00
Jimmy Zelinskie
f406942984
converging on proper rotation
2016-04-29 13:38:25 -04:00
Jimmy Zelinskie
aaf9e83278
basically finish superuser key api
2016-04-29 13:38:25 -04:00
Jimmy Zelinskie
35ed73e195
rework superuser api
2016-04-29 13:38:25 -04:00
Jimmy Zelinskie
4079dba167
service keys: do all the right stuff
2016-04-29 13:38:25 -04:00
Jimmy Zelinskie
6ecff950ab
service keys: add txs and select4update
2016-04-29 13:38:25 -04:00
Jimmy Zelinskie
499bb16306
service key server wip
2016-04-29 13:38:25 -04:00
josephschorr
9e88b1413d
Merge pull request #1325 from coreos-inc/blobuncompressedsize
...
Fix uncompressed size for blob store and add test
2016-04-28 13:15:33 -04:00
Jimmy Zelinskie
7239c465bf
improve stale cutoff id perf ( #1392 )
2016-04-20 15:03:06 -04:00
josephschorr
b0cc55276f
Merge pull request #1373 from coreos-inc/orgconvert
...
Org conversion improvements
2016-04-19 16:16:35 -04:00
Jimmy Zelinskie
5585e16c90
Merge pull request #1356 from jzelinskie/actionlogarchive
...
logrotateworker: save to storage via userfiles
2016-04-15 13:57:11 -04:00
Jimmy Zelinskie
3d190b786f
userfiles: make handler optional
2016-04-15 13:56:07 -04:00
Joseph Schorr
c604dbd0f6
Fix permissions when converting a user to an org
...
Fixes #1366
2016-04-14 17:39:45 -04:00
Joseph Schorr
1009362d26
Have recovery auto-verify the user
...
Fixes #1355
2016-04-08 13:41:16 -04:00
Jake Moshenko
bd5b44cbd2
Move the sequence fixer to a separate tool which can be run
2016-04-01 13:46:13 -04:00
josephschorr
edb157c5cb
Merge pull request #1294 from coreos-inc/partialperms
...
Change permissions to only load required by default
2016-03-30 16:40:40 -04:00
Joseph Schorr
42e934d84f
Make notification lookup faster and fix repo pagination on Postgres
2016-03-30 14:46:31 -04:00
Joseph Schorr
0dffdb87c9
Fix uncompressed size for blob store and add test
2016-03-29 14:16:56 -04:00
Joseph Schorr
a3aa4592cf
Change permissions to only load required by default
...
Permissions now load just the namespace and/or repository permissions requested, with a fallback to a full permissions load if necessary.
2016-03-28 16:33:32 -04:00
Jake Moshenko
eed07722cb
Add even larger plans for enterprises on SaaS
2016-03-21 16:38:34 -04:00
Jake Moshenko
fe2cd240bc
Revert "Remove old search API which is no longer in use"
2016-03-07 10:07:41 -05:00
Jimmy Zelinskie
b5d904f373
Merge pull request #1218 from jzelinskie/logrotate5ever
...
vastly simplify log rotation
2016-03-04 13:48:21 -05:00
josephschorr
57430a18b4
Merge pull request #1224 from coreos-inc/removeoldsearch
...
Remove old search API which is no longer in use
2016-03-04 12:05:07 -05:00
Joseph Schorr
f498e92d58
Implement against new Clair paginated notification system
2016-02-25 15:58:42 -05:00
Joseph Schorr
c0374d71c9
Refactor the security worker and API calls and add a bunch of tests
2016-02-25 12:29:41 -05:00
Quentin Machu
0183c519f7
Merge pull request #1253 from Quentin-M/clair2
...
Adapt securityworker, secscan API and Quay UI for Clair 1.0
2016-02-19 18:21:25 -05:00
Quentin Machu
e5da33578c
Adapt security worker for Clair v1.0 (except notifications)
2016-02-19 17:44:14 -05:00
Joseph Schorr
abd2e3c234
V1 Docker ID <-> V2 layer SHA mismatch fix
...
Fix handling of V1 Docker ID <-> V2 layer SHA mismatch by dynamically rewriting the manifest to use new synthesized IDs for all layers above the mismatch. Also adds a bunch of tests for this and other use cases, fixes a bug around manifest digest uniqueness and fixes the 5.5 migration for MySQL.
2016-02-12 17:39:27 +02:00
josephschorr
a9c64545fa
Merge pull request #1228 from coreos-inc/v2storagevalidation
...
Add a check that will fail if we try to mislink V1 layers
2016-02-11 22:49:33 +02:00
Joseph Schorr
27f1cc0a13
Add a check that will fail if we try to mislink V1 layers
...
Also logs some useful information
2016-02-11 22:40:00 +02:00
Jake Moshenko
59a6f5bc77
Replace incompatible MySQL 5.5 server_default
2016-02-11 15:07:16 -05:00
Joseph Schorr
1887dc879c
Remove old search API which is no longer in use
2016-02-10 15:02:27 +02:00
Jimmy Zelinskie
ee705fe7a9
vastly simplify log rotation
2016-02-09 18:20:14 -05:00
Joseph Schorr
4e771e667f
Change sec scan candidate query to match parents to the expected version only
2016-02-09 22:23:48 +02:00
Joseph Schorr
534ec9cb2b
Add pagination to the repository list API to make it better for public
...
Fixes #1166
2016-02-01 22:42:44 +02:00
Joseph Schorr
1536709c02
Small fixes
2016-01-29 20:01:17 +02:00
Joseph Schorr
a80ac8eabb
Fix import for alembic
2016-01-29 17:59:23 +02:00
Joseph Schorr
bd0a098282
Add ID-based pagination to logs using new decorators and an encrypted token
...
Fixes #599
2016-01-26 12:50:48 -05:00
Jimmy Zelinskie
e54b86c6eb
s/TORRENT/BITTORRENT
2016-01-22 15:52:28 -05:00
Jake Moshenko
fe2bdeb6cb
Require some data from all models in initdb
2016-01-19 15:30:27 -05:00
Jake Moshenko
1b392dcb9a
Remove dependent signatures before removing image storages
2016-01-19 14:56:02 -05:00
Jake Moshenko
7d0be20842
Formatting and syntax improvements
2016-01-19 14:56:02 -05:00
Joseph Schorr
e4da61a05d
Fix piece hash calculation
2016-01-12 17:44:19 -05:00
josephschorr
047c2c2c0f
Merge pull request #1129 from coreos-inc/backfill
...
Add checksum and torrent info backfill
2016-01-12 14:20:58 -05:00
Jake Moshenko
96c72e73df
Clean up torrents before removing referenced storages
2016-01-12 11:43:07 -05:00
Jake Moshenko
8ab6c8a22d
Fix torrent hash generation to work in mixed stacks
2016-01-11 16:43:46 -05:00
Joseph Schorr
c36a7c21c8
Order sadly matters with this check in peewee
2016-01-11 15:10:46 -05:00
Joseph Schorr
bd715c0c71
Add checksum and torrent info backfill
2016-01-08 17:32:30 -05:00
Jake Moshenko
1ae101c917
Address torrent feature review comments.
2016-01-08 16:38:21 -05:00
Jake Moshenko
073b68cf0d
Fix torrent migration and update backfill to compute torrent pieces
2016-01-08 11:15:34 -05:00
Jake Moshenko
77aa58996a
Fix the db definition for torrentinfo and add migration
2016-01-06 14:04:03 -05:00
Jake Moshenko
fd1e5f2407
Remove an unnecessary outer join
2016-01-05 14:43:40 -05:00
Jake Moshenko
8f80d7064b
Hash v1 uploads for torrent chunks
2016-01-05 14:43:40 -05:00
Jake Moshenko
8d5f4466d6
Cleanup some indentation and imports
2016-01-05 12:12:57 -05:00
Jimmy Zelinskie
9b0a84c02f
implement get_torrent_info
2016-01-04 16:17:51 -05:00
Jake Moshenko
a9b7ac6b48
Rotate robot user uuid when the credentials change
2016-01-04 16:17:51 -05:00
Jake Moshenko
ce8fcbeaae
Update the pieces to use base64 encoded binary
2016-01-04 16:17:51 -05:00
Jake Moshenko
5c6e033d21
Fix indentation
2016-01-04 16:17:51 -05:00
Jake Moshenko
fe87d3c796
Hash and track layer file chunks for torrenting
2016-01-04 16:17:51 -05:00
josephschorr
f748d4348d
Merge pull request #1106 from coreos-inc/billingemail
...
Add support for custom billing invoice email address
2016-01-04 14:34:30 -05:00
Joseph Schorr
31a8a0fba4
Better UX when recovering organization emails
...
Fixes #291
2015-12-28 15:25:31 -05:00
Joseph Schorr
10efa96009
Add support for custom billing invoice email address
...
Fixes #782
2015-12-28 13:59:50 -05:00
Joseph Schorr
01723d5546
Catch other cases where the queue item has been removed
...
Fixes #1096
2015-12-22 15:58:51 -05:00
Jake Moshenko
9c1a2e7e1b
Improve performance by removing unnecessary group by fields
2015-12-22 11:35:49 -05:00
josephschorr
5ac7369bf5
Merge pull request #1068 from coreos-inc/slowqueryfix
...
Remove check for derived image storages on image storage
2015-12-18 16:32:22 -05:00
Joseph Schorr
94ece129d4
Remove remaining recursive queries on repo delete and add test
2015-12-18 16:04:03 -05:00
josephschorr
16f814b7d9
Merge pull request #1075 from coreos-inc/userdeletefix
...
Fix user deletion under MySQL
2015-12-17 15:09:18 -05:00
Joseph Schorr
2e7835c372
Fix user deletion under MySQL
...
Fixes #973
2015-12-17 15:05:15 -05:00
Jimmy Zelinskie
e1f955a3f6
add a log rotation worker
...
Fixes #609 .
2015-12-16 17:22:28 -05:00
Joseph Schorr
f59f4e51e8
Remove check for derived image storages on image storage
...
Derived image storages are now 1-to-1 with image storages, so we know they have already been removed at this point
Fixes #1067
2015-12-16 13:41:25 -05:00
Joseph Schorr
73531d08b5
Add server default for the chunk_count column
2015-12-15 15:44:33 -05:00
Joseph Schorr
141f664bf7
Fix subquery delete which messes up MySQL
...
Fixes #1061
2015-12-15 13:15:10 -05:00
Joseph Schorr
9698d6f6a0
Add created column to blob upload
...
Fixes first half of #1054
2015-12-14 15:27:48 -05:00
Joseph Schorr
54095eb5cb
Handle the common case of one chunk when calculating the uncompressed size
...
Reference #992
2015-12-14 15:27:48 -05:00
josephschorr
94effb5aaa
Merge pull request #1023 from coreos-inc/getblobopt
...
Optimize blob lookup
2015-12-04 16:11:28 -05:00
Jake Moshenko
38cb63d195
Fix indentation on build model operations
2015-12-04 15:46:07 -05:00
Joseph Schorr
f07b940bc5
Optimize blob lookup
...
Fixes #1013
2015-12-04 14:47:09 -05:00
Joseph Schorr
c324ebd7f6
Only write exceptions for manifest gen when a tag exists
...
Fixes #1019
Currently, we just raise an exception to the logs regardless, which can make it appear as if there is an issue (when there isn't).
2015-12-03 16:04:17 -05:00
Silas Sewell
502e4c04d0
Fix seq migration down_revision
2015-11-30 17:59:04 -05:00
Silas Sewell
3833fb6530
Merge pull request #888 from coreos-inc/remove-hardcoded-ids
...
Fix seq generators for enum tables in postgres
2015-11-30 17:54:13 -05:00
Joseph Schorr
0f7c8105b0
Remove DerivedImageStorage table
2015-11-25 11:46:59 -05:00
Joseph Schorr
762cd56e64
Change derived storage to be based on image
...
Fixes #971
2015-11-24 12:44:07 -05:00
Jake Moshenko
3a29dfc535
Reducing in a tree to avoid recursion depth limits
2015-11-23 15:57:13 -05:00
Joseph Schorr
f4266d08d2
Fix handling of aggregate size in V2
...
Fixes #931
2015-11-20 11:44:03 -05:00
Joseph Schorr
4981ccbc4e
Fix issue with query when manifest count is 0
2015-11-19 17:44:16 -05:00
Jake Moshenko
c352050b07
For the last time, you can't delete with a subquery on the same table!
2015-11-19 16:44:27 -05:00
Jake Moshenko
7b53797677
Fix garbage collection when manifests may reference tags
2015-11-19 16:01:36 -05:00
Jake Moshenko
7ae94f414c
Alias our subqueries to appease the MySQL beast
2015-11-19 12:58:06 -05:00
Silas Sewell
1162814734
securityworker: mark children we can't analyze
...
This allows us to differentiate between images that are queued and those we
can't analyze in constant time.
2015-11-19 11:22:15 -05:00
Jake Moshenko
e6bd5488c9
Ensure that manifest tags are still alive
2015-11-19 11:01:47 -05:00
Jake Moshenko
b564492ea7
Improve the performance of fetching manifest blobs by checksum.
2015-11-19 11:01:47 -05:00
Quentin Machu
f2d874386b
Fix security worker (ok last time before I give up on engineering)
2015-11-18 21:21:00 -05:00
Quentin Machu
04f2688944
Merge pull request #917 from Quentin-M/fix_secwor
...
Fix security worker (again?)
2015-11-18 19:45:36 -05:00
Quentin Machu
88e85cded0
Fix security worker (again?)
2015-11-18 19:45:09 -05:00
Quentin Machu
6d89f259f5
Merge pull request #894 from Quentin-M/fix_secwor
...
Refactor security worker
2015-11-18 14:40:34 -05:00
Quentin Machu
605ed1fc77
Refactor security worker
2015-11-18 14:38:32 -05:00
Jake Moshenko
18b14001b4
Add indices for the security worker fields on Image
...
Fixes #906
2015-11-18 13:29:51 -05:00
Jake Moshenko
206e18d160
Image parents do not have to be nulled transitively on repo delete
2015-11-17 16:48:26 -05:00
Jake Moshenko
e252397292
Switch parent back to a ForeignKeyField without a constraint
2015-11-17 16:09:33 -05:00
Jake Moshenko
3374e8c812
Do not constrain deferred fields in SQLAlchemy bridge
2015-11-17 15:55:18 -05:00
Jake Moshenko
ae61ebeac9
The translate placements query was renamed in v2
2015-11-17 12:24:05 -05:00
Jake Moshenko
7205bf5e7f
Merge pull request #885 from jakedt/python-registry-v2
...
Python registry v2 mega merge
2015-11-16 16:15:40 -05:00
Silas Sewell
30b0101584
Fix seq generators for enum tables in postgres
...
This attempts to insert a temporary entry into each enum table until it
succeeds. It re-synchronizes the postgres sequence generators with the max id
of the table.
Fixes #883 and #880
2015-11-16 15:29:51 -05:00
Matt Jibson
13aa6cfcfc
No PUT for logarchive
...
fixes #862
2015-11-16 15:01:12 -05:00
Jake Moshenko
0459c3bc54
Merge remote-tracking branch 'upstream/master' into python-registry-v2
2015-11-16 14:22:54 -05:00
Jake Moshenko
52125bbfed
Fix gc by using the v1/v2 storage location helper everywhere
2015-11-16 14:13:37 -05:00
Joseph Schorr
819d461ed6
Remove migration re-added by merge accidentally
2015-11-12 22:02:26 -05:00
Joseph Schorr
030c69d7d2
Further merge fixes
2015-11-12 22:00:28 -05:00
Joseph Schorr
7816b0c657
Merge master into vulnerability-tool
2015-11-12 21:52:47 -05:00
Joseph Schorr
25b8b7590f
Fix all the things!
2015-11-12 20:55:41 -05:00
Jake Moshenko
44d06b0c2e
Fix v1 backward compatibility
2015-11-12 16:22:19 -05:00
Jake Moshenko
cf1ec68046
Correlate a specific blob storage with its placements
2015-11-12 16:20:59 -05:00
Jake Moshenko
ab340e20ea
Merge remote-tracking branch 'upstream/master' into python-registry-v2
2015-11-11 16:41:40 -05:00
Jake Moshenko
1d7be74a1f
Revert "Drop the v1 checksum column from imagestorage"
...
This reverts commit d292a34343
.
2015-11-11 16:39:46 -05:00
Jake Moshenko
a1ccd860e7
Merge pull request #823 from coreos-inc/phase3-11-07-2015
...
Phase3 11 07 2015
2015-11-11 14:22:19 -05:00
Jake Moshenko
1c6919dd93
We must fill in the parent_id on linking
2015-11-10 14:31:46 -05:00
Silas Sewell
e826b14ca4
Merge pull request #725 from coreos-inc/setup-tool-georeplication
...
superuser: add storage replication config
2015-11-09 17:43:38 -05:00
Silas Sewell
5000b1621c
superuser: add storage replication config
2015-11-09 17:34:22 -05:00
Joseph Schorr
a69c9e12fd
Update quay sec code to fix problems identified in previous review
...
- Change get_repository_images_recursive to operate over a single docker image and storage uuid
- Move endpoints/sec to endpoints/secscan
- Change notification system to work with new Quay-sec format
Fixes #768
2015-11-09 17:14:35 -05:00
Silas Sewell
c739c453da
Merge pull request #807 from coreos-inc/storage-preference
...
Enable storage preference
2015-11-09 16:30:47 -05:00