Commit graph

282 commits

Author SHA1 Message Date
Joseph Schorr
6043bf45b4 Add configurable stale_timeout and max_connections on pool 2017-07-27 15:26:26 -04:00
Joseph Schorr
f79542fefb Enable connection pooling in the registry 2017-07-27 14:00:23 -04:00
Jake Moshenko
e9a5fdbad1 Use a more pro-active approach to identify an uninitialized db proxy. 2017-07-26 18:23:05 -04:00
Joseph Schorr
50c144a7c4 Remove the configure call in UseThenDisconnect
This hopefully avoids us creating secondary connections
2017-07-26 14:14:52 -04:00
Joseph Schorr
9febb539a7 Close the database connection after operations in buildman
Also adds a *temporary* hack to prevent this from breaking tests
2017-07-26 12:10:48 -04:00
Charlton Austin
993f2a174c feat(full-stack): disable notifications after 3 failures
This stops notifications from firing over and over again if they are repeatedly failing.

[TESTING -> locally with docker compose, DATABASE MIGRATION -> there is a single migration]

Issue: https://www.pivotaltracker.com/story/show/b144646649n

- [ ] It works!
- [ ] Comments provide sufficient explanations for the next contributor
- [ ] Tests cover changes and corner cases
- [ ] Follows Quay syntax patterns and format
2017-05-19 16:58:46 -04:00
Evan Cordell
2661db7485 Add flag to enable trust per repo (#2541)
* Add flag to enable trust per repo

* Add api for enabling/disabling trust

* Add new LogEntryKind for changing repo trust settings
Also add tests for repo trust api

* Add `set_trust` method to repository

* Expose new logkind to UI

* Fix registry tests

* Rebase migrations and regen test.db

* Raise downstreamissue if trust metadata can't be removed

* Refactor change_repo_trust

* Add show_if to change_repo_trust endpoint
2017-04-15 08:26:33 -04:00
Joseph Schorr
df3f47c79a Add a RepositorySearchScore table and calculation to the RAC worker
This will be used in a followup PR to order search results instead of the RAC join. Currently, the join with the RAC table in search results in a lookup of ~600K rows, which causes searching to take ~6s. This PR denormalizes the data we need, as well as allowing us to score based on a wider band (6 months vs the current 1 week).
2017-04-10 14:29:02 -04:00
Joseph Schorr
df603462b8 Add database migration for TeamSync 2017-04-03 11:31:29 -04:00
Joseph Schorr
94b07e6de9 Allow nulls in last_updated field to accurately report the last updated time to users for newly sync teams 2017-04-03 11:31:29 -04:00
Joseph Schorr
f5a854c189 Add TeamSync database and API support
Teams can now have a TeamSync entry in the database, indicating how they are synced via an external group. If found, then the user membership of the team cannot be changed via the API.
2017-04-03 11:31:28 -04:00
Jimmy Zelinskie
3ccf3c5f33 Merge pull request #2447 from jzelinskie/cnr-step2
CNR Step 2
2017-03-22 18:45:51 -04:00
Jimmy Zelinskie
5b362da8ac data.database: add RepositoryKind 2017-03-21 15:38:38 -04:00
Joseph Schorr
76de324ca8 Change blob upload ints into bigints 2017-03-21 13:14:11 -04:00
Jimmy Zelinskie
6a538647e4 data.database: beta classes skip transitive delete 2017-03-20 18:41:39 -04:00
Jimmy Zelinskie
c915a40531 data.database: rm tag_kind from Tag indexes
These shouldn't be necessary.
2017-03-17 11:35:16 -04:00
Jimmy Zelinskie
0e32e77e99 data.database: document all CNR/OCI models 2017-03-17 11:35:16 -04:00
Jimmy Zelinskie
1e9ce85af6 data.database/migrations: remove repo_id from db
This also manually organizes and removes broken parts of the migration.
2017-03-17 11:33:16 -04:00
Antoine Legrand
c61024586d data.database: add CNR/OCI models 2017-03-17 11:33:16 -04:00
Joseph Schorr
62312e6461 Add warning when CAS paths are skipped and ensure we are under a transaction 2017-03-08 17:01:07 -05:00
Joseph Schorr
478b1642b2 Eat AttributeError in peewee close database call
Fixes https://sentry.io/coreos/backend-production/issues/104257892/
2017-02-22 13:21:12 -05:00
Joseph Schorr
973a110ac7 Full text search for repository name and description
Adds support for searching full text against the name and description of a repository

[Delivers #134867401]
2017-01-31 11:38:31 -05:00
Joseph Schorr
d89c79b92d Full text support in peewee
Adds support for full text search in peewee with the creation of two new field types: `FullIndexedCharField` and `FullIndexedTextField`.

Note that this change depends upon https://github.com/zzzeek/sqlalchemy/pull/339

[Delivers #137453279]
[Delivers #137453317]
2017-01-31 11:38:31 -05:00
Joseph Schorr
71ec23b550 Switch QueueItem state_id to be unique after a backfill 2017-01-18 17:43:41 -05:00
josephschorr
e2748fccd9 Merge pull request #2282 from coreos-inc/motd-updates
Severity and Markdown support in MOTD
2017-01-18 17:41:27 -05:00
Joseph Schorr
3106504f39 Severity and Markdown support in MOTD
[Delivers #133555165]
2017-01-18 16:55:32 -05:00
Joseph Schorr
af23d2bedd Remove unique from queue item state_id 2017-01-18 15:04:26 -05:00
Joseph Schorr
8c4e86f48b Change queue to use state-field for claiming items
Before this change, the queue code would check that none of the fields on the item to be claimed had changed between the time when the item was selected and the item is claimed. While this is a safe approach, it also causes quite a bit of lock contention in MySQL, because InnoDB will take a lock on *any* rows examined by the `where` clause of the `update`, even if they will ultimately thrown out due to other clauses (See: http://dev.mysql.com/doc/refman/5.7/en/innodb-locks-set.html: "A ..., an UPDATE, ... generally set record locks on every index record that is scanned in the processing of the SQL statement. It does not matter whether there are WHERE conditions in the statement that would exclude the row. InnoDB does not remember the exact WHERE condition, but only knows which index ranges were scanned").

As a result, we want to minimize the number of fields accessed in the `where` clause on an update to the QueueItem row. To do so, we introduce a new `state_id` column, which is updated on *every change* to the QueueItem rows with a unique, random value. We can then have the queue item claiming code simply check that the `state_id` column has not changed between the retrieval and claiming steps. This minimizes the number of columns being checked to two (`id` and `state_id`), and thus, should significantly reduce lock contention. Note that we can not (yet) reduce to just a single `state_id` column (which should work in theory), because we need to maintain backwards compatibility with existing items in the QueueItem table, which will be given empty `state_id` values when the migration in this change runs.

Also adds a number of tests for other queue operations that we want to make sure operate correctly following this change.

[Delivers #133632501]
2017-01-17 13:29:26 -05:00
Charlton Austin
ca832df975 Adding in new indices for queueitem table. 2017-01-17 10:04:31 -05:00
Joseph Schorr
624b2a8385 Have security scanner analyze only send notifications for *new* layers
Following this change, anytime a layer is indexed by the security scanner, we only send notifications out if the layer previously had a security_indexed_engine value of `-1`, thus ensuring it has *never* been indexed previously. This will allow us to change to version of the security scanner upwards, and have all the images be re-indexed, without firing off notifications in a spammy manner.
2016-12-13 23:17:11 -05:00
Charlton Austin
0aa6e6cd58 Merge pull request #2203 from charltonaustin/fix_build_component_cleanup
Adding in a cancel method to the build component so we can properly c…
2016-12-06 14:13:10 -05:00
Charlton Austin
c6be12e31e Adding in a cancel method to the build component so we can properly clean up the job task. 2016-12-06 13:37:49 -05:00
Jake Moshenko
7c490b46c8 Only save dirty fields on Queue queries. 2016-12-05 18:12:14 -05:00
Charlton Austin
fd7c566d31 Adding in cancel for a build that is building. 2016-11-16 17:40:24 -05:00
Joseph Schorr
1a61ef4e04 Report the user's name and company to Marketo
Also fixes the API to report the other changes (username and email) as well
2016-11-14 17:34:50 -05:00
Jake Moshenko
b5834a8a66 Collapse all migrations prior to 2.0.0 into one. 2016-11-10 17:31:00 -05:00
Joseph Schorr
0f2eb61f4a Add collection of user metadata: name and company 2016-11-08 16:15:02 -05:00
Joseph Schorr
1e3b354201 Add support for temp usernames and an interstitial to confirm username
When a user now logs in for the first time for any external auth (LDAP, JWT, Keystone, Github, Google, Dex), they will be presented with a confirmation screen that affords them the opportunity to change their Quay-assigned username.

Addresses most of the user issues around #74
2016-11-03 15:59:14 -04:00
josephschorr
edc2bc8b93 Merge pull request #1698 from coreos-inc/delete-namespace
Add support for deleting namespaces (users, organizations)
2016-10-21 16:54:52 -04:00
Joseph Schorr
73eb66eac5 Add support for deleting namespaces (users, organizations)
Fixes #102
Fixes #105
2016-10-21 15:41:09 -04:00
Joseph Schorr
715fc27474 Add a default database connect timeout
Fixes #1760
2016-10-17 13:33:30 -04:00
Charlton Austin
97d644d95d Adding in the delete api and the delete and create UI. 2016-10-13 10:40:52 -04:00
charltonaustin
4ae6e6efa9 Fixing some database integration errors 2016-10-10 10:51:30 -04:00
charltonaustin
1e733ddffb Adding in a new message data model and the corresponding methods to in the API. 2016-10-07 15:56:58 -04:00
Jimmy Zelinskie
a1a930b833 database: fix indices post-rebase 2016-09-26 14:49:58 -04:00
Jimmy Zelinskie
ca883e5662 port label support to refactored v2 registry 2016-09-26 14:49:58 -04:00
Joseph Schorr
3c8b87e086 Fix verbs in manifestlist
All registry_tests now pass
2016-09-26 14:49:58 -04:00
Jimmy Zelinskie
783c9e7a73 stop exporting experimental database models 2016-09-26 14:49:23 -04:00
Joseph Schorr
b775458d4b lifetimes on Tags should now be in milliseconds
Fixes #1779
2016-09-26 14:49:04 -04:00
Jimmy Zelinskie
e6c99bb471 re-ordered BlobUploading fields 2016-09-26 14:48:05 -04:00