Commit graph

299 commits

Author SHA1 Message Date
Joseph Schorr
a007332d4c Temporarily change to storing logs in a new LogEntry2 table
This will prevent us from running out of auto-incrementing ID values until such time as we can upgrade to peewee 3 and change the field type to a BigInt

Fixes https://jira.coreos.com/browse/QUAY-943
2018-05-18 20:15:16 -04:00
Joseph Schorr
f5fafc480f Notifications can target robots, but the model did not reflect that
This fixes a breakage when deleting robots that have notifications targeting them.
2018-05-15 16:19:46 -04:00
Joseph Schorr
2ea13e86a0 Add last_accessed information to User and expose for robot accounts
Fixes https://jira.coreos.com/browse/QUAY-848
2018-03-21 15:28:34 -04:00
Joseph Schorr
254cdfe43a Add support for metadata on robot accounts
Fixes https://jira.coreos.com/browse/QUAY-847
Fixes https://jira.coreos.com/browse/QUAY-816
2018-03-12 20:32:05 -04:00
Joseph Schorr
c4a6273e00 Add creation date to User table 2018-03-09 13:31:29 -05:00
Joseph Schorr
ed84835d23 Changes for code review 2018-03-01 16:49:52 -05:00
Joseph Schorr
3837a7c2ed Add datetime of when a trigger was disabled 2018-03-01 16:49:51 -05:00
Joseph Schorr
93d79e777e Automatically disable build triggers with successive failures or internal errors
We allow users to reenable them manually once disabled
2018-03-01 16:49:51 -05:00
Joseph Schorr
c35eec0615 Add ability for triggers to be disabled
Will be used in the followup commit to automatically disable broken triggers
2018-03-01 16:49:28 -05:00
Joseph Schorr
8bc55a5676 Make namespace deletion asynchronous
Instead of deleting a namespace synchronously as before, we now mark the namespace for deletion, disable it, and rename it. A worker then comes along and deletes the namespace in the background. This results in a *significantly* better user experience, as the namespace deletion operation now "completes" in under a second, where before it could take 10s of minutes at the worse.

Fixes https://jira.coreos.com/browse/QUAY-838
2018-02-27 13:12:51 -05:00
Joseph Schorr
9a452ace11 Add configurable limits for number of builds allowed under a namespace
We also support that limit being increased automatically once a successful billing charge has gone through
2018-02-20 16:54:22 -05:00
Brad Ison
3de6b4a646 Add location metadata field for users 2018-02-06 16:06:17 -05:00
Joseph Schorr
182c4594e7 Remove unsupported DB connection arguments for non-pooled DB handler 2018-01-26 13:37:08 -05:00
Joseph Schorr
3bc5840496 Extend our logs about whether we are using connection pooling for a DB 2018-01-11 16:43:46 -05:00
josephschorr
d8fde005d8
Merge pull request #2961 from coreos-inc/joseph.schorr/QS-107/create-repo-opt
Small optimizations around create repository code
2018-01-05 15:40:30 -05:00
Joseph Schorr
524d77f527 Add an AppSpecificAuthToken data model for app-specific auth tokens. These will be used for the Docker CLI in place of username+password 2018-01-04 15:27:41 -05:00
Joseph Schorr
3f99e864e1 Reduce the number of DB queries for creating a repo 2018-01-04 14:32:39 -05:00
Joseph Schorr
6043bf45b4 Add configurable stale_timeout and max_connections on pool 2017-07-27 15:26:26 -04:00
Joseph Schorr
f79542fefb Enable connection pooling in the registry 2017-07-27 14:00:23 -04:00
Jake Moshenko
e9a5fdbad1 Use a more pro-active approach to identify an uninitialized db proxy. 2017-07-26 18:23:05 -04:00
Joseph Schorr
50c144a7c4 Remove the configure call in UseThenDisconnect
This hopefully avoids us creating secondary connections
2017-07-26 14:14:52 -04:00
Joseph Schorr
9febb539a7 Close the database connection after operations in buildman
Also adds a *temporary* hack to prevent this from breaking tests
2017-07-26 12:10:48 -04:00
Charlton Austin
993f2a174c feat(full-stack): disable notifications after 3 failures
This stops notifications from firing over and over again if they are repeatedly failing.

[TESTING -> locally with docker compose, DATABASE MIGRATION -> there is a single migration]

Issue: https://www.pivotaltracker.com/story/show/b144646649n

- [ ] It works!
- [ ] Comments provide sufficient explanations for the next contributor
- [ ] Tests cover changes and corner cases
- [ ] Follows Quay syntax patterns and format
2017-05-19 16:58:46 -04:00
Evan Cordell
2661db7485 Add flag to enable trust per repo (#2541)
* Add flag to enable trust per repo

* Add api for enabling/disabling trust

* Add new LogEntryKind for changing repo trust settings
Also add tests for repo trust api

* Add `set_trust` method to repository

* Expose new logkind to UI

* Fix registry tests

* Rebase migrations and regen test.db

* Raise downstreamissue if trust metadata can't be removed

* Refactor change_repo_trust

* Add show_if to change_repo_trust endpoint
2017-04-15 08:26:33 -04:00
Joseph Schorr
df3f47c79a Add a RepositorySearchScore table and calculation to the RAC worker
This will be used in a followup PR to order search results instead of the RAC join. Currently, the join with the RAC table in search results in a lookup of ~600K rows, which causes searching to take ~6s. This PR denormalizes the data we need, as well as allowing us to score based on a wider band (6 months vs the current 1 week).
2017-04-10 14:29:02 -04:00
Joseph Schorr
df603462b8 Add database migration for TeamSync 2017-04-03 11:31:29 -04:00
Joseph Schorr
94b07e6de9 Allow nulls in last_updated field to accurately report the last updated time to users for newly sync teams 2017-04-03 11:31:29 -04:00
Joseph Schorr
f5a854c189 Add TeamSync database and API support
Teams can now have a TeamSync entry in the database, indicating how they are synced via an external group. If found, then the user membership of the team cannot be changed via the API.
2017-04-03 11:31:28 -04:00
Jimmy Zelinskie
3ccf3c5f33 Merge pull request #2447 from jzelinskie/cnr-step2
CNR Step 2
2017-03-22 18:45:51 -04:00
Jimmy Zelinskie
5b362da8ac data.database: add RepositoryKind 2017-03-21 15:38:38 -04:00
Joseph Schorr
76de324ca8 Change blob upload ints into bigints 2017-03-21 13:14:11 -04:00
Jimmy Zelinskie
6a538647e4 data.database: beta classes skip transitive delete 2017-03-20 18:41:39 -04:00
Jimmy Zelinskie
c915a40531 data.database: rm tag_kind from Tag indexes
These shouldn't be necessary.
2017-03-17 11:35:16 -04:00
Jimmy Zelinskie
0e32e77e99 data.database: document all CNR/OCI models 2017-03-17 11:35:16 -04:00
Jimmy Zelinskie
1e9ce85af6 data.database/migrations: remove repo_id from db
This also manually organizes and removes broken parts of the migration.
2017-03-17 11:33:16 -04:00
Antoine Legrand
c61024586d data.database: add CNR/OCI models 2017-03-17 11:33:16 -04:00
Joseph Schorr
62312e6461 Add warning when CAS paths are skipped and ensure we are under a transaction 2017-03-08 17:01:07 -05:00
Joseph Schorr
478b1642b2 Eat AttributeError in peewee close database call
Fixes https://sentry.io/coreos/backend-production/issues/104257892/
2017-02-22 13:21:12 -05:00
Joseph Schorr
973a110ac7 Full text search for repository name and description
Adds support for searching full text against the name and description of a repository

[Delivers #134867401]
2017-01-31 11:38:31 -05:00
Joseph Schorr
d89c79b92d Full text support in peewee
Adds support for full text search in peewee with the creation of two new field types: `FullIndexedCharField` and `FullIndexedTextField`.

Note that this change depends upon https://github.com/zzzeek/sqlalchemy/pull/339

[Delivers #137453279]
[Delivers #137453317]
2017-01-31 11:38:31 -05:00
Joseph Schorr
71ec23b550 Switch QueueItem state_id to be unique after a backfill 2017-01-18 17:43:41 -05:00
josephschorr
e2748fccd9 Merge pull request #2282 from coreos-inc/motd-updates
Severity and Markdown support in MOTD
2017-01-18 17:41:27 -05:00
Joseph Schorr
3106504f39 Severity and Markdown support in MOTD
[Delivers #133555165]
2017-01-18 16:55:32 -05:00
Joseph Schorr
af23d2bedd Remove unique from queue item state_id 2017-01-18 15:04:26 -05:00
Joseph Schorr
8c4e86f48b Change queue to use state-field for claiming items
Before this change, the queue code would check that none of the fields on the item to be claimed had changed between the time when the item was selected and the item is claimed. While this is a safe approach, it also causes quite a bit of lock contention in MySQL, because InnoDB will take a lock on *any* rows examined by the `where` clause of the `update`, even if they will ultimately thrown out due to other clauses (See: http://dev.mysql.com/doc/refman/5.7/en/innodb-locks-set.html: "A ..., an UPDATE, ... generally set record locks on every index record that is scanned in the processing of the SQL statement. It does not matter whether there are WHERE conditions in the statement that would exclude the row. InnoDB does not remember the exact WHERE condition, but only knows which index ranges were scanned").

As a result, we want to minimize the number of fields accessed in the `where` clause on an update to the QueueItem row. To do so, we introduce a new `state_id` column, which is updated on *every change* to the QueueItem rows with a unique, random value. We can then have the queue item claiming code simply check that the `state_id` column has not changed between the retrieval and claiming steps. This minimizes the number of columns being checked to two (`id` and `state_id`), and thus, should significantly reduce lock contention. Note that we can not (yet) reduce to just a single `state_id` column (which should work in theory), because we need to maintain backwards compatibility with existing items in the QueueItem table, which will be given empty `state_id` values when the migration in this change runs.

Also adds a number of tests for other queue operations that we want to make sure operate correctly following this change.

[Delivers #133632501]
2017-01-17 13:29:26 -05:00
Charlton Austin
ca832df975 Adding in new indices for queueitem table. 2017-01-17 10:04:31 -05:00
Joseph Schorr
624b2a8385 Have security scanner analyze only send notifications for *new* layers
Following this change, anytime a layer is indexed by the security scanner, we only send notifications out if the layer previously had a security_indexed_engine value of `-1`, thus ensuring it has *never* been indexed previously. This will allow us to change to version of the security scanner upwards, and have all the images be re-indexed, without firing off notifications in a spammy manner.
2016-12-13 23:17:11 -05:00
Charlton Austin
0aa6e6cd58 Merge pull request #2203 from charltonaustin/fix_build_component_cleanup
Adding in a cancel method to the build component so we can properly c…
2016-12-06 14:13:10 -05:00
Charlton Austin
c6be12e31e Adding in a cancel method to the build component so we can properly clean up the job task. 2016-12-06 13:37:49 -05:00
Jake Moshenko
7c490b46c8 Only save dirty fields on Queue queries. 2016-12-05 18:12:14 -05:00