Commit graph

285 commits

Author SHA1 Message Date
Joseph Schorr
e30b746aef Fix TagManifests with shared digests under the same repository.
TagManifests can (apparently, in very rare scenarios) share manifests with the exact same digests, so we need to support that case in the backfill worker. We also need to remove a unique constraint on the manifest column in the mapping table to support this case.
2018-08-20 11:32:59 -04:00
Joseph Schorr
2b4d52a357 Migrate the manifest bytes column to UTF8 2018-08-15 11:41:26 -04:00
Joseph Schorr
56222f95dc Change manifest creation to take in the map of blobs that form the manifest
We need to lookup the blobs *specific to the images in that manifest*, so we now pass them in from the locations in which we know that information
2018-08-07 16:28:50 -04:00
Joseph Schorr
89582438cd Fix the V22 phase 1 migrations to use new tables for mapping rather than editing existing tables
The ALTER TABLE operations previously used were causing the DB to die when run on the production TagManifest table which has 7 million rows. We instead now use new mapping tables, which is less nice, but these are temporary anyway, so hopefully we only have to deal with their ugliness for a short duration.
2018-08-06 16:58:27 -04:00
Joseph Schorr
a46660a06f Add new Manifest, ManifestLabel, ManifestLegacyImage and ManifestBlob tables and start writing and GCing to/from them
This change also starts passing in the manifest interface, rather than the raw data, to the model for writing.

Note that this change does *not* backfill the existing rows in to the new tables; that will occur in a followup PR. The new columns in `tagmanifest` and `tagmanifestlabel` will be used to track the backfill, as it will occur in a worker.
2018-08-06 13:01:11 -04:00
Joseph Schorr
7325b22c90 Change id column in LogEntry to a BigInt and migrate back to using that table 2018-07-30 12:42:56 -04:00
Joseph Schorr
487edf0ba1 Phase 3 of Appr migration
Deletes the old models and their code
2018-07-21 15:43:00 -04:00
Joseph Schorr
acb7d626a4 Phase 2 of Appr migration
Backfills the new tables from the old ones, and switches all code to read and write from the new tables
2018-07-21 15:25:04 -04:00
Joseph Schorr
113bb96f29 Phase 1 of migrating APPR-specific tables to tables with the Appr prefix
Fixes https://jira.coreos.com/browse/QUAY-950
2018-07-20 13:19:01 -04:00
Joseph Schorr
6622f27c93 Rename oci_model to appr_model 2018-07-20 12:46:19 -04:00
Sam Chow
872be8605a Fix error case in uploading tar, more comments 2018-06-22 13:23:08 -04:00
Sam Chow
aff1a08a83 Fix main app migration pathway 2018-06-21 15:33:26 -04:00
Sam Chow
b5f630ba29 Fix alembic migrations importing app
Ensure we connect to loaded config db
2018-06-20 17:17:35 -04:00
Joseph Schorr
f6ff0d6ca0 Add support for populating test data during migration testing
This change ensures that the tables in the database during migration have at least one row of "real" data, which should help catch issues in the future where we forget to set column defaults and other such schema oversights that can only be truly tested with non-empty tables

Fixes https://jira.coreos.com/browse/QUAY-913
2018-06-18 14:14:32 -04:00
Joseph Schorr
a007332d4c Temporarily change to storing logs in a new LogEntry2 table
This will prevent us from running out of auto-incrementing ID values until such time as we can upgrade to peewee 3 and change the field type to a BigInt

Fixes https://jira.coreos.com/browse/QUAY-943
2018-05-18 20:15:16 -04:00
Joseph Schorr
f06eec8a35 Fix issue where we didn't delete robots immediately under a namespace
This could result in "hanging" robot accounts, although that would only leak the names of said accounts. Now we delete them immediately AND we proactively delete them before replacing the namespace (just to be sure)
2018-05-09 17:53:30 +03:00
Joseph Schorr
4418772745 Postgres requires defaults for non-nullable cols, even integer ones
Fixes https://jira.coreos.com/browse/QUAY-912
2018-04-08 18:54:23 -04:00
Joseph Schorr
2ea13e86a0 Add last_accessed information to User and expose for robot accounts
Fixes https://jira.coreos.com/browse/QUAY-848
2018-03-21 15:28:34 -04:00
Joseph Schorr
254cdfe43a Add support for metadata on robot accounts
Fixes https://jira.coreos.com/browse/QUAY-847
Fixes https://jira.coreos.com/browse/QUAY-816
2018-03-12 20:32:05 -04:00
Joseph Schorr
c4a6273e00 Add creation date to User table 2018-03-09 13:31:29 -05:00
Joseph Schorr
ff1773bd28 Change name field back to a normal string
Our prod MySQL does not support an index on a UTF8 field of this size; thankfully, we don't need UTF8 here
2018-03-02 13:23:30 -05:00
Joseph Schorr
4db4a44491 Add missing server_default on enabled col in repositorybuildtrigger
Without it, we'll default all existing triggers to disabled
2018-03-02 13:13:32 -05:00
Joseph Schorr
ed84835d23 Changes for code review 2018-03-01 16:49:52 -05:00
Joseph Schorr
3837a7c2ed Add datetime of when a trigger was disabled 2018-03-01 16:49:51 -05:00
Joseph Schorr
93d79e777e Automatically disable build triggers with successive failures or internal errors
We allow users to reenable them manually once disabled
2018-03-01 16:49:51 -05:00
Joseph Schorr
c35eec0615 Add ability for triggers to be disabled
Will be used in the followup commit to automatically disable broken triggers
2018-03-01 16:49:28 -05:00
Joseph Schorr
8bc55a5676 Make namespace deletion asynchronous
Instead of deleting a namespace synchronously as before, we now mark the namespace for deletion, disable it, and rename it. A worker then comes along and deletes the namespace in the background. This results in a *significantly* better user experience, as the namespace deletion operation now "completes" in under a second, where before it could take 10s of minutes at the worse.

Fixes https://jira.coreos.com/browse/QUAY-838
2018-02-27 13:12:51 -05:00
Brad Ison
f89a16a64d Make byte_count on BlobUpload table not nullable
The byte_count field on the BlobUpload model is marked as not
nullable, but the migration to make the field a big integer removed
that restriction (#2388 :: 76de324) in the database.  It's still in
the model though, which means they are out of sync.  This adds a
migration to mark the field as not nullable in the database again.
2018-02-26 11:36:56 -05:00
Brad Ison
5bd826e7b5 Drop checksum from ImageStorage table
The checksum field was removed from the ImageStorage model in #815,
but was never dropped from the database.  This adds a migration to
drop the unused column.
2018-02-24 17:00:07 -05:00
Joseph Schorr
9a452ace11 Add configurable limits for number of builds allowed under a namespace
We also support that limit being increased automatically once a successful billing charge has gone through
2018-02-20 16:54:22 -05:00
Brad Ison
3de6b4a646 Add location metadata field for users 2018-02-06 16:06:17 -05:00
Joseph Schorr
6a876a6b73 Change title to be UTF8 2018-01-04 15:27:42 -05:00
Joseph Schorr
524d77f527 Add an AppSpecificAuthToken data model for app-specific auth tokens. These will be used for the Docker CLI in place of username+password 2018-01-04 15:27:41 -05:00
Jimmy Zelinskie
9e09612851 Revert "Merge pull request #2844 from coreos-inc/alegrand/use_latest_appr_server_code"
This reverts commit 646fafb2fd, reversing
changes made to 5c1b635439.
2017-08-09 20:45:46 -04:00
Antoine Legrand
6336a4a971 Migrate from cnr -> appr 2017-08-08 11:48:59 +02:00
Joseph Schorr
99d7fde8ee Add UI for viewing and changing the expiration of tags 2017-07-19 17:13:48 -04:00
Charlton Austin
52750e74ee fix(add notification number): add explicit default
there is a possibility that this will not work with postgres

Issue: https://www.pivotaltracker.com/story/show/144646649

- [ ] It works!
- [ ] Comments provide sufficient explanations for the next contributor
- [ ] Tests cover changes and corner cases
- [ ] Follows Quay syntax patterns and format
2017-05-22 09:55:40 -04:00
Charlton Austin
993f2a174c feat(full-stack): disable notifications after 3 failures
This stops notifications from firing over and over again if they are repeatedly failing.

[TESTING -> locally with docker compose, DATABASE MIGRATION -> there is a single migration]

Issue: https://www.pivotaltracker.com/story/show/b144646649n

- [ ] It works!
- [ ] Comments provide sufficient explanations for the next contributor
- [ ] Tests cover changes and corner cases
- [ ] Follows Quay syntax patterns and format
2017-05-19 16:58:46 -04:00
Charlton Austin
c79711b6dc feat(data): remove subdir
### Description of Changes
  This is the last step in the four phase migration of the config
2017-04-28 13:23:51 -04:00
EvB
c5b411b704 fix(migrations): fix column additional w/ boolean default 2017-04-21 11:56:13 -04:00
Evan Cordell
2661db7485 Add flag to enable trust per repo (#2541)
* Add flag to enable trust per repo

* Add api for enabling/disabling trust

* Add new LogEntryKind for changing repo trust settings
Also add tests for repo trust api

* Add `set_trust` method to repository

* Expose new logkind to UI

* Fix registry tests

* Rebase migrations and regen test.db

* Raise downstreamissue if trust metadata can't be removed

* Refactor change_repo_trust

* Add show_if to change_repo_trust endpoint
2017-04-15 08:26:33 -04:00
Joseph Schorr
68331859b0 Add backfill for repository search score table 2017-04-13 12:30:44 -04:00
Joseph Schorr
df3f47c79a Add a RepositorySearchScore table and calculation to the RAC worker
This will be used in a followup PR to order search results instead of the RAC join. Currently, the join with the RAC table in search results in a lookup of ~600K rows, which causes searching to take ~6s. This PR denormalizes the data we need, as well as allowing us to score based on a wider band (6 months vs the current 1 week).
2017-04-10 14:29:02 -04:00
Joseph Schorr
b10608e277 Change teamsync config to be a UTF8 field 2017-04-03 14:16:34 -04:00
Joseph Schorr
7f0aa19292 Code cleanup and style improvements in team sync 2017-04-03 11:36:41 -04:00
Joseph Schorr
df603462b8 Add database migration for TeamSync 2017-04-03 11:31:29 -04:00
Charlton Austin
9ff189b16e fix(migration merge issue): missing .save() on migration 2017-03-28 15:17:51 -04:00
Charlton Austin
d559dc7b3e Fixing the migration path so we don't have incorrect branches. 2017-03-28 14:54:21 -04:00
Charlton Austin
e6d201e0b0 feat(build runner): added in context, dockerfile_location
this is a new feature meant to allow people to use any file as
  a dockerfile and any folder as a context directory
2017-03-28 13:55:31 -04:00
Jimmy Zelinskie
3ccf3c5f33 Merge pull request #2447 from jzelinskie/cnr-step2
CNR Step 2
2017-03-22 18:45:51 -04:00