Addresses the slow query:
```
SELECT `candidates` . `id` FROM ( SELECT `t1` . `id` FROM `repositorybuild` AS `t1` WHERE ( ( ( `t1` . `phase` IN (...) ) OR ( `t1` . `started` < ? ) ) AND ( `t1` . `logs_archived` = ? ) ) LIMIT ? ) AS `candidates` ORDER BY `Rand` ( ) LIMIT ? OFFSET ?
```
While the cardinality on `logs_archived` will be low, it should also only be in the `false` state for a very small number of records, so it should make this query very, very fast.
Because we run MySQL 5.6 in production, making this field a utf-8 went outside the 767 byte max size for indexes on the `tag_repository_id_name_lifetime_end_ms` index.
TagManifests can (apparently, in very rare scenarios) share manifests with the exact same digests, so we need to support that case in the backfill worker. We also need to remove a unique constraint on the manifest column in the mapping table to support this case.
The ALTER TABLE operations previously used were causing the DB to die when run on the production TagManifest table which has 7 million rows. We instead now use new mapping tables, which is less nice, but these are temporary anyway, so hopefully we only have to deal with their ugliness for a short duration.
This change also starts passing in the manifest interface, rather than the raw data, to the model for writing.
Note that this change does *not* backfill the existing rows in to the new tables; that will occur in a followup PR. The new columns in `tagmanifest` and `tagmanifestlabel` will be used to track the backfill, as it will occur in a worker.
This change ensures that the tables in the database during migration have at least one row of "real" data, which should help catch issues in the future where we forget to set column defaults and other such schema oversights that can only be truly tested with non-empty tables
Fixes https://jira.coreos.com/browse/QUAY-913
This will prevent us from running out of auto-incrementing ID values until such time as we can upgrade to peewee 3 and change the field type to a BigInt
Fixes https://jira.coreos.com/browse/QUAY-943
This could result in "hanging" robot accounts, although that would only leak the names of said accounts. Now we delete them immediately AND we proactively delete them before replacing the namespace (just to be sure)
Instead of deleting a namespace synchronously as before, we now mark the namespace for deletion, disable it, and rename it. A worker then comes along and deletes the namespace in the background. This results in a *significantly* better user experience, as the namespace deletion operation now "completes" in under a second, where before it could take 10s of minutes at the worse.
Fixes https://jira.coreos.com/browse/QUAY-838
The byte_count field on the BlobUpload model is marked as not
nullable, but the migration to make the field a big integer removed
that restriction (#2388 :: 76de324) in the database. It's still in
the model though, which means they are out of sync. This adds a
migration to mark the field as not nullable in the database again.
The checksum field was removed from the ImageStorage model in #815,
but was never dropped from the database. This adds a migration to
drop the unused column.
there is a possibility that this will not work with postgres
Issue: https://www.pivotaltracker.com/story/show/144646649
- [ ] It works!
- [ ] Comments provide sufficient explanations for the next contributor
- [ ] Tests cover changes and corner cases
- [ ] Follows Quay syntax patterns and format
This stops notifications from firing over and over again if they are repeatedly failing.
[TESTING -> locally with docker compose, DATABASE MIGRATION -> there is a single migration]
Issue: https://www.pivotaltracker.com/story/show/b144646649n
- [ ] It works!
- [ ] Comments provide sufficient explanations for the next contributor
- [ ] Tests cover changes and corner cases
- [ ] Follows Quay syntax patterns and format
