Commit graph

9 commits

Author SHA1 Message Date
Joseph Schorr
b05ebbf2c0 Have storage replication wait up to 20 minutes before trying again
Copying a file can be a long operation, so make this configurable and far above the default 5 minutes
2017-03-21 16:58:36 -04:00
Joseph Schorr
e6ee538e15 Fix full database test script to not fail randomly
- Switches database schema creation to alembic, which solves the MySQL issue (and makes sure we test migrations as well)
- Adds a few time.sleep(1) to work around MySQL's second-precision issue when adding items to queues and then immediately retrieving them
- Disables the storage proxy tests when running against non-SQLite databases, as it causes failures with the multiple process and multiple transactions
- Changes initdb to support only populating the database, as well as fixing a few small items around the test data when working with non-SQLite data
2016-11-30 18:24:08 -05:00
Joseph Schorr
b8d2570725 Don't raise an error on duplicate placements
This can happen if two pushes are racing on the same storage.
2016-07-19 16:44:05 -04:00
Joseph Schorr
5cd793331e Fix storage replication for CAS and add tests 2016-07-12 13:46:06 -04:00
Jake Moshenko
2f626f2691 Unify the database connection lifecycle across all workers 2015-12-04 15:51:53 -05:00
Jake Moshenko
e7a6176594 Merge remote-tracking branch 'upstream/v2-phase4' into python-registry-v2 2015-10-22 16:59:28 -04:00
Jake Moshenko
ce94931540 Stop writing to deprecated columns for image data. 2015-10-22 12:14:39 -04:00
Silas Sewell
03f5fe6143 workers.storagereplication: error on LocalStorage
Ensure we don't start when LocalStorage is in the config.

Fixes #502
2015-10-20 19:04:31 -04:00
Joseph Schorr
724b1607d7 Add automatic storage replication
Adds a worker to automatically replicate data between storages and update the database accordingly
2015-09-01 14:53:32 -04:00