Joseph Schorr
3a0adfcb11
Fix stream_write to properly raise an exception on failure, instead of just silently failing
...
This was causing problems for customers using georeplication over unstable storage engines
Also adds tests for stream_write and copy, to ensure we detect failure
2018-10-31 12:46:49 -04:00
Joseph Schorr
b1411d4d52
Make sure empty segments in Swift are deleted on upload cancelation
2018-09-27 15:12:04 -04:00
Kenny Lee Sin Cheong
b6336393de
Make IPResolver run the update in a separate thread
...
A separate thread will cache the results of parsing the range
file, and the IPResolver will hit the cache instead of blocking while recomputing the
ranges everytime. The thread updates every 600s, and retry every 60s on
failures.
2018-08-31 14:00:53 -04:00
Brad Ison
d3d9cca182
Upgrade Peewee to latest 3.x
...
This requires a number of small changes in the data model code, as well as additional testing.
2018-07-23 12:43:23 -04:00
Joseph Schorr
be58605f44
Move storage proxy test into the storage directory
2018-07-18 17:33:15 -04:00
Joseph Schorr
4aee811611
Move cloud storage tests to pytest
2018-07-17 16:14:23 -04:00
Joseph Schorr
7426f9e93a
Change cloudfront storage to report non-cloudfront (i.e. S3) URLs for all requests missing IP information
...
This ensures the security scanner gets S3 URLs
Fixes https://jira.coreos.com/browse/QUAY-954
2018-07-05 16:16:20 +03:00
Joseph Schorr
57523d22de
Have Swift storage delete segments when deleting dynamic large objects
...
This ensures that we reclaim the space, rather than simply deleting the manifest
Fixes https://jira.coreos.com/browse/QUAY-942
2018-05-16 16:01:49 -04:00
Joseph Schorr
d488517b36
Implement storage driver for Azure Blob Storage
2018-02-06 13:48:40 -05:00
Joseph Schorr
6cd8140c34
Pad out the segment identifier for Swift segments and change test back to string comparison
...
Makes us follow the docs closer
2017-11-28 09:46:40 +02:00
Joseph Schorr
3bbcb93977
Add additional Swift chunking tests
2017-11-28 09:46:40 +02:00
Joseph Schorr
ffaff5a27e
Add caching of get_auth call in Swift
...
Should make calls significantly faster for our Swift customers
Fixes https://jira.prod.coreos.systems/browse/QS-39
2017-10-19 11:34:13 -04:00
josephschorr
7cfe66336a
Merge pull request #2874 from coreos-inc/joseph.schorr/QS-22/swift-copy-to
...
Add missing `copy_to` method to Swift
2017-10-09 17:54:02 -04:00
Joseph Schorr
ab886226f3
Add missing copy_to method to Swift
...
Along with tests
Fixes https://coreosdev.atlassian.net/browse/QS-22
2017-10-09 17:17:36 -04:00
Joseph Schorr
61f22621d6
Fix unbound variable in new cloud fronted storage
...
Fixes https://sentry.io/coreos/backend-staging/issues/363196446/
2017-10-06 13:54:49 -04:00
Joseph Schorr
05b4a7d457
Add worker to update ipresolver data files every few hours
2017-09-28 14:40:59 -04:00
Joseph Schorr
010dda2c52
Add CloudFrontedS3Storage, which redirects to CloudFront for non-S3 ips
2017-09-28 14:40:58 -04:00
Joseph Schorr
688312bb29
Check the returned ETag in Swift when streaming data
...
This ensures that if Swift mis-writes the data, we know immediately and can fail
2017-06-27 16:21:37 +03:00
Joseph Schorr
6b272cf7e6
Move Swift tests to pytest
2017-06-27 16:05:32 +03:00
