vbatts/quay
Archived
1
0
Fork 0
Code Issues Pull requests Releases Wiki Activity
8587 commits 2 branches 62 tags 205 MiB
Commit graph

40 commits

Author SHA1 Message Date
Joseph Schorr
57523d22de Have Swift storage delete segments when deleting dynamic large objects 
This ensures that we reclaim the space, rather than simply deleting the manifest

Fixes https://jira.coreos.com/browse/QUAY-942
 2018-05-16 16:01:49 -04:00
Joseph Schorr
6cd8140c34 Pad out the segment identifier for Swift segments and change test back to string comparison 
Makes us follow the docs closer
 2017-11-28 09:46:40 +02:00
Joseph Schorr
3bbcb93977 Add additional Swift chunking tests 2017-11-28 09:46:40 +02:00
Joseph Schorr
ffaff5a27e Add caching of get_auth call in Swift 
Should make calls significantly faster for our Swift customers

Fixes https://jira.prod.coreos.systems/browse/QS-39
 2017-10-19 11:34:13 -04:00
Joseph Schorr
ab886226f3 Add missing copy_to method to Swift 
Along with tests

Fixes https://coreosdev.atlassian.net/browse/QS-22
 2017-10-09 17:17:36 -04:00
Joseph Schorr
56fbbcf7cf Add request IP to get_direct_download_url method 2017-09-25 17:14:28 -04:00
Joseph Schorr
688312bb29 Check the returned ETag in Swift when streaming data 
This ensures that if Swift mis-writes the data, we know immediately and can fail
 2017-06-27 16:21:37 +03:00
Charlton Austin
e7b029c993 Fixing how we encode some JSON. 2016-11-28 11:37:26 -05:00
Joseph Schorr
5f99448adc Add a chunk cleanup queue for async GC of empty chunks 
Instead of having the Swift storage engine try to delete the empty chunk(s) synchronously, we simply queue them and have a worker come along after 30s to delete the empty chunks. This has a few key benefits: it is async (doesn't slow down the push code), helps deal with Swift's eventual consistency (less retries necessary) and is generic for other storage engines if/when they need this as well
 2016-11-15 15:07:41 -05:00
Joseph Schorr
59cb6bd216 Make sure to not log exceptions if Swift deletes fail 2016-11-11 14:17:32 -05:00
Joseph Schorr
29b8905051 Add retry and ignore around Swift deleting empty chunks 
Fixes #1979
 2016-10-13 12:53:59 -04:00
Joseph Schorr
4f0ddc8d6e Don't exception log for expected 404s in Swift storage 
Fixes #1850
 2016-09-20 14:07:29 -04:00
Joseph Schorr
8c67ed14e8 Delete empty swift chunks 
Fixes #1844
 2016-09-19 11:41:20 -04:00
josephschorr
630d022b56 Merge pull request #1675 from coreos-inc/fix-storage-validation 
Add missing arg to validate call and add logging
 2016-08-03 14:05:37 -04:00
Joseph Schorr
0b47fce1cb Add logging when Swift deletion fails 2016-08-03 12:16:06 -04:00
Joseph Schorr
974ab6c42c Add missing arg to validate call and add logging 2016-08-03 11:13:27 -04:00
josephschorr
83849f4320 Merge pull request #1660 from coreos-inc/storage-status 
Add storage validation to the status endpoint
 2016-08-03 11:09:48 -04:00
Joseph Schorr
770ac0016e Change validate method to work for all storages 2016-08-02 15:01:37 -04:00
Joseph Schorr
a41ccf0356 Add retry support to Swift 
Fixes #1636
 2016-07-26 09:56:00 -07:00
Joseph Schorr
cbf7c2bf44 Add better logging to blob uploads 
Fixes #1635
 2016-07-20 17:53:43 -04:00
Joseph Schorr
13a30d45b1 Add swift configuration options and default connection timeout 
Fixes #1627
 2016-07-19 17:19:11 -04:00
Jimmy Zelinskie
2b07b6d8a9 allow HEAD on ACI images 
Fixes #911.
 2016-02-12 16:28:44 -05:00
Jake Moshenko
909e7d45b7 Add a test for swift path computation 2016-01-15 15:35:04 -05:00
Jake Moshenko
c6d7eba98d Fix paths in swift driver to be more intelligible 2016-01-15 15:35:04 -05:00
Jake Moshenko
0b1951a4a4 Remove list directory from storage driver 2016-01-15 15:35:04 -05:00
Silas Sewell
2dcc1f13a6 Handle IOErrors in v2 uploads 2015-12-14 11:58:24 -05:00
Joseph Schorr
ee0eb80c8f Fix blob content types 
Fixes #990
 2015-12-04 16:13:58 -05:00
Joseph Schorr
a3ebb9028d Add full unit tests for the file-like objects and fix them 
Fixes #568
 2015-09-30 14:19:25 -04:00
Joseph Schorr
6c59161527 Add V2 storage methods to Swift storage engine 
Fixes #508
 2015-09-28 16:46:19 -04:00
Jake Moshenko
26cea9a07c Merge remote-tracking branch 'upstream/master' into python-registry-v2 2015-09-17 16:16:27 -04:00
Joseph Schorr
6f2271d0ae Add support for direct download in Swift storage engine 
Fixes #483
 2015-09-14 18:00:03 -04:00
Jake Moshenko
210ed7cf02 Merge remote-tracking branch 'upstream/master' into python-registry-v2 2015-09-04 16:32:01 -04:00
Joseph Schorr
587ef85c7f Allow users to choose the version of Swift to use 
Fixes #442
 2015-09-02 17:46:14 -04:00
Jake Moshenko
398202e6fc Implement some new methods on the storage engines. 2015-08-27 11:29:19 -04:00
Joseph Schorr
9798f871ec Always create a new connection to Swift 
Fixes #335
 2015-08-07 13:41:40 -04:00
Jake Moshenko
18100be481 Refactor the util directory to use subpackages. 2015-08-03 16:04:19 -04:00
Joseph Schorr
4333bb9e14 Implement stream_read_file for the Swift storage engine 
Note that Swift doesn't seem to have a file-like interface, so we need to wrap the generator we get back from it.

Fixes #210
 2015-07-02 17:52:43 +03:00
Joseph Schorr
375d7670a8 Explain why we re-raise ClientException in the swift storage engine 2015-05-26 16:35:12 -04:00
Joseph Schorr
7001fb05bf Add further comments on the TODO in get_direct_download_url 2015-05-26 16:34:59 -04:00
Joseph Schorr
5845e37e32 Add Swift storage library 2015-05-26 16:05:21 -04:00