Joseph Schorr
6cd8140c34
Pad out the segment identifier for Swift segments and change test back to string comparison
...
Makes us follow the docs closer
2017-11-28 09:46:40 +02:00
Joseph Schorr
3bbcb93977
Add additional Swift chunking tests
2017-11-28 09:46:40 +02:00
Joseph Schorr
ffaff5a27e
Add caching of get_auth call in Swift
...
Should make calls significantly faster for our Swift customers
Fixes https://jira.prod.coreos.systems/browse/QS-39
2017-10-19 11:34:13 -04:00
Joseph Schorr
ab886226f3
Add missing copy_to method to Swift
...
Along with tests
Fixes https://coreosdev.atlassian.net/browse/QS-22
2017-10-09 17:17:36 -04:00
Joseph Schorr
56fbbcf7cf
Add request IP to get_direct_download_url method
2017-09-25 17:14:28 -04:00
Joseph Schorr
688312bb29
Check the returned ETag in Swift when streaming data
...
This ensures that if Swift mis-writes the data, we know immediately and can fail
2017-06-27 16:21:37 +03:00
Charlton Austin
e7b029c993
Fixing how we encode some JSON.
2016-11-28 11:37:26 -05:00
Joseph Schorr
5f99448adc
Add a chunk cleanup queue for async GC of empty chunks
...
Instead of having the Swift storage engine try to delete the empty chunk(s) synchronously, we simply queue them and have a worker come along after 30s to delete the empty chunks. This has a few key benefits: it is async (doesn't slow down the push code), helps deal with Swift's eventual consistency (less retries necessary) and is generic for other storage engines if/when they need this as well
2016-11-15 15:07:41 -05:00
Joseph Schorr
59cb6bd216
Make sure to not log exceptions if Swift deletes fail
2016-11-11 14:17:32 -05:00
Joseph Schorr
29b8905051
Add retry and ignore around Swift deleting empty chunks
...
Fixes #1979
2016-10-13 12:53:59 -04:00
Joseph Schorr
4f0ddc8d6e
Don't exception log for expected 404s in Swift storage
...
Fixes #1850
2016-09-20 14:07:29 -04:00
Joseph Schorr
8c67ed14e8
Delete empty swift chunks
...
Fixes #1844
2016-09-19 11:41:20 -04:00
josephschorr
630d022b56
Merge pull request #1675 from coreos-inc/fix-storage-validation
...
Add missing arg to validate call and add logging
2016-08-03 14:05:37 -04:00
Joseph Schorr
0b47fce1cb
Add logging when Swift deletion fails
2016-08-03 12:16:06 -04:00
Joseph Schorr
974ab6c42c
Add missing arg to validate call and add logging
2016-08-03 11:13:27 -04:00
josephschorr
83849f4320
Merge pull request #1660 from coreos-inc/storage-status
...
Add storage validation to the status endpoint
2016-08-03 11:09:48 -04:00
Joseph Schorr
770ac0016e
Change validate method to work for all storages
2016-08-02 15:01:37 -04:00
Joseph Schorr
a41ccf0356
Add retry support to Swift
...
Fixes #1636
2016-07-26 09:56:00 -07:00
Joseph Schorr
cbf7c2bf44
Add better logging to blob uploads
...
Fixes #1635
2016-07-20 17:53:43 -04:00
Joseph Schorr
13a30d45b1
Add swift configuration options and default connection timeout
...
Fixes #1627
2016-07-19 17:19:11 -04:00
Jimmy Zelinskie
2b07b6d8a9
allow HEAD on ACI images
...
Fixes #911 .
2016-02-12 16:28:44 -05:00
Jake Moshenko
909e7d45b7
Add a test for swift path computation
2016-01-15 15:35:04 -05:00
Jake Moshenko
c6d7eba98d
Fix paths in swift driver to be more intelligible
2016-01-15 15:35:04 -05:00
Jake Moshenko
0b1951a4a4
Remove list directory from storage driver
2016-01-15 15:35:04 -05:00
Silas Sewell
2dcc1f13a6
Handle IOErrors in v2 uploads
2015-12-14 11:58:24 -05:00
Joseph Schorr
ee0eb80c8f
Fix blob content types
...
Fixes #990
2015-12-04 16:13:58 -05:00
Joseph Schorr
a3ebb9028d
Add full unit tests for the file-like objects and fix them
...
Fixes #568
2015-09-30 14:19:25 -04:00
Joseph Schorr
6c59161527
Add V2 storage methods to Swift storage engine
...
Fixes #508
2015-09-28 16:46:19 -04:00
Jake Moshenko
26cea9a07c
Merge remote-tracking branch 'upstream/master' into python-registry-v2
2015-09-17 16:16:27 -04:00
Joseph Schorr
6f2271d0ae
Add support for direct download in Swift storage engine
...
Fixes #483
2015-09-14 18:00:03 -04:00
Jake Moshenko
210ed7cf02
Merge remote-tracking branch 'upstream/master' into python-registry-v2
2015-09-04 16:32:01 -04:00
Joseph Schorr
587ef85c7f
Allow users to choose the version of Swift to use
...
Fixes #442
2015-09-02 17:46:14 -04:00
Jake Moshenko
398202e6fc
Implement some new methods on the storage engines.
2015-08-27 11:29:19 -04:00
Joseph Schorr
9798f871ec
Always create a new connection to Swift
...
Fixes #335
2015-08-07 13:41:40 -04:00
Jake Moshenko
18100be481
Refactor the util directory to use subpackages.
2015-08-03 16:04:19 -04:00
Joseph Schorr
4333bb9e14
Implement stream_read_file for the Swift storage engine
...
Note that Swift doesn't seem to have a file-like interface, so we need to wrap the generator we get back from it.
Fixes #210
2015-07-02 17:52:43 +03:00
Joseph Schorr
375d7670a8
Explain why we re-raise ClientException in the swift storage engine
2015-05-26 16:35:12 -04:00
Joseph Schorr
7001fb05bf
Add further comments on the TODO in get_direct_download_url
2015-05-26 16:34:59 -04:00
Joseph Schorr
5845e37e32
Add Swift storage library
2015-05-26 16:05:21 -04:00
