vbatts/registry
1
0
Fork 0
Code Issues Pull requests Releases Wiki Activity
685 commits 18 branches 0 tags 22 MiB
Commit graph

46 commits

Author SHA1 Message Date
Stefan Majewsky
ea5abc9935 wait for DLO segments to show up when Close()ing the writer 
Not just when Commit()ing the result. This fixes some errors I observed
when the layer (i.e. the DLO) is Stat()ed immediately after closing,
and reports the wrong file size because the container listing is not
yet up-to-date.

Signed-off-by: Stefan Majewsky <stefan.majewsky@sap.com>
 2016-04-19 14:31:49 +02:00
Stefan Majewsky
84aa48b56c detect outdated container listings during Stat() and getAllSegments() 
Signed-off-by: Stefan Majewsky <stefan.majewsky@sap.com>
 2016-04-06 15:21:27 +02:00
Stefan Majewsky
b015bf0676 don't swallow errors in Swift driver's GetContent() 
In 000dec3c6f, which was only intended to
be a refactoring commit, the behavior of this block subtly changed so
that unknown types of errors would be swallowed instead of propagated.

I noticed this while investigating an error similar to #1539 aka
docker/docker#21290. It appears that during GetContent() for a
hashstate, the Swift proxy produces an error. Since this error was
silently swallowed, an empty []byte is used to restart the hash, then
producing the digest of the empty string instead of the layer's digest.

This PR will not fix the issue, but it should make the actual error more
visible by propagating it into `blobWriter#resumeDigest' and
'blobWriter#validateBlob', respectively.

Signed-off-by: Stefan Majewsky <stefan.majewsky@sap.com>
 2016-03-30 16:11:29 +02:00
Brian Bland
490a2f5a55 Updates Swift driver to support new storagedriver.FileWriter interface 
Signed-off-by: Brian Bland <brian.bland@docker.com>
 2016-03-08 16:38:39 -08:00
davidli
aa08ced9d7 driver/swift: treat empty object list as a PathNotFoundError 
Swift returns an empty object list when trying to read a non-existing object path, treat it as a
PathNotFoundError when trying to list a non existing virtual directory.

Signed-off-by: David li <wenquan.li@hpe.com>
 2015-12-08 12:04:04 -08:00
Sylvain Baubeau
b596464d38 Use bulk delete to remove segments in Swift driver 
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
 2015-12-04 12:16:49 +01:00
Richard Scothern
aaf448b2de Merge pull request #1141 from lebauce/swift-no-missing-segment 
Ensure read after write for segments
 2015-11-10 10:39:43 -08:00
Aaron Lehmann
accfa46f9b Fix empty delete requests with Swift fs driver 
The Delete method lists objects under the given path and tries to delete
all of them with a bulk delete request. If the path has no objects
underneath it, the body of this request will be empty, which causes
HTTP-level issues. Specifically, Go's HTTP client senses the empty
request buffer and doesn't include a Content-Length, which causes the
Swift server to fail the request.

This commit fixes the problem by avoiding sending empty bulk delete
requests. This is the correct thing to do anyway, since there's no
reason to request deletion of zero objects.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
 2015-11-09 15:33:05 -08:00
amitshukla
7840a5bc8f Fix for issue 664: https://github.com/docker/distribution/issues/664 
Errors thrown by storage drivers don't have the name of the driver, causing user
confusion about whether the error is coming from Docker or from a storage driver.
This change adds the storage driver name to each error message.

This required changing ErrUnsupportedDriver to a type, leading to code changes
whenever ErrUnsupportedDriver is used.  The tests check whether the driver name
appears in the error message.

Signed-off-by: Amit Shukla <amit.shukla@docker.com>
 2015-11-03 11:19:17 -08:00
Sylvain Baubeau
34c1d0ed50 Ensure read after write for segments 
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
 2015-11-03 17:09:52 +01:00
Sylvain Baubeau
11546b5309 Add support for temporary URL for Swift driver 
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
 2015-11-03 09:59:50 +01:00
Richard Scothern
1039e2dc26 Merge pull request #950 from jackpgriffin/master 
Skip creating swift container if already exists
 2015-10-01 10:15:54 -07:00
Jack Griffin
8ceca304b0 Skip creating swift container if already exists 
Signed-off-by: Jack Griffin <jackpg14@gmail.com>
 2015-09-04 18:53:17 +00:00
Hua Wang
8f5f6a4e59 Add TrustId parameter to swift driver 
github/ncw/swift has added support for trust, so let's add it.

Signed-off-by: Hua Wang <wanghua.humble@gmail.com>
 2015-09-02 15:05:29 +08:00
davidli
b2935158b2 Remove IPC support from test file 
Signed-off-by: Li Wenquan <wenquan.li@hp.com>
 2015-07-21 23:55:10 +02:00
Sylvain Baubeau
81765f8cbb Catch either missing containers or objects 
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
 2015-07-21 23:55:10 +02:00
Sylvain Baubeau
52d28ec81a Do not use Swift server side copy for manifests to handle >5G files 
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
 2015-07-21 23:55:10 +02:00
Sylvain Baubeau
000dec3c6f Inline Swift errors handling 
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
 2015-07-21 23:55:10 +02:00
Sylvain Baubeau
661f197f68 Retrieve all the objects using pagination 
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
 2015-07-21 23:55:09 +02:00
Sylvain Baubeau
704e082254 Do not create objects for directories 
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
 2015-07-21 23:55:09 +02:00
Sylvain Baubeau
f190aa4a7c Refactor segment path concatenation code 
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
 2015-07-21 23:55:09 +02:00
Sylvain Baubeau
0807282859 Use http.StatusRequestedRangeNotSatisfiable instead of error code 
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
 2015-07-21 23:55:09 +02:00
Sylvain Baubeau
7a5aa32a64 Use file instead of filepath as it may cause troubles on Windows 
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
 2015-07-21 23:55:09 +02:00
Sylvain Baubeau
91d74a3ee2 Protect against deletion of objects with the same prefix 
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
 2015-07-21 23:55:09 +02:00
Sylvain Baubeau
01686e2c07 Show distribution version in User-Agent 
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
 2015-07-21 23:55:09 +02:00
Sylvain Baubeau
913fe195fd Do not use suite style testing for Swift specific tests 
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
 2015-07-21 23:55:09 +02:00
Sylvain Baubeau
1b28eea232 Rename environment variables to run Swift testsuite 
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
 2015-07-21 23:55:09 +02:00
Sylvain Baubeau
2524f300dc Check file has been opened before closing it 
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
 2015-07-21 23:55:09 +02:00
Sylvain Baubeau
fbc74a6457 Rename DriverParameters structure to Parameters 
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
 2015-07-21 23:55:09 +02:00
Sylvain Baubeau
80bfcb68a8 Change folder mime type to application/vnc.swift.directory 
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
 2015-07-21 23:55:09 +02:00
Sylvain Baubeau
7b0276dce5 Add code documentation 
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
 2015-07-21 23:55:09 +02:00
Sylvain Baubeau
5cce023aa9 Do not read segment if no padding is necessary 
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
 2015-07-21 23:55:09 +02:00
Sylvain Baubeau
d91c4cb694 Improve 404 errors handling 
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
 2015-07-21 23:55:09 +02:00
Sylvain Baubeau
9ab55eae39 Use only one Swift container for both files and manifests 
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
 2015-07-21 23:55:09 +02:00
Sylvain Baubeau
a1ae7f7122 Increase default chunk size to 20M 
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
 2015-07-21 23:55:09 +02:00
davidli
062d6266cf Add support for Openstack Identity v3 API 
Signed-off-by: Li Wenquan <wenquan.li@hp.com>
 2015-07-21 23:55:09 +02:00
Sylvain Baubeau
3f9e7ed169 Use 'prefix' parameter instead of 'path' when listing files 
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
 2015-07-21 23:55:09 +02:00
Sylvain Baubeau
1d46bb2bcc Create full folder hierarchy instead of just the top level folder 
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
 2015-07-21 23:55:09 +02:00
Sylvain Baubeau
4e619bc9b1 Remove one level of indentation in swift path handling code 
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
 2015-07-21 23:55:09 +02:00
Sylvain Baubeau
75ce67c469 Use mitchellh/mapstructure library to parse Swift parameters 
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
 2015-07-21 23:55:09 +02:00
Sylvain Baubeau
ea81e208a4 Move Dynamic Large Object handling to dedicated methods 
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
 2015-07-21 23:55:09 +02:00
Sylvain Baubeau
8a22c0f4e1 Simplify code that handles non existing manifests 
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
 2015-07-21 23:55:09 +02:00
Sylvain Baubeau
16a49ade16 Handle error during copy of original content 
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
 2015-07-21 23:55:09 +02:00
nevermosby
9f7f23e373 Update the import path for swift driver test 
Signed-off-by: Li Wenquan <wenquan.li@hp.com>
 2015-07-21 23:55:09 +02:00
davidli
1f4eb7b735 Use gofmt to format the code of swift driver. 
Signed-off-by: Li Wenquan <wenquan.li@hp.com>
 2015-07-21 23:55:09 +02:00
Sylvain Baubeau
cce4956131 Add Openstack Swift storage driver 
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
 2015-07-21 23:55:08 +02:00