Commit graph

313 commits

Author SHA1 Message Date
Sylvain Baubeau
661f197f68 Retrieve all the objects using pagination
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
704e082254 Do not create objects for directories
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
f190aa4a7c Refactor segment path concatenation code
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
0807282859 Use http.StatusRequestedRangeNotSatisfiable instead of error code
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
7a5aa32a64 Use file instead of filepath as it may cause troubles on Windows
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
91d74a3ee2 Protect against deletion of objects with the same prefix
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
01686e2c07 Show distribution version in User-Agent
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
913fe195fd Do not use suite style testing for Swift specific tests
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
1b28eea232 Rename environment variables to run Swift testsuite
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
2524f300dc Check file has been opened before closing it
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
fbc74a6457 Rename DriverParameters structure to Parameters
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
80bfcb68a8 Change folder mime type to application/vnc.swift.directory
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
7b0276dce5 Add code documentation
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
5cce023aa9 Do not read segment if no padding is necessary
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
d91c4cb694 Improve 404 errors handling
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
9ab55eae39 Use only one Swift container for both files and manifests
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
a1ae7f7122 Increase default chunk size to 20M
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
davidli
062d6266cf Add support for Openstack Identity v3 API
Signed-off-by: Li Wenquan <wenquan.li@hp.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
3f9e7ed169 Use 'prefix' parameter instead of 'path' when listing files
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
1d46bb2bcc Create full folder hierarchy instead of just the top level folder
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
4e619bc9b1 Remove one level of indentation in swift path handling code
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
75ce67c469 Use mitchellh/mapstructure library to parse Swift parameters
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
ea81e208a4 Move Dynamic Large Object handling to dedicated methods
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
8a22c0f4e1 Simplify code that handles non existing manifests
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
16a49ade16 Handle error during copy of original content
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
nevermosby
9f7f23e373 Update the import path for swift driver test
Signed-off-by: Li Wenquan <wenquan.li@hp.com>
2015-07-21 23:55:09 +02:00
davidli
1f4eb7b735 Use gofmt to format the code of swift driver.
Signed-off-by: Li Wenquan <wenquan.li@hp.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
cce4956131 Add Openstack Swift storage driver
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:08 +02:00
Stephen Day
50de6d914c Merge pull request #713 from stevvooe/descriptor-size-field
Use "Size" field to describe blobs over "Length"
2015-07-17 22:06:59 -07:00
Stephen J Day
249ad3b76d Use "Size" field to describe blobs over "Length"
After consideration, we've changed the main descriptor field name to for number
of bytes to "size" to match convention. While this may be a subjective
argument, commonly we refer to files by their "size" rather than their
"length". This will match other conventions, like `(FileInfo).Size()` and
methods on `io.SizeReaderAt`. Under more broad analysis, this argument doesn't
necessarily hold up. If anything, "size" is shorter than "length".

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-07-17 17:07:11 -07:00
yuzou
feebd69d26 Close reader after the test is finished.
Signed-off-by: yuzou <zouyu7@huawei.com>
2015-07-17 14:55:31 +08:00
Stephen Day
5c6c88196d Merge pull request #705 from stevvooe/export-servejson-errors
Export ServeJSON for serving error codes
2015-07-16 14:35:27 -07:00
Brian Bland
41aadeac9a Reduces log level of auth error lines from error->warn
An error level log is already produced within app.authorized() if an
actual unexpected error occurs during authorization, so this warning
level log remains for auditability purposes, but should not be
considered an error condition.

Addresses #704

Signed-off-by: Brian Bland <brian.bland@docker.com>
2015-07-16 12:43:33 -07:00
Stephen J Day
81c21411e8 Export ServeJSON for serving error codes
This changeset provides a common http handler for serving errcodes. This should
unify http responses across webservices in the face of errors.

Several type assertions have been added, as well, to ensure the error interface
is implemented.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-07-16 12:14:14 -07:00
Richard
cd31d466e4 Allow Manifest Service to be configured with function arguments
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-07-15 12:25:16 -07:00
Stephen Day
b318e01f7d Merge pull request #651 from duglin/ErrArgs
Add ability to pass in substitution args into an Error
2015-07-15 11:22:59 -07:00
Doug Davis
db30d384e0 Add ability to pass in substitution args into an Error
Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-07-15 11:02:10 -07:00
Derek McGowan
c7d538aefa Merge pull request #699 from RichardScothern/client-manifest-etags-clean
Allow conditional fetching of manifests with the registry client.
2015-07-14 17:29:59 -07:00
Richard Scothern
caf989a572 Allow conditional fetching of manifests with the registry client.
Add a functional argument to pass a digest to (ManifestService).GetByTag().
If the digest matches an empty manifest and nil error are returned.

See 6bedf7d1cd for server implementation.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-07-14 16:52:52 -07:00
Richard Scothern
6f2f84996d Fix build when using build tag 'noresumabledigest'
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-07-14 11:14:09 -07:00
Stephen J Day
a58848a0b7 Allow single character repository names
The main goal of this changeset is to allow repository name components to
consist of a single character. The number of components allowed and the slash
separation requirements have also been clarified.

To go along with this simplification, errant constants and unneeded error types
have been removed.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-07-10 12:10:42 -06:00
Derek McGowan
970efb6ba7 Fix typo in Version doc
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-07-08 11:02:47 -07:00
Derek McGowan
376cc5fe75 Add challenge manager interface
Challenger manager interface is used to handle getting authorization challenges from an endpoint as well as extracting challenges from responses.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-07-01 15:00:25 -07:00
Derek McGowan
5a3a9c6a77 Separate version and challenge parsing from ping
Replace ping logic with individual functions to extract API version and authorization challenges. The response from a ping operation can be passed into these function. If an error occurs in parsing, the version or challenge will not be used. Sending the ping request is the responsibility of the caller.
APIVersion has been converted from a string to a structure type. A parse function was added to convert from string to the structure type.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-07-01 15:00:25 -07:00
Derek McGowan
b66ee14e62 Refactor client auth
Move client auth into a separate package.
Separate ping from the authorizer and export Challenges type.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-07-01 15:00:25 -07:00
Stephen Day
940b865bc0 Merge pull request #670 from stevvooe/remove-ipc
Remove half-baked Storage Driver IPC support
2015-06-30 19:21:54 -07:00
Jon Poler
855ecb8440 Remove timeout for http.Client in registry/client/repository.go.
Timeouts should not be a discrete period of time, because they end
up being arbitrary and may be difficult to gauge correctly against
very large Docker layers. Rather, timeouts should be set at the
transport level using the SetDeadline attribute on a net.Conn
object.

Signed-off-by: Jon Poler <jonathan.poler@apcera.com>
2015-06-30 17:35:24 -07:00
Stephen Day
dc19b5496a Merge pull request #675 from RichardScothern/fix-ctx
Pass correct context into tracer
2015-06-30 11:18:51 -07:00
Richard Scothern
fa17f9254f Pass correct context into tracer
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-06-30 10:28:14 -07:00
Stephen J Day
6167220cdd Remove half-baked Storage Driver IPC support
This removes documentation and code related to IPC based storage driver
plugins. The existence of this functionality was an original feature goal but
is now not maintained and actively confusing incoming contributions. We will
likely explore some driver plugin mechanism in the future but we don't need
this laying around in the meantime.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-06-29 16:52:12 -07:00
Richard Scothern
6d46ae5fdb Prevent the ErrUnsupportedMethod error from being returned up the stack.
It eventually causes the go http library to do a double WriteHeader()
which is an error

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-06-29 16:44:09 -07:00
Stephen Day
d745c5a2c9 Merge pull request #645 from RichardScothern/manifest-cache-headers
Cache headers for manifests.
2015-06-25 13:20:39 -07:00
Richard Scothern
6bedf7d1cd Add Etag header for manifests.
Return 304 (Not Modified) if retrieved with If-None-Match header

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-06-19 10:44:21 -07:00
Doug Davis
805b135bcc Add 'message' back to BlobTest sample json
Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-06-18 18:24:54 -07:00
Doug Davis
365de1b215 Add back in the "errors" wrapper in the Errors serialization
See: https://github.com/docker/distribution/pull/548/files#r32794064

Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-06-18 18:00:26 -07:00
Stephen Day
92e2636de0 Merge pull request #638 from duglin/MoveHTTPRC
Move challenge http status code logic
2015-06-18 12:03:29 -07:00
Doug Davis
cff1a5ffdc Move challenge http status code logic
See: d796729b6b/registry/handlers/app.go (L498)

Per the comment on line 498, this moves the logic of setting the http
status code into the serveJSON func, leaving the auth.Challenge.ServeHTTP()
func to just set the auth challenge header.

Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-06-17 18:23:55 -07:00
Stephen J Day
f9e152d912 Ensure that rados is disabled without build tag
This ensures that rados is not required when building the registry. This was
slightly tricky in that when the flags were applied, the rados package was
completely missing. This led to a problem where rados was basically unlistable
and untestable as a package. This was fixed by simply adding a doc.go file that
is included whether rados is built or not.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-06-16 20:00:02 -07:00
Stephen Day
d796729b6b Merge pull request #623 from ahmetalpbalkan/azure-vendor
storage/driver/azure: Update vendored Azure SDK
2015-06-16 17:41:38 -07:00
Ahmet Alp Balkan
5c372ded1b storage/driver/azure: Update vendored Azure SDK
This change refreshes the updated version of Azure SDK
for Go that has the latest changes.

I manually vendored the new SDK (github.com/Azure/azure-sdk-for-go)
and I removed `management/` `core/` packages manually simply because
they're not used here and they have a fork of `net/http` and `crypto/tls`
for a particular reason. It was introducing a 44k SLOC change otherwise...

This also undoes the `include_azure` flag (actually Steven removed the
driver from imports but forgot to add the build flag apparently, so the
flag wasn't really including azure. 😄 ). This also must be obsolete
now.

Fixes #620, #175.

Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
2015-06-16 17:13:44 -07:00
Olivier Gambier
fa67bab1c7 Merge pull request #608 from dmcgowan/http-basic-auth
Implementation of a basic authentication scheme using standard .htpasswd
2015-06-15 17:40:58 -07:00
Stephen Day
728ca4b391 Merge pull request #548 from duglin/MoveErrors
Move ErrorCode logic to new errcode package
2015-06-15 14:33:28 -07:00
Doug Davis
56349665b7 Round 4
Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-06-11 21:33:35 -07:00
Stephen J Day
f6ee0f46af Minor formatting fixes related to htpasswd auth
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-06-11 17:06:35 -07:00
Olivier Gambier
280b9c50ac Saner default data location
Signed-off-by: Olivier Gambier <olivier@docker.com>
2015-06-10 20:54:24 -07:00
Stephen J Day
e667be389a Rename the basic access controller to htpasswd
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-06-10 19:41:35 -07:00
Stephen J Day
14f3b07db0 Harden basic auth implementation
After consideration, the basic authentication implementation has been
simplified to only support bcrypt entries in an htpasswd file. This greatly
increases the security of the implementation by reducing the possibility of
timing attacks and other problems trying to detect the password hash type.

Also, the htpasswd file is only parsed at startup, ensuring that the file can
be edited and not effect ongoing requests. Newly added passwords take effect on
restart. Subsequently, password hash entries are now stored in a map.

Test cases have been modified accordingly.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-06-10 19:38:56 -07:00
Stephen J Day
427c457801 Refactor Basic Authentication package
This change refactors the basic authentication implementation to better follow
Go coding standards. Many types are no longer exported. The parser is now a
separate function from the authentication code. The standard functions
(*http.Request).BasicAuth/SetBasicAuth are now used where appropriate.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-06-10 19:06:38 -07:00
Dave Trombley
3504445680 Unexported function to comply with golint
Signed-off-by: Dave Trombley <dave.trombley@gmail.com>
2015-06-10 19:06:38 -07:00
Dave Trombley
fe9ca88946 Removed dashes from comments, unexported htpasswd struct
Signed-off-by: Dave Trombley <dave.trombley@gmail.com>
2015-06-10 19:06:38 -07:00
Dave Trombley
15bbde99c1 Fixed golint, gofmt warning advice.
Signed-off-by: Dave Trombley <dave.trombley@gmail.com>
2015-06-10 19:06:38 -07:00
Dave Trombley
ff67393b2b Added support for bcrypt, plaintext; extension points for other htpasswd hash methods.
Signed-off-by: Dave Trombley <dave.trombley@gmail.com>
2015-06-10 19:06:38 -07:00
Dave Trombley
d2b7988b7f Aligned formatting with gofmt
Signed-off-by: Dave Trombley <dave.trombley@gmail.com>
2015-06-10 19:06:38 -07:00
Dave Trombley
7733b6c892 Fixed WWW-Authenticate: header, added example config and import into main, fixed golint warnings
Signed-off-by: Dave Trombley <dave.trombley@gmail.com>
2015-06-10 19:06:38 -07:00
BadZen
60262521bd Implementation of a basic authentication scheme using standard .htpasswd files
Signed-off-by: BadZen <dave.trombley@gmail.com>
Signed-off-by: Dave Trombley <dave.trombley@gmail.com>
2015-06-10 19:06:37 -07:00
Stephen Day
f654e9a96e Merge pull request #599 from stevvooe/clarify-deletion-by-digest-constraint
Clarify digest in API specification
2015-06-08 19:04:53 -07:00
Stephen J Day
60967cbd6b Clarify that manifests can only be deleted by digest
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-06-04 18:59:08 -07:00
Stephen J Day
630334b304 Add more repository name validation test cases
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-06-04 16:12:35 -07:00
Doug Davis
b8b16b78f4 Round 3 - Add Register function
Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-06-02 10:01:21 -07:00
Vincent Giersch
040d7038b8 doc: coherence between requests and parameters + typo
In the request parameters lists `tag` was used instead of
`reference` present in the HTTP requests paths

Signed-off-by: Vincent Giersch <vincent.giersch@ovh.net>
2015-06-02 14:36:55 +00:00
Stephen Day
c703b9318e Merge pull request #394 from xiekeyang/feature-panic-hook
Feature: Add Hook for Web Application Panic
2015-06-01 13:23:32 -07:00
Vincent Giersch
4bc53818cb Fix rados build, remove uuid dependency
Signed-off-by: Vincent Giersch <vincent.giersch@ovh.net>
2015-06-01 09:57:40 +00:00
xiekeyang
9d7c6923c1 Feature: Web Panic Reporting via hooks
This PR is for issue of "email after registry webapp panic" #41, improving my
previous design (closed).
It use self setting up hooks, to catch panic in web application.
And, send email in hooks handle directly, to no use new http server and
handler.

Signed-off-by: xiekeyang <keyangxie@126.com>
2015-05-31 14:21:22 +00:00
Philip Misiowiec
589123441b fixed typos
Signed-off-by: Phil Misiowiec <phil.misiowiec@atlashealth.com>
2015-05-30 18:22:41 -07:00
Stephen Day
af987fd3dc Merge pull request #443 from gierschv/driver-rados
Storage Driver: Ceph Object Storage (RADOS)
2015-05-29 18:10:49 -07:00
Doug Davis
00b1e8fca0 Round 2
Make Errors a []Error

Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-05-26 17:18:32 -07:00
Doug Davis
f565d6abb7 Move ErrorCode logic to new errcode package
Make HTTP status codes match the ErrorCode by looking it up in the Descriptors

Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-05-26 13:18:54 -07:00
Stephen Day
5f553b3cfc Merge pull request #556 from stevvooe/remove-uuid-dependency
Replace uuid dependency with internal library
2015-05-26 10:39:35 -07:00
Stephen J Day
f8c0086e93 Replace uuid dependency with internal library
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-05-22 18:59:41 -07:00
Stephen Day
601960573d Merge pull request #546 from stevvooe/resumable-digest-refactor
Remove digest package's dependency on external sha implementation
2015-05-22 18:15:37 -07:00
Stephen J Day
bdaed4c789 Refactor specification of supported digests
To make the definition of supported digests more clear, we have refactored the
digest package to have a special Algorithm type. This represents the digest's
prefix and we associated various supported hash implementations through
function calls.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-05-22 18:08:23 -07:00
Stephen J Day
d4c50637f9 Better error message when failing to get AWS auth
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-05-22 16:45:45 -07:00
Stephen J Day
a0d242d9df Remove digest package's dependency on external sha implementation
The change relies on a refactor of the upstream resumable sha256/sha512 package
that opts to register implementations with the standard library. This allows
the resumable support to be detected where it matters, avoiding unnecessary and
complex code. It also ensures that consumers of the digest package don't need
to depend on the forked sha implementations.

We also get an optimization with this change. If the size of data written to a
digester is the same as the file size, we check to see if the digest has been
verified. This works if the blob is written and committed in a single request.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-05-20 23:44:08 -07:00
Stephen J Day
812c8099a6 Decouple redis dependency from blob descriptor cache
Ensure that clients can use the blob descriptor cache provider without needing
the redis package.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-05-20 18:58:10 -07:00
Tibor Vass
cd5430916f client: fix a typo preventing compilation
Signed-off-by: Tibor Vass <teabee89@gmail.com>
2015-05-20 21:06:13 -04:00
Stephen Day
e2596ca75e Merge pull request #387 from dmcgowan/registry-client-update
Add client implementation of distribution interface
2015-05-20 15:55:20 -07:00
Derek McGowan
754a8e80f2 Remove error message shortening
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-05-20 14:55:59 -07:00
Derek McGowan
aac3ce46c7 Only do auth checks for endpoints starting with v2
Changes behavior so ping doesn't happen if /v2/ is anywhere in a request path, but instead only at the beginning. This fixes attempts to ping on redirected URLs.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-05-20 13:35:23 -07:00
Derek McGowan
7e4d5eafae Update transport package to sever distribution dependency
The transport package no longer requires importing distribution for the ReadSeekCloser, instead declares its own.
Added comments on the Authenication handler in session.
Added todo on http seek reader to highlight its lack of belonging to the client transport.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-05-20 10:09:37 -07:00
Derek McGowan
3b5a2bbebc Add unauthorized error check
Add check for unauthorized error code and explicitly set the error code if the content could not be parsed.
Updated repository test for unauthorized tests and nit feedback.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-05-20 10:05:44 -07:00