Commit graph

4707 commits

Author SHA1 Message Date
Stephen Day
03406ab708 Merge pull request #73 from BrianBland/godeps
Updates Godeps to a compiling/stable state
2015-01-16 14:05:16 -08:00
Brian Bland
e1ab3443db Updates Godeps to a compiling/stable state
Includes newer docker/libtrust and crowdmob/goamz
2015-01-16 13:56:37 -08:00
Stephen Day
81ea264dea Merge pull request #70 from BrianBland/go-yaml-dep
Remove dependency on BrianBland/yaml fork
2015-01-16 13:55:55 -08:00
Brian Bland
24155c0431 Remove dependency on BrianBland/yaml fork
Switched back to only using the go-yaml/yaml mainline repo.
Fixes #69
2015-01-16 11:36:25 -08:00
Stephen Day
10c9dfb5a6 Merge pull request #67 from BrianBland/layerhandler-http-method
Forwards http method to the layerhandler for layer serving
2015-01-16 10:54:20 -08:00
Brian Bland
bd9f3702f7 DelegateLayerHandler now uses http method in url generation 2015-01-15 18:15:26 -08:00
Derek McGowan
9c24fc93ad Add token cache
Token cache prevents the need to get a new token for every registry interaction.
Since the tokens are short lived, the cache expires after only a minute.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-01-15 14:05:06 -08:00
Derek McGowan
5bf94a6438 Cleanup v2 session to require endpoint
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-01-15 14:05:06 -08:00
Derek McGowan
735a112415 Fix list tags
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-01-15 14:05:06 -08:00
Derek McGowan
e5744a3bad Refactor from feedback
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-01-15 14:05:06 -08:00
Josh Hawn
826bde851b Add Tarsum Calculation during v2 Pull operation
While the v2 pull operation is writing the body of the layer blob to disk
it now computes the tarsum checksum of the archive before extracting it to
the backend storage driver. If the checksum does not match that from the
image manifest an error is raised.

Also adds more debug logging to the pull operation and fixes existing test
cases which were failing. Adds a reverse lookup constructor to the tarsum
package so that you can get a tarsum object using a checksum label.

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
2015-01-15 14:05:05 -08:00
Stephen J Day
6f09abd5c9 Correctly check and propagate errors in v2 session
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-01-15 14:05:05 -08:00
Derek McGowan
22c7328529 Get token on each request
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2015-01-15 14:05:05 -08:00
Derek McGowan
6f36ce3a01 Allow private V2 registry endpoints
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2015-01-15 14:05:05 -08:00
Derek McGowan
751a1a8dd0 Update push and pull to registry 2.1 specification
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2015-01-15 14:05:05 -08:00
Stephen J Day
ee1e1abb15 Remove dependencies on registry packages
Because docker core cannot vendor non-master Go dependencies, we need to remove
dependencies on registry package. The definition of digest.Digest has been
changed to a string and the regular expressions have been ported from
docker-registry/common library.

We'll likely change this be dependent on the registry in the future when the
API stabilizies and use of the master branch becomes the norm.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-01-15 14:05:05 -08:00
Stephen J Day
2fcad2a10f Registry V2 HTTP route and error code definitions
This package, ported from next-generation docker regsitry, includes route and
error definitions. These facilitate compliant V2 client implementation. The
portions of the HTTP API that are included in this package are considered to be
locked down and should only be changed through a careful change proposal.
Descriptor definitions package layout may change without affecting API behavior
until the exported Go API is ready to be locked down.

When the new registry stabilizes and becomes the master branch, this package
can be vendored from the registry.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-01-15 14:05:05 -08:00
Josh Hawn
e256a0e0bc Update token response handling
Registry authorization token is now taken from the response body rather than
the repsonse header.

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
2015-01-15 14:05:05 -08:00
Derek McGowan
24895820bd Update push to use mount blob endpoint
Using mount blob prevents repushing images which have already been uploaded

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-01-15 14:05:05 -08:00
Derek McGowan
06d0ef4179 Push flow
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-01-15 14:05:05 -08:00
Josh Hawn
6b400cd63c Adds support for v2 registry login
summary of changes:

registry/auth.go
  - More logging around the login functions
  - split Login() out to handle different code paths for v1 (unchanged logic)
    and v2 (does not currently do account creation)
  - handling for either basic or token based login attempts
registry/authchallenge.go
  - New File
  - credit to Brian Bland <brian.bland@docker.com> (github: BrianBland)
  - handles parsing of WWW-Authenticate response headers
registry/endpoint.go
  - EVEN MOAR LOGGING
  - Many edits throught to make the coad less dense. Sparse code is more
    readable code.
  - slit Ping() out to handle different code paths for v1 (unchanged logic)
    and v2.
  - Updated Endpoint struct type to include an entry for authorization
    challenges discovered during ping of a v2 registry.
  - If registry endpoint version is unknown, v2 code path is first attempted,
    then fallback to v1 upon failure.
registry/service.go
  - STILL MOAR LOGGING
  - simplified the logic around starting the 'auth' job.
registry/session.go
  - updated use of a registry.Endpoint struct field.
registry/token.go
  - New File
  - Handles getting token from the parameters of a token auth challenge.
  - Modified from function written by Brian Bland (see above credit).
registry/types.go
  - Removed 'DefaultAPIVersion' in lieu of 'APIVersionUnknown = 0'`

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
2015-01-15 14:04:14 -08:00
Stephen Day
e5f0622a14 Merge pull request #64 from stevvooe/separate-signature-storage
Refactor backend storage layout to meet new requirements (addresses #25, #46)
2015-01-15 13:33:46 -08:00
Stephen Day
effa09bd77 Merge pull request #63 from AndreyKostov/add-urlfor-method-param
Add functionality to make a url signed for a HEAD request to S4 driver
2015-01-15 10:44:05 -08:00
Stephen J Day
83d62628fc Refactor storage to use new backend layout
This change refactors the storage backend to use the new path layout. To
facilitate this, manifest storage has been separated into a revision store and
tag store, supported by a more general blob store. The blob store is a hybrid
object, effectively providing both small object access, keyed by content
address, as well as methods that can be used to manage and traverse links to
underlying blobs. This covers common operations used in the revision store and
tag store, such as linking and traversal. The blob store can also be updated to
better support layer reading but this refactoring has been left for another
day.

The revision store and tag store support the manifest store's compound view of
data. These underlying stores provide facilities for richer access models, such
as content-addressable access and a richer tagging model. The highlight of this
change is the ability to sign a manifest from different hosts and have the
registry merge and serve those signatures as part of the manifest package.

Various other items, such as the delegate layer handler, were updated to more
directly use the blob store or other mechanism to fit with the changes.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-01-15 10:32:18 -08:00
Olivier Gambier
82a7101f21 Merge pull request #66 from stevvooe/autogenerate-authors-file
Autogenerate AUTHORS file for distribution repo
2015-01-15 10:07:21 -08:00
Stephen J Day
deddda5d29 Autogenerate AUTHORS file for distribution repo
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-01-14 16:25:27 -08:00
Alexander Morozov
1f98347924 Fix format calls as suggested by vet
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2015-01-14 14:12:03 -08:00
Andrey Kostov
bdd5d35622 Add functionality to make a url signed for a HEAD request to S4 driver 2015-01-14 13:37:39 -08:00
Stephen Day
d64e70b0e1 Merge pull request #59 from stevvooe/use-godep-for-dependencies
Use Godep to vendor distribution dependencies
2015-01-14 12:13:25 -08:00
Stephen J Day
3277d9fc74 Redesign path layout for backend storage
Several requirements for storing registry data have been compiled and the
backend layout has been refactored to comply. Specifically, we now store most
data as blobs that are linked from repositories. All data access is traversed
through repositories. Manifest updates are no longer destructive and support
references by digest or tag. Signatures for manifests are now stored externally
to the manifest payload to allow merging of signatures posted at different
time.

The design is detailed in the documentation for pathMapper.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-01-14 11:34:47 -08:00
Stephen J Day
10a4605ec2 Explicitly select digest method for content
Detecting tar files then falling back for calculating digests turned out to be
fairly unreliable. Likely, the implementation was broken for content that was
not a tarfile. Also, for the use case of the registry, it is really not needed.
This functionality has been removed in FromReader and FromBytes. FromTarArchive
has been added for convenience.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-01-14 10:35:42 -08:00
Stephen Day
6b18639eac Merge pull request #60 from BrianBland/s3-docs
Clarifies s3 driver "secure" flag documentation
2015-01-13 19:39:47 -08:00
Brian Bland
14c8e2b5f0 Clarifies s3 driver "secure" flag documentation
Signed-off-by: Brian Bland <brian.bland@docker.com>
2015-01-13 11:38:19 -08:00
Stephen J Day
01bc08351d Ignore Godeps for gofmt and golint
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-01-12 14:01:13 -08:00
Stephen J Day
fc2a840e8f Use Godep to vendor distribution dependencies
As we get closer to release, we need to ensure that builds are repeatable.
Godep provides a workable solution to managing dependencies in Go to support
this requirement. This commit should be bolstered by updates to documentation
and build configuration.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-01-12 14:01:04 -08:00
Stephen Day
972a95f077 Merge pull request #58 from BrianBland/s3-enable
Enables s3 driver
2015-01-12 13:41:35 -08:00
Brian Bland
16a258ce69 Enables s3 driver, limits permission check to scope of rootdirectory
Signed-off-by: Brian Bland <brian.bland@docker.com>
2015-01-12 13:34:49 -08:00
Stephen Day
df4441d03c Merge pull request #18 from AndreyKostov/ng-add-s3driver-features
Add features to the s3 driver
2015-01-12 13:10:33 -08:00
Stephen Day
e9e26bd362 Merge pull request #56 from stevvooe/blob-upload-unknown
Correctly handle missing layer upload
2015-01-12 12:14:32 -08:00
Olivier Gambier
bbc94aef39 Clarify maturity of this
Fixes https://github.com/docker/docker-registry/issues/894
2015-01-12 11:10:13 -08:00
Stephen Day
16e1c441f1 Merge pull request #57 from icecrime/groups_in_readme
Add Google Groups to README.md
2015-01-12 11:09:18 -08:00
Arnaud Porterie
cbb493e881 Add Google Groups to README.md
Signed-off-by: Arnaud Porterie <arnaud.porterie@docker.com>
2015-01-12 08:31:04 -08:00
Stephen J Day
594263a3f5 Correctly handle missing layer upload
Because we guarded the error check, nil Upload on the handler was getting
through to unexpected branches. This directly handles the missing upload
ensuring its set as expected.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-01-09 16:09:45 -08:00
Stephen J Day
c78335a023 Correctly handle missing layer upload
Because we guarded the error check, nil Upload on the handler was getting
through to unexpected branches. This directly handles the missing upload
ensuring its set as expected.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-01-09 16:09:45 -08:00
Stephen Day
887b940ec1 Merge pull request #53 from stevvooe/spool-uploads-remotely
Spool uploads remotely
2015-01-09 15:40:22 -08:00
Stephen Day
21a69f53b5 Merge pull request #53 from stevvooe/spool-uploads-remotely
Spool uploads remotely
2015-01-09 15:40:22 -08:00
Stephen J Day
cd92071caa Directly manage layerUploadState in webapp
Most of this change follows from the modifications to the storage api. The
driving factor is the separation of layerUploadState from the storage backend,
leaving it to the web application to store and update it. As part of the
updates to meet changes in the storage api, support for the size parameter has
been completely removed.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-01-09 14:51:13 -08:00
Stephen J Day
4aa7837f83 Directly manage layerUploadState in webapp
Most of this change follows from the modifications to the storage api. The
driving factor is the separation of layerUploadState from the storage backend,
leaving it to the web application to store and update it. As part of the
updates to meet changes in the storage api, support for the size parameter has
been completely removed.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-01-09 14:51:13 -08:00
Stephen J Day
fdcfc56f7b Refactor handling of hmac state packing
This refactors the hmac state token to take control of the layerUploadState
json message, which has been removed from the storage backend. It also moves
away from the concept of a LayerUploadStateStore callback object, which was
short-lived. This allows for upload offset to be managed by the web application
logic in the face of an inconsistent backend. By controlling the upload offset
externally, we reduce the possibility of misreporting upload state to a client.

We may still want to modify the way this works after getting production
experience.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-01-09 14:50:39 -08:00
Stephen J Day
2f2445a335 Refactor handling of hmac state packing
This refactors the hmac state token to take control of the layerUploadState
json message, which has been removed from the storage backend. It also moves
away from the concept of a LayerUploadStateStore callback object, which was
short-lived. This allows for upload offset to be managed by the web application
logic in the face of an inconsistent backend. By controlling the upload offset
externally, we reduce the possibility of misreporting upload state to a client.

We may still want to modify the way this works after getting production
experience.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-01-09 14:50:39 -08:00