vbatts/registry
1
0
Fork 0
Code Issues Pull requests Releases Wiki Activity

Spool layer uploads to remote storage

Browse source 
To smooth initial implementation, uploads were spooled to local file storage,
validated, then pushed to remote storage. That approach was flawed in that it
present easy clustering of registry services that share a remote storage
backend. The original plan was to implement resumable hashes then implement
remote upload storage. After some thought, it was found to be better to get
remote spooling working, then optimize with resumable hashes.

Moving to this approach has tradeoffs: after storing the complete upload
remotely, the node must fetch the content and validate it before moving it to
the final location. This can double bandwidth usage to the remote backend.
Modifying the verification and upload code to store intermediate hashes should
be trivial once the layer digest format has settled.

The largest changes for users of the storage package (mostly the registry app)
are the LayerService interface and the LayerUpload interface. The LayerService
now takes qualified repository names to start and resume uploads. In corallry,
the concept of LayerUploadState has been complete removed, exposing all aspects
of that state as part of the LayerUpload object. The LayerUpload object has
been modified to work as an io.WriteSeeker and includes a StartedAt time, to
allow for upload timeout policies. Finish now only requires a digest, eliding
the requirement for a size parameter.

Resource cleanup has taken a turn for the better. Resources are cleaned up
after successful uploads and during a cancel call. Admittedly, this is probably
not completely where we want to be. It's recommend that we bolster this with a
periodic driver utility script that scans for partial uploads and deletes the
underlying data. As a small benefit, we can leave these around to better
understand how and why these uploads are failing, at the cost of some extra
disk space.

Many other changes follow from the changes above. The webapp needs to be
updated to meet the new interface requirements.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
This commit is contained in:
Stephen J Day 2015-01-08 14:24:02 -08:00
parent 219bd48c24
commit ba6b774aea
6 changed files with 181 additions and 353 deletions
Download patch file Download diff file Expand all files Collapse all files

29
storage/layer.go
View file

@ -24,8 +24,7 @@ type Layer interface {
// layers.
Digest() digest.Digest
// CreatedAt returns the time this layer was created. Until we implement
// Stat call on storagedriver, this just returns the zero time.
// CreatedAt returns the time this layer was created.
CreatedAt() time.Time
}
@ -33,26 +32,22 @@ type Layer interface {
// Instances can be obtained from the LayerService.Upload and
// LayerService.Resume.
type LayerUpload interface {
io.WriteCloser
// UUID returns the identifier for this upload.
UUID() string
io.WriteSeeker
io.Closer
// Name of the repository under which the layer will be linked.
Name() string
// Offset returns the position of the last byte written to this layer.
Offset() int64
// UUID returns the identifier for this upload.
UUID() string
// TODO(stevvooe): Consider completely removing the size check from this
// interface. The digest check may be adequate and we are making it
// optional in the HTTP API.
// StartedAt returns the time this layer upload was started.
StartedAt() time.Time
// Finish marks the upload as completed, returning a valid handle to the
// uploaded layer. The final size and digest are validated against the
// contents of the uploaded layer. If the size is negative, only the
// digest will be checked.
Finish(size int64, digest digest.Digest) (Layer, error)
// uploaded layer. The digest is validated against the contents of the
// uploaded layer.
Finish(digest digest.Digest) (Layer, error)
// Cancel the layer upload process.
Cancel() error
@ -84,11 +79,11 @@ func (err ErrUnknownLayer) Error() string {
// ErrLayerInvalidDigest returned when tarsum check fails.
type ErrLayerInvalidDigest struct {
FSLayer manifest.FSLayer
Digest digest.Digest
}
func (err ErrLayerInvalidDigest) Error() string {
return fmt.Sprintf("invalid digest for referenced layer: %v", err.FSLayer.BlobSum)
return fmt.Sprintf("invalid digest for referenced layer: %v", err.Digest)
}
// ErrLayerInvalidSize returned when length check fails.