This PR addresses some issues contributors have been having with updating
dependencies. The tarsum package had required an external google package which
is no longer required. This was being picked up as an import rewrite and
getting removed. It is no longer present, so we can just update the package and
delete the tar package.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
To make the definition of supported digests more clear, we have refactored the
digest package to have a special Algorithm type. This represents the digest's
prefix and we associated various supported hash implementations through
function calls.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
The change relies on a refactor of the upstream resumable sha256/sha512 package
that opts to register implementations with the standard library. This allows
the resumable support to be detected where it matters, avoiding unnecessary and
complex code. It also ensures that consumers of the digest package don't need
to depend on the forked sha implementations.
We also get an optimization with this change. If the size of data written to a
digester is the same as the file size, we check to see if the digest has been
verified. This works if the blob is written and committed in a single request.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
This driver implements the storagedriver.StorageDriver interface and
uses Ceph Object Storage as storage backend.
Since RADOS is an object storage and no hierarchy notion, the
following convention is used to keep the filesystem notions stored in
this backend:
* All the objects data are stored with opaque UUID names prefixed
(e.g. "blob:d3d232ff-ab3a-4046-9ab7-930228d4c164).
* All the hierarchy information are stored in rados omaps, where the
omap object identifier is the virtual directory name, the keys in
a specific are the relative filenames and the values the blob
object identifier (or empty value for a sub directory).
e.g. For the following hierarchy:
/directory1
/directory1/object1
/directory1/object2
/directory1/directory2/object3
The omap "/directory1" will contains the following key / values:
- "object1" "blob:d3d232ff-ab3a-4046-9ab7-930228d4c164"
- "object2" "blob:db2e359d-4af0-4bfb-ba1d-d2fd029866a0"
- "directory2" ""
The omap "/directory1/directory2" will contains:
- "object3" "blob:9ae2371c-81fc-4945-80ac-8bf7f566a5d9"
* The MOVE is implemented by changing the reference to a specific
blob in its parent virtual directory omap.
This driver stripes rados objects to a fixed size (e.g. 4M). The idea
is to keep small objects (as done by RBD on the top of RADOS) that
will be easily synchronized accross OSDs. The information of the
original object (i.e total size of the chunks) is stored as a Xattr
in the first chunk object.
Signed-off-by: Vincent Giersch <vincent.giersch@ovh.net>
Vendored resumable sha256/sha512 library. Digest package new exports a
resumable variant of the Digester.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Some dependencies in a PR from an older branch broke the master build. The
change adds the missing dependencies and updates some others.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
This dependency added a method to access the current standard logger. This is
required to properly configure the logger for context awareness. The plan is to
have all loggers descend from the standard logger.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
As we get closer to release, we need to ensure that builds are repeatable.
Godep provides a workable solution to managing dependencies in Go to support
this requirement. This commit should be bolstered by updates to documentation
and build configuration.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
