To make restarting after failed pull less racy, we define `Truncate(size
int64) error` on `content.Writer` for the zero offset. Truncating a
writer will dump any existing data and digest state and start from the
beginning. All subsequent writes will start from the zero offset.
For the service, we support this by defining the behavior for a write
that changes the offset. To keep this narrow, we only support writes out
of order at the offset 0, which causes the writer to dump existing data
and reset the local hash.
This makes restarting failed pulls much smoother when there was a
previously encountered error and the source doesn't support arbitrary
seeks or reads at arbitrary offsets. By allowing this to be done while
holding the write lock on a ref, we can restart the full download
without causing a race condition.
Once we implement seeking on the `io.Reader` returned by the fetcher,
this will be less useful, but it is good to ensure that our protocol
properly supports this use case for when streaming is the only option.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Allow deletion of content over the GRPC interface. For now, we are going
with a model that conducts reference management outside of the content
store, in the metadata store but this design is valid either way.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Two test cases added
- Prepare, Stat on new layer, should return Active layer.
- Prepare & Commit , Stat on new layer, should return Committed layer
Signed-off-by: Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp>
Allow creating actives without an upper directory for
capturing changes. Actives without the upper directory
will always be mounted read only. Read only actives
must have a parent.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
When using the fetcher concurrently, the loop modifying the closed
`base` parameter was causing urls from different digests to be returned
randomly. We copy the the value and then modify it to make it work
correctly.
Luckily, we are using content addressable storage or this would have
been undetectable.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
A previous PR placed the version string replacement in the `init`
function in the other commands. This makes this same change consistently
in the `dist` tool.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Because of the plugin findings and having the default runtime builtin
this makes it much better for development and testing.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
For clients which only want to know about one container this is simpler than
searching the result of execution.List.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
After implementing pull, a few changes are required to the content store
interface to make sure that the implementation works smoothly.
Specifically, we work to make sure the predeclaration path for digests
works the same between remote and local writers. Before, we were
hesitent to require the the size and digest up front, but it became
clear that having this provided significant benefit.
There are also several cleanups related to naming. We now call the
expected digest `Expected` consistently across the board and `Total` is
used to mark the expected size.
This whole effort comes together to provide a very smooth status
reporting workflow for image pull and push. This will be more obvious
when the bulk of pull code lands.
There are a few other changes to make `content.WriteBlob` more broadly
useful. In accordance with addition for predeclaring expected size when
getting a `Writer`, `WriteBlob` now supports this fully. It will also
resume downloads if provided an `io.Seeker` or `io.ReaderAt`. Coupled
with the `httpReadSeeker` from `docker/distribution`, we should only be
a lines of code away from resumable downloads.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
This setup will now correctly set the version number from the git tag.
When using `--version`, we will see the binary name, the package it was
built from and a git hash based on the tag:
```console
$./bin/dist -v
./bin/dist github.com/docker/containerd 0b45d91.m
```
Note that in the above example, if we set a tag of `v1.0.0-dev`, that
will show up in the version number, as follows:
```console
$./bin/dist -v
./bin/dist github.com/docker/containerd v1.0.0-dev
```
Once commits are made past that tag, the version number will be
expressed relative to that tag and include a git hash:
```console
$./bin/dist -v
./bin/dist github.com/docker/containerd v1.0.0-dev-1-g7953e96.m
```
Some these examples include a `.m` postfix. This indicates that the
binary was build from a source tree with local modifications.
We can add a dev tag to start getting 1.0 version numbers for test
builds.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Add registration for more subsystems via plugins
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Move content service to separate package
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
