Commit Graph

1236 Commits

Author SHA1 Message Date
Stephen Day be20bb1eb8 Merge pull request #566 from samuelkarp/snapshot-design
design: Update snapshots.md with current design
2017-02-27 14:55:28 -08:00
Stephen Day 4edcaa32be Merge pull request #532 from dmcgowan/snapshot-overlay-view
Snapshot overlay view
2017-02-27 13:53:27 -08:00
Derek McGowan c48e9a763f Allow ro mounts without a parent
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-02-27 13:39:52 -08:00
Derek McGowan eeb88554ac snapshot/overlay: use readonly bindmount for single parent view
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-02-27 13:26:03 -08:00
Derek McGowan 1723444ba2 snapshot/overlay: prevent committing view
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-02-27 13:26:03 -08:00
Derek McGowan ca56448263 snapshot/overlay: implement view
Allow creating actives without an upper directory for
capturing changes. Actives without the upper directory
will always be mounted read only. Read only actives
must have a parent.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-02-27 13:26:03 -08:00
Michael Crosby 5bdfeead5e Merge pull request #567 from samuelkarp/snapshotter-context
snapshot: Add context to Snapshotter interface
2017-02-27 09:53:47 -08:00
Michael Crosby f4715533da Merge pull request #569 from crosbymichael/linux-builtin
Change linux runtime(w shim) to builtin
2017-02-27 09:51:47 -08:00
Samuel Karp b471a84885 rootfs: Add context to Prepare
Signed-off-by: Samuel Karp <skarp@amazon.com>
2017-02-26 23:53:29 -08:00
Stephen Day c2c057a5db Merge pull request #572 from stevvooe/fetcher-concurrency-bug
cmd/dist/fetch: address subtle concurrency bug
2017-02-24 18:41:38 -08:00
Stephen J Day 2e0c92b168
cmd/dist/fetch: address subtle concurrency bug
When using the fetcher concurrently, the loop modifying the closed
`base` parameter was causing urls from different digests to be returned
randomly. We copy the the value and then modify it to make it work
correctly.

Luckily, we are using content addressable storage or this would have
been undetectable.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-24 18:31:26 -08:00
Stephen Day 0f76e0a5b3 Merge pull request #568 from crosbymichael/summit-report
Add report for feb 24
2017-02-24 16:47:44 -08:00
Michael Crosby e04df4e3e5 Merge pull request #571 from stevvooe/use-init-func
cmd/dist: consistently replace version string
2017-02-24 16:33:13 -08:00
Michael Crosby f10e473aac Merge pull request #570 from stevvooe/close-writer
content: close writer after opening
2017-02-24 16:28:25 -08:00
Stephen J Day 1cdf9dc834
cmd/dist: consistently replace version string
A previous PR placed the version string replacement in the `init`
function in the other commands. This makes this same change consistently
in the `dist` tool.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-24 16:09:19 -08:00
Stephen J Day 850f8addc2
content: close writer after opening
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-24 16:08:31 -08:00
Michael Crosby e693cad926 Change linux runtime(w shim) to builtin
Because of the plugin findings and having the default runtime builtin
this makes it much better for development and testing.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-02-24 15:50:59 -08:00
Michael Crosby eaf11039b5 Add report for feb 24
This includes the summary from the containerd summit.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-02-24 15:41:36 -08:00
Samuel Karp 4382d553ea snapshot: Add context to Snapshotter interface
Signed-off-by: Samuel Karp <skarp@amazon.com>
2017-02-24 15:23:23 -08:00
Samuel Karp 92b9262f63 design: Update snapshots.md with current design
Signed-off-by: Samuel Karp <skarp@amazon.com>
2017-02-24 13:43:16 -08:00
Phil Estes 994aebc698 Merge pull request #565 from ijc25/execution-service-get-container-info
Add ContainerService.Info to get info about a single container
2017-02-24 13:37:18 -08:00
Stephen Day 8de7b6062c Merge pull request #536 from kevinetc123/patch-2
simplify code & correct some typo mistakes
2017-02-24 13:27:41 -08:00
Stephen Day d377454659 Merge pull request #564 from mikebrow/readme-go-version-requirement
golang plugin package dependency requires 1.8
2017-02-24 11:48:46 -08:00
Ian Campbell cc253b0f18 Add ContainerService.Info to get info about a single container
For clients which only want to know about one container this is simpler than
searching the result of execution.List.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-02-24 17:50:46 +00:00
Mike Brown 6eb6bf700f golang plugin package dependency requires 1.8
Signed-off-by: Mike Brown <brownwm@us.ibm.com>
2017-02-24 10:37:33 -06:00
kevinetc123 fa584958fc using github.com/pkg/errors & correct some typo mistakes
Signed-off-by: kevinetc123 <kaiwentan@harmonycloud.cn>
2017-02-24 10:59:11 +08:00
Daniel, Dao Quang Minh 660783cb00 Merge pull request #548 from fate-grand-order/fixed
Use errors.New() directly to output the error message
2017-02-23 11:06:04 +00:00
Stephen Day 26e9ad41d4 Merge pull request #562 from estesp/todo-fixup
Remove no longer applicable TODO comment
2017-02-22 14:31:27 -08:00
Phil Estes 81c6e437d5 Remove no longer applicable TODO
After the version PR, this TODO message no longer applies.

Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
2017-02-22 16:58:14 -05:00
Phil Estes a463ba33fc Merge pull request #561 from stevvooe/correct-versioning
version: finish version setup
2017-02-22 13:48:05 -08:00
Michael Crosby 4bce28a4a5 Merge pull request #559 from stevvooe/content-service-cleanup
content: cleanup service and interfaces
2017-02-22 13:35:36 -08:00
Stephen J Day c062a85782
content: cleanup service and interfaces
After implementing pull, a few changes are required to the content store
interface to make sure that the implementation works smoothly.
Specifically, we work to make sure the predeclaration path for digests
works the same between remote and local writers. Before, we were
hesitent to require the the size and digest up front, but it became
clear that having this provided significant benefit.

There are also several cleanups related to naming. We now call the
expected digest `Expected` consistently across the board and `Total` is
used to mark the expected size.

This whole effort comes together to provide a very smooth status
reporting workflow for image pull and push. This will be more obvious
when the bulk of pull code lands.

There are a few other changes to make `content.WriteBlob` more broadly
useful. In accordance with addition for predeclaring expected size when
getting a `Writer`, `WriteBlob` now supports this fully. It will also
resume downloads if provided an `io.Seeker` or `io.ReaderAt`. Coupled
with the `httpReadSeeker` from `docker/distribution`, we should only be
a lines of code away from resumable downloads.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-22 13:30:01 -08:00
Stephen J Day 935144fadd
version: finish version setup
This setup will now correctly set the version number from the git tag.
When using `--version`, we will see the binary name, the package it was
built from and a git hash based on the tag:

```console
$./bin/dist -v
./bin/dist github.com/docker/containerd 0b45d91.m
```

Note that in the above example, if we set a tag of `v1.0.0-dev`, that
will show up in the version number, as follows:

```console
$./bin/dist -v
./bin/dist github.com/docker/containerd v1.0.0-dev
```

Once commits are made past that tag, the version number will be
expressed relative to that tag and include a git hash:

```console
$./bin/dist -v
./bin/dist github.com/docker/containerd v1.0.0-dev-1-g7953e96.m
```

Some these examples include a `.m` postfix. This indicates that the
binary was build from a source tree with local modifications.

We can add a dev tag to start getting 1.0 version numbers for test
builds.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-22 13:16:06 -08:00
Kenfe-Mickaël Laventure 0a5544d8c4 Merge pull request #556 from crosbymichael/plugins
Use go 1.8 plugins for extending core functionality
2017-02-22 13:12:08 -08:00
Stephen Day 0b45d91340 Merge pull request #560 from stevvooe/update-image-spec
vendor: update oci image spec dependency
2017-02-22 11:23:02 -08:00
Michael Crosby fceafeb4d6 Remvoe go1.7 from travis
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-02-22 10:28:05 -08:00
Michael Crosby 401edf5728 Merge pull request #557 from estesp/snapshot-edits
Some language cleanup in snapshots design doc
2017-02-22 09:37:18 -08:00
Stephen J Day 24397e7169
vendor: update oci image spec dependency
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-22 00:52:35 -08:00
Phil Estes 25f66bd21f
Some language cleanup in snapshots design doc
Mostly minor changes for readability and a few corrections to words or
word use.

Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
2017-02-21 23:09:33 -05:00
fate-grand-order 08405824ad Use errors.New() directly to output the error message
Signed-off-by: fate-grand-order <chenjg@harmonycloud.cn>
2017-02-22 10:53:16 +08:00
Michael Crosby 3101be93bc Load runtimes dynamically via go1.8 plugins
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

Add registration for more subsystems via plugins

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

Move content service to separate package

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-02-21 16:29:46 -08:00
Stephen Day b7805198b1 Merge pull request #555 from stevvooe/remove-cli-dependence-content-store-main
cmd/containerd: remove cli.Context dependence for content.Store
2017-02-21 16:13:10 -08:00
Stephen J Day baa307cd4b
cmd/containerd: remove cli.Context dependence for content.Store
We've moved to using the config, directly. This removes the argument and
gets rid of a few extra lines of code.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-21 15:27:12 -08:00
Michael Crosby 8ae905b92b Merge pull request #520 from stevvooe/content-service
api/services/content: define the content service
2017-02-21 15:03:13 -08:00
Stephen J Day e6efb397cf
cmd/dist: port commands over to use GRPC content store
Following from the rest of the work in this branch, we now are porting
the dist command to work directly against the containerd content API.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-21 13:10:31 -08:00
Stephen J Day 621164bc84
content: refactor content store for API
After iterating on the GRPC API, the changes required for the actual
implementation are now included in the content store. The begin change
is the move to a single, atomic `Ingester.Writer` method for locking
content ingestion on a key. From this, comes several new interface
definitions.

The main benefit here is the clarification between `Status` and `Info`
that came out of the GPRC API. `Status` tells the status of a write,
whereas `Info` is for querying metadata about various blobs.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-21 13:10:22 -08:00
Stephen J Day baaf7543dc
api/services/content: define the content service
Bring the content service into the containerd API. This allows the
content store to be coordinated in the containerd daemon with minimal
effort. For the most part, this API follows the conventions and behavior
of the existing content store implementation with a few caveats.
Specifically, we remove the object oriented transaction mechanism in
favor of a very rich `Write` call.

Pains are taken to reduce race conditions around when having multiple
writers to a single piece of content. Clients should be able to race
towards getting a write lock on a reference, then wait on each other.

For the most part, this should be generically pluggable to allow
implementations of the content store to be swapped out.

We'll follow this up with an implementation to validate the model.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-21 13:10:10 -08:00
Stephen J Day 4cb31d9615
vendor: update gogo/protobuf dependency
Updates to the gogo/protobuf dependency are required to correctly
generate time types. We also remove an unused windows dependency.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-21 13:09:58 -08:00
Stephen Day 3fdd1a688d Merge pull request #554 from alexellis/http_to_https_readme
Update short url http url to https.
2017-02-21 11:54:23 -08:00
Michael Crosby f01434283c Merge pull request #553 from crosbymichael/config
Add config file
2017-02-21 11:52:08 -08:00