Commit graph

1204 commits

Author SHA1 Message Date
Derek McGowan
95a5758641 Merge pull request #585 from stevvooe/content-service-ignore-eof
services/content: ignore io.EOF after successful commit
2017-03-01 10:14:56 -08:00
Ian Campbell
186a9a2acd Return DeleteResponse from ContainerService.Delete
The message was defined but the method was returning empty, plumb through the
result from the shim layer.

Compile tested only.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-03-01 14:59:29 +00:00
Ian Campbell
eae8ca69b7 Do not leak open fd to config.json in newBundle
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-03-01 11:15:10 +00:00
Stephen J Day
5da4e1d0d2 services/content: move service client into package
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-28 17:12:24 -08:00
Stephen Day
4793f968e5 Merge pull request #586 from stevvooe/dist-grpc-connect
cmd/dist: add global connect-timeout for GRPC
2017-02-28 17:12:00 -08:00
Michael Crosby
20a1222aa9 Merge pull request #587 from stevvooe/content-service-module
cmd/containerd: set correct module on interceptor for content
2017-02-28 16:50:58 -08:00
Stephen J Day
0c00d4a64c
cmd/containerd: set correct module on interceptor for content
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-28 16:47:41 -08:00
Stephen J Day
d3fdfd85f4
services/content: ignore io.EOF after successful commit
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-28 16:46:01 -08:00
Stephen J Day
d61d0b5aef
cmd/dist: add global connect-timeout for GRPC
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-28 16:43:08 -08:00
Stephen Day
317b884110 Merge pull request #576 from stevvooe/seek-reset-handling
content: allow reset via Truncate
2017-02-28 10:42:21 -08:00
Stephen J Day
d99756a8a2
content: allow reset via Truncate
To make restarting after failed pull less racy, we define `Truncate(size
int64) error` on `content.Writer` for the zero offset. Truncating a
writer will dump any existing data and digest state and start from the
beginning. All subsequent writes will start from the zero offset.

For the service, we support this by defining the behavior for a write
that changes the offset. To keep this narrow, we only support writes out
of order at the offset 0, which causes the writer to dump existing data
and reset the local hash.

This makes restarting failed pulls much smoother when there was a
previously encountered error and the source doesn't support arbitrary
seeks or reads at arbitrary offsets. By allowing this to be done while
holding the write lock on a ref, we can restart the full download
without causing a race condition.

Once we implement seeking on the `io.Reader` returned by the fetcher,
this will be less useful, but it is good to ensure that our protocol
properly supports this use case for when streaming is the only option.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-28 10:40:02 -08:00
Stephen Day
06701b23cb Merge pull request #577 from stevvooe/content-service-delete
api/services/content: define delete method
2017-02-28 10:25:37 -08:00
Kunal Kushwaha
8bbce4b80f Testcase for Transitivity of snapshot layers.
Given snapshots A <- B <- C,
B is the parent of C and A is a transitive parent of C

Signed-off-by: Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp>
2017-02-28 18:35:03 +09:00
Stephen Day
9743a1531d Merge pull request #575 from dmcgowan/snapshot-test-checks
Update basic snapshotter test
2017-02-27 20:09:22 -08:00
Stephen J Day
706c629354
api/services/content: define delete method
Allow deletion of content over the GRPC interface. For now, we are going
with a model that conducts reference management outside of the content
store, in the metadata store but this design is valid either way.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-27 20:06:29 -08:00
Derek McGowan
78e4c590a0 Merge pull request #574 from kunalkushwaha/testsuite-PrepareStatActive
Test cases added for to test Stat() Interface.
2017-02-27 19:26:21 -08:00
Kunal Kushwaha
d9dd68ef5d Testcases added for to test Stat() Interface.
Two test cases added
- Prepare, Stat on new layer, should return Active layer.
- Prepare & Commit , Stat on new layer, should return Committed layer

Signed-off-by: Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp>
2017-02-28 11:19:32 +09:00
Derek McGowan
b8855a8fa6 Update basic snapshotter test
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-02-27 16:49:02 -08:00
Stephen Day
be20bb1eb8 Merge pull request #566 from samuelkarp/snapshot-design
design: Update snapshots.md with current design
2017-02-27 14:55:28 -08:00
Stephen Day
4edcaa32be Merge pull request #532 from dmcgowan/snapshot-overlay-view
Snapshot overlay view
2017-02-27 13:53:27 -08:00
Derek McGowan
c48e9a763f Allow ro mounts without a parent
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-02-27 13:39:52 -08:00
Derek McGowan
eeb88554ac snapshot/overlay: use readonly bindmount for single parent view
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-02-27 13:26:03 -08:00
Derek McGowan
1723444ba2 snapshot/overlay: prevent committing view
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-02-27 13:26:03 -08:00
Derek McGowan
ca56448263 snapshot/overlay: implement view
Allow creating actives without an upper directory for
capturing changes. Actives without the upper directory
will always be mounted read only. Read only actives
must have a parent.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-02-27 13:26:03 -08:00
Michael Crosby
5bdfeead5e Merge pull request #567 from samuelkarp/snapshotter-context
snapshot: Add context to Snapshotter interface
2017-02-27 09:53:47 -08:00
Michael Crosby
f4715533da Merge pull request #569 from crosbymichael/linux-builtin
Change linux runtime(w shim) to builtin
2017-02-27 09:51:47 -08:00
Samuel Karp
b471a84885 rootfs: Add context to Prepare
Signed-off-by: Samuel Karp <skarp@amazon.com>
2017-02-26 23:53:29 -08:00
Stephen Day
c2c057a5db Merge pull request #572 from stevvooe/fetcher-concurrency-bug
cmd/dist/fetch: address subtle concurrency bug
2017-02-24 18:41:38 -08:00
Stephen J Day
2e0c92b168
cmd/dist/fetch: address subtle concurrency bug
When using the fetcher concurrently, the loop modifying the closed
`base` parameter was causing urls from different digests to be returned
randomly. We copy the the value and then modify it to make it work
correctly.

Luckily, we are using content addressable storage or this would have
been undetectable.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-24 18:31:26 -08:00
Stephen Day
0f76e0a5b3 Merge pull request #568 from crosbymichael/summit-report
Add report for feb 24
2017-02-24 16:47:44 -08:00
Michael Crosby
e04df4e3e5 Merge pull request #571 from stevvooe/use-init-func
cmd/dist: consistently replace version string
2017-02-24 16:33:13 -08:00
Michael Crosby
f10e473aac Merge pull request #570 from stevvooe/close-writer
content: close writer after opening
2017-02-24 16:28:25 -08:00
Stephen J Day
1cdf9dc834
cmd/dist: consistently replace version string
A previous PR placed the version string replacement in the `init`
function in the other commands. This makes this same change consistently
in the `dist` tool.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-24 16:09:19 -08:00
Stephen J Day
850f8addc2
content: close writer after opening
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-24 16:08:31 -08:00
Michael Crosby
e693cad926 Change linux runtime(w shim) to builtin
Because of the plugin findings and having the default runtime builtin
this makes it much better for development and testing.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-02-24 15:50:59 -08:00
Michael Crosby
eaf11039b5 Add report for feb 24
This includes the summary from the containerd summit.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-02-24 15:41:36 -08:00
Samuel Karp
4382d553ea snapshot: Add context to Snapshotter interface
Signed-off-by: Samuel Karp <skarp@amazon.com>
2017-02-24 15:23:23 -08:00
Samuel Karp
92b9262f63 design: Update snapshots.md with current design
Signed-off-by: Samuel Karp <skarp@amazon.com>
2017-02-24 13:43:16 -08:00
Phil Estes
994aebc698 Merge pull request #565 from ijc25/execution-service-get-container-info
Add ContainerService.Info to get info about a single container
2017-02-24 13:37:18 -08:00
Stephen Day
8de7b6062c Merge pull request #536 from kevinetc123/patch-2
simplify code & correct some typo mistakes
2017-02-24 13:27:41 -08:00
Stephen Day
d377454659 Merge pull request #564 from mikebrow/readme-go-version-requirement
golang plugin package dependency requires 1.8
2017-02-24 11:48:46 -08:00
Ian Campbell
cc253b0f18 Add ContainerService.Info to get info about a single container
For clients which only want to know about one container this is simpler than
searching the result of execution.List.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-02-24 17:50:46 +00:00
Mike Brown
6eb6bf700f golang plugin package dependency requires 1.8
Signed-off-by: Mike Brown <brownwm@us.ibm.com>
2017-02-24 10:37:33 -06:00
kevinetc123
fa584958fc using github.com/pkg/errors & correct some typo mistakes
Signed-off-by: kevinetc123 <kaiwentan@harmonycloud.cn>
2017-02-24 10:59:11 +08:00
Daniel, Dao Quang Minh
660783cb00 Merge pull request #548 from fate-grand-order/fixed
Use errors.New() directly to output the error message
2017-02-23 11:06:04 +00:00
Stephen Day
26e9ad41d4 Merge pull request #562 from estesp/todo-fixup
Remove no longer applicable TODO comment
2017-02-22 14:31:27 -08:00
Phil Estes
81c6e437d5 Remove no longer applicable TODO
After the version PR, this TODO message no longer applies.

Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
2017-02-22 16:58:14 -05:00
Phil Estes
a463ba33fc Merge pull request #561 from stevvooe/correct-versioning
version: finish version setup
2017-02-22 13:48:05 -08:00
Michael Crosby
4bce28a4a5 Merge pull request #559 from stevvooe/content-service-cleanup
content: cleanup service and interfaces
2017-02-22 13:35:36 -08:00
Stephen J Day
c062a85782
content: cleanup service and interfaces
After implementing pull, a few changes are required to the content store
interface to make sure that the implementation works smoothly.
Specifically, we work to make sure the predeclaration path for digests
works the same between remote and local writers. Before, we were
hesitent to require the the size and digest up front, but it became
clear that having this provided significant benefit.

There are also several cleanups related to naming. We now call the
expected digest `Expected` consistently across the board and `Total` is
used to mark the expected size.

This whole effort comes together to provide a very smooth status
reporting workflow for image pull and push. This will be more obvious
when the bulk of pull code lands.

There are a few other changes to make `content.WriteBlob` more broadly
useful. In accordance with addition for predeclaring expected size when
getting a `Writer`, `WriteBlob` now supports this fully. It will also
resume downloads if provided an `io.Seeker` or `io.ReaderAt`. Coupled
with the `httpReadSeeker` from `docker/distribution`, we should only be
a lines of code away from resumable downloads.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-22 13:30:01 -08:00