Commit graph

964 commits

Author SHA1 Message Date
Stephen J Day
ab08944aa7
snapshot: clarify active and committed snapshots
After receiving feedback on the `snapshot.Driver` interface, it was
found that the behavior of active and committed snapshots was confusing.
We attempt to clean this up by doing the following:

1. Define the concept of "active" and "committed" snapshots and their
lifecycle relationship. Active snapshots can be created from a parent.
Committed snapshots can only be created from active snapshots.
2. Only committed snapshots can be a parent.
3. Unify the keyspace of snapshots. For common operations, such as
removal and stat, we only have a single method that works for both
active and committed snapshots. For methods that take one or the other,
the restriction is called out. `Remove` and `Delete` are consolidated
for this purpose.
4. Define the `Info` data type to include name, parent, kind and
readonly state. This allows us to collect `Exists` and `Parent` into a
single method `Stat` and simplifies the `Walk` method, eliding `Active`.
5. The `Driver` has been renamed to `Snapshotter` due to the overuse of
the term `Driver`.

Effectively, we now have snapshots that are either active or committed.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-07 17:18:14 -08:00
Michael Crosby
42a17f9391 Merge pull request #501 from mlaventure/new-shim-continued
New shim continued
2017-02-07 15:52:08 -08:00
Phil Estes
eb2191c744 Merge pull request #493 from hqhq/specify_shim_socket
Remove duplicated close
2017-02-07 09:47:14 -08:00
Phil Estes
51f1ebfca0 Merge pull request #495 from stevvooe/btrfs-hash-once
btrfs: calculate hash key only once
2017-02-07 09:45:34 -08:00
Phil Estes
11a0c8f8cd Merge pull request #498 from kunalkushwaha/bug-profiler
closing http connection
2017-02-07 09:44:30 -08:00
Phil Estes
41fe09748c Merge pull request #499 from fate-grand-order/branch-1
fix misspell "scrutiny" in content/content.go
2017-02-07 09:43:19 -08:00
Kenfe-Mickael Laventure
b4a299e61d supervisor/shim: add exec support
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-02-07 09:25:55 -08:00
fate-grand-order
d44c8525d7 fix misspell "scrutiny" in content/content.go
Signed-off-by: fate-grand-order <chenjg@harmonycloud.cn>
2017-02-07 15:06:40 +08:00
Kunal Kushwaha
1191b209ae closing http connection
Signed-off-by: Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp>
2017-02-07 14:57:44 +09:00
Qiang Huang
81246555c9 Remove duplicated close
Signed-off-by: Qiang Huang <h.huangqiang@huawei.com>
2017-02-07 10:17:47 +08:00
Stephen J Day
74949be676
btrfs: calculate hash key only once
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-06 17:09:11 -08:00
Kenfe-Mickael Laventure
78d7e8b256 supervisor: implement monitoring
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-02-06 15:16:55 -08:00
Stephen Day
f861817a8e Merge pull request #492 from AkihiroSuda/fix-dist-ls
content: fix panic during filepath.Walk()
2017-02-06 14:27:30 -08:00
Kenfe-Mickael Laventure
31f26fed18 Move to a single Event type
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-02-06 08:46:37 -08:00
Kenfe-Mickael Laventure
8fbdf1c0d7 Remove execution/executors
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-02-06 08:38:11 -08:00
Akihiro Suda
e9510eb69c content: fix panic during filepath.Walk()
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-02-06 05:16:21 +00:00
Derek McGowan
deaa17b3c4 Merge pull request #489 from stevvooe/btrfs-driver-suite
btrfs: test btrfs snapshots with driver suite
2017-02-03 17:09:18 -08:00
Stephen J Day
aeffd4f92c
btrfs: test btrfs snapshots with driver suite
We now include btrfs in the snapshot driver test suite. This includes
the addition of parent links and name hashing into the btrfs driver.
We'll probably endup replacing this with a common metadata store, as
these relationships are generally identical between implementations.

A small bug was discovered in the delete implementation in the course
testing, so the btrfs package has been updated with a fix.

The overlay driver was modified accordingly with the btrfs driver to use
`Driver` as the exported type of each driver packge.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-03 15:54:11 -08:00
Stephen Day
127882fca7 Merge pull request #483 from dmcgowan/snapshot-diff
Add fs package
2017-02-03 12:14:33 -08:00
Derek McGowan
aaf18b5962 Rename CopyDirectory to CopyDir
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-02-03 12:08:53 -08:00
Derek McGowan
d96e6e3952 Refactor changes and test functions
Remove change type in favor of explicit change function.
Using change function makes it more difficult to unnecessarily
add to the change interface.

Update test apply functions to use an interface rather
than a function type.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-02-03 11:28:06 -08:00
Derek McGowan
65e8c07847 Update diff interface to use callback
The change computation will be done on the callers thread
and use callbacks rather than running a goroutine and
returning a channel.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-02-03 11:28:06 -08:00
Derek McGowan
aa3be3b0fe Vendor errgroup
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-02-03 11:28:06 -08:00
Derek McGowan
572dbcdbd4 Use copy file range from sysx
Use pooled buffers for copy

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-02-03 11:28:06 -08:00
Derek McGowan
bb9f6b568d Update continuity to use sysx branch
Pulls in changes for copy file range

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-02-03 11:28:06 -08:00
Derek McGowan
245495d54e Use full mode for making devices node
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-02-03 11:28:06 -08:00
Derek McGowan
bf8f37ba78 Remove incorrect and unused timespec check
Compare is using its own time check comparison and
doing byte comparison when ambiguous rather than
ignoring it like this function does.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-02-03 11:28:06 -08:00
Derek McGowan
f78105d832 Add support for devices
Address code comments from previous commit

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-02-03 11:28:06 -08:00
Derek McGowan
200cd6e877 Update snapshots to use fs package
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-02-03 11:28:06 -08:00
Derek McGowan
574862fd89 Add fs package
Add diff comparison with support for double walking
two trees for comparison or single walking a diff
tree. Single walking requires further implementation
for specific mount types.

Add directory copy function which is intended to provide
fastest possible local copy of file system directories
without hardlinking.

Add test package to make creating filesystems for
test easy and comparisons deep and informative.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-02-03 11:27:40 -08:00
Derek McGowan
5f08e609c0 Vendor continuity
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-02-03 11:27:40 -08:00
Kenfe-Mickaël Laventure
c6446b93da Merge pull request #484 from stevvooe/snapshot-drivers
[WIP] snapshot: define the snapshot driver interface
2017-02-03 07:56:12 -08:00
Stephen J Day
d0b4ce8d17
snapshot/overlay: add snapshot test suite to overlay
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-02 18:55:53 -08:00
Stephen J Day
68fd25221d
snapshot: define the snapshot driver interface
We now define the `snapshot.Driver` interface based on earlier work.
Many details of the model are worked out, such as snapshot lifecycle and
parentage of commits against "Active" snapshots.

The impetus of this change is to provide a snapshot POC that does a
complete push/pull workflow. The beginnings of a test suite for snapshot
drivers is included that we can use to verify the assumptions of
drivers. The intent is to port the existing tests over to this test
suite and start scaling contributions and test to the snapshot driver
subsystem.

There are still some details that need to be worked out, such as listing
and metadata access. We can do this activity as we further integrate
with tooling.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-02 18:06:02 -08:00
Derek McGowan
29132c4061 Merge pull request #487 from stevvooe/use-pool-in-copy
content: use buffer pool in hash copy
2017-02-02 15:42:46 -08:00
Kenfe-Mickael Laventure
6f9eda1134 api/execution: remove ProcessID from rpc calls
Now that the shim handles all container's processes the system pid is
sufficient.

Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-02-02 15:36:10 -08:00
Kenfe-Mickael Laventure
40b0b211b7 api/execution: add Container suffix to relevant rpc calls
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-02-02 14:40:23 -08:00
Stephen J Day
5ff837e2e7
content: use buffer pool in hash copy
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-02 14:27:52 -08:00
Michael Crosby
8c3158bf5f Merge pull request #485 from crosbymichael/supervisor
Port over supervisor to use grpc shim
2017-02-01 16:15:56 -08:00
Michael Crosby
f187da9485 Port over supervisor to use grpc shim
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-02-01 16:01:53 -08:00
Daniel, Dao Quang Minh
e1eeb40d1d Merge pull request #478 from kunalkushwaha/profiler
Exposing pprof and expvars interfaces on default containerd socket
2017-02-01 18:17:04 +00:00
Kunal Kushwaha
6af4f5fee8 Exposing pprof and expvars interfaces on default containerd socket
Signed-off-by: Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp>
2017-02-01 11:05:55 +09:00
Stephen Day
78b55a1301 Merge pull request #482 from samuelkarp/remove-binary-file
snapshot: Remove mistakenly-added binary file
2017-01-30 14:08:49 -08:00
Samuel Karp
3c37c9e6b1 snapshot: Remove mistakenly-added binary file
Signed-off-by: Samuel Karp <skarp@amazon.com>
2017-01-30 13:57:58 -08:00
Kenfe-Mickaël Laventure
5f28d28738 Merge pull request #477 from teddyking/mkdirall-root-dir
os.MkdirAll the containerd root dir
2017-01-30 07:34:43 -08:00
Ed King
89607a10db os.MkdirAll the containerd root dir
Signed-off-by: Ed King <ed@teddyking.co.uk>
2017-01-28 15:47:51 +00:00
Stephen Day
d7bea77cc3 Merge pull request #473 from stevvooe/dmcgowan-maintainer
MAINTAINERS: add dmcgowan
2017-01-27 13:23:31 -08:00
Michael Crosby
9278a3310f Merge pull request #475 from stevvooe/dev-report-2017-01-27
reports: development report for 2017-01-27
2017-01-27 13:15:18 -08:00
Stephen J Day
ff0ddaa28e
reports: development report for 2017-01-27
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-01-27 12:38:40 -08:00
Michael Crosby
b12308a5be Merge pull request #437 from crosbymichael/lifecycle
Add container lifecycle document
2017-01-27 12:03:53 -08:00