Commit graph

1033 commits

Author SHA1 Message Date
Stephen Day
4a0c86cceb Merge pull request #516 from crosbymichael/report
Add dev report for 2-10-2017
2017-02-10 16:55:21 -08:00
Michael Crosby
a0fd5036c3 Add dev report for 2-10-2017
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-02-10 15:53:12 -08:00
Michael Crosby
89f62d4fae Merge pull request #515 from fate-grand-order/branch
Use errors.New() directly to output the error message
2017-02-10 13:55:10 -08:00
Phil Estes
43e42b4222 Merge pull request #503 from jmzwcn/master
Allow specifying the log level
2017-02-10 12:35:14 -08:00
Phil Estes
8617925271 Merge pull request #497 from kunalkushwaha/ctr-pprof
pprof support through ``ctr`` added
2017-02-10 12:20:24 -08:00
Michael Crosby
5c8b989305 Merge pull request #512 from stevvooe/return-of-the-mount
api/types/mount: bring back the mount type
2017-02-10 11:14:50 -08:00
Michael Crosby
f109ce26e0 Merge pull request #513 from fate-grand-order/branch-2
fix some typos
2017-02-10 11:12:23 -08:00
Daniel Zhang
ca52365780 Allow specifying the log level
Signed-off-by: Daniel Zhang <jmzwcn@gmail.com>
2017-02-10 17:29:06 +08:00
fate-grand-order
af86cd4d2f Use error.New () directly to output the error message
Signed-off-by: fate-grand-order <chenjg@harmonycloud.cn>
2017-02-10 14:31:49 +08:00
Phil Estes
fc85c1de78 Merge pull request #514 from sakeven/master
Fix copyFile error message
2017-02-09 21:54:06 -08:00
sakeven
0e2822f892 fix copyFile error message
Signed-off-by: sakeven <jc5930@sina.cn>
2017-02-10 11:56:58 +08:00
fate-grand-order
8ce64b2000 fix some typos
Signed-off-by: fate-grand-order <chenjg@harmonycloud.cn>
2017-02-10 10:52:25 +08:00
Stephen J Day
a9edb0d230
api/types/mount: bring back the mount type
The mount type is used across common GRPC services to express a deferred
access of a filesystem. Right now, they are generated by snapshotters,
but eventually, they can be passed for containers for creation at
runtime. With this flow, we can separate the generation and use of a
root container filesystem.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-09 18:23:08 -08:00
Kunal Kushwaha
a9ea9d80a3 ctr pprof support added
Signed-off-by: Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp>
2017-02-10 11:16:40 +09:00
Michael Crosby
696e88c813 Merge pull request #496 from stevvooe/unify-snapshot-keyspace
snapshot: clarify active and committed snapshots
2017-02-09 16:52:59 -08:00
Phil Estes
c9ba5dc9a0 Merge pull request #506 from kevinetc123/patch-fmt
use bitclear instead of seperating it and fix some typo
2017-02-09 13:39:11 -08:00
kevinetc123
6425680d94 use bitclear instead of seperating them and fix some typo
Signed-off-by: kevinetc123 <kaiwentan@harmonycloud.cn>
2017-02-09 17:30:21 +08:00
Stephen J Day
9c4b235954
snapshot/btrfs: update btrfs to snappshotter
Updates the btrfs snapshotter to meet the interface and current tests.
Mostly, we merge the keyspace into a common index. Like with the overlay
driver, we will still need to do more verification work to ensure
idempotence of key collisions.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-07 17:21:19 -08:00
Stephen J Day
e6c1bb0ff2
snapshot/overlay: port overlay driver to Snapshotter
With the change to the snapshotter interface, we've now updated the
overlay driver to follow the conventions of the current test suite. To
support key unification, an hashed index was added to active and
committed directories. We still need to do some testing around
collisions, but we'll leave that for a future PR.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-07 17:20:21 -08:00
Stephen J Day
ab08944aa7
snapshot: clarify active and committed snapshots
After receiving feedback on the `snapshot.Driver` interface, it was
found that the behavior of active and committed snapshots was confusing.
We attempt to clean this up by doing the following:

1. Define the concept of "active" and "committed" snapshots and their
lifecycle relationship. Active snapshots can be created from a parent.
Committed snapshots can only be created from active snapshots.
2. Only committed snapshots can be a parent.
3. Unify the keyspace of snapshots. For common operations, such as
removal and stat, we only have a single method that works for both
active and committed snapshots. For methods that take one or the other,
the restriction is called out. `Remove` and `Delete` are consolidated
for this purpose.
4. Define the `Info` data type to include name, parent, kind and
readonly state. This allows us to collect `Exists` and `Parent` into a
single method `Stat` and simplifies the `Walk` method, eliding `Active`.
5. The `Driver` has been renamed to `Snapshotter` due to the overuse of
the term `Driver`.

Effectively, we now have snapshots that are either active or committed.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-07 17:18:14 -08:00
Michael Crosby
42a17f9391 Merge pull request #501 from mlaventure/new-shim-continued
New shim continued
2017-02-07 15:52:08 -08:00
Phil Estes
eb2191c744 Merge pull request #493 from hqhq/specify_shim_socket
Remove duplicated close
2017-02-07 09:47:14 -08:00
Phil Estes
51f1ebfca0 Merge pull request #495 from stevvooe/btrfs-hash-once
btrfs: calculate hash key only once
2017-02-07 09:45:34 -08:00
Phil Estes
11a0c8f8cd Merge pull request #498 from kunalkushwaha/bug-profiler
closing http connection
2017-02-07 09:44:30 -08:00
Phil Estes
41fe09748c Merge pull request #499 from fate-grand-order/branch-1
fix misspell "scrutiny" in content/content.go
2017-02-07 09:43:19 -08:00
Kenfe-Mickael Laventure
b4a299e61d supervisor/shim: add exec support
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-02-07 09:25:55 -08:00
fate-grand-order
d44c8525d7 fix misspell "scrutiny" in content/content.go
Signed-off-by: fate-grand-order <chenjg@harmonycloud.cn>
2017-02-07 15:06:40 +08:00
Kunal Kushwaha
1191b209ae closing http connection
Signed-off-by: Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp>
2017-02-07 14:57:44 +09:00
Qiang Huang
81246555c9 Remove duplicated close
Signed-off-by: Qiang Huang <h.huangqiang@huawei.com>
2017-02-07 10:17:47 +08:00
Stephen J Day
74949be676
btrfs: calculate hash key only once
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-06 17:09:11 -08:00
Kenfe-Mickael Laventure
78d7e8b256 supervisor: implement monitoring
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-02-06 15:16:55 -08:00
Stephen Day
f861817a8e Merge pull request #492 from AkihiroSuda/fix-dist-ls
content: fix panic during filepath.Walk()
2017-02-06 14:27:30 -08:00
Kenfe-Mickael Laventure
31f26fed18 Move to a single Event type
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-02-06 08:46:37 -08:00
Kenfe-Mickael Laventure
8fbdf1c0d7 Remove execution/executors
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-02-06 08:38:11 -08:00
Akihiro Suda
e9510eb69c content: fix panic during filepath.Walk()
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-02-06 05:16:21 +00:00
Derek McGowan
deaa17b3c4 Merge pull request #489 from stevvooe/btrfs-driver-suite
btrfs: test btrfs snapshots with driver suite
2017-02-03 17:09:18 -08:00
Stephen J Day
aeffd4f92c
btrfs: test btrfs snapshots with driver suite
We now include btrfs in the snapshot driver test suite. This includes
the addition of parent links and name hashing into the btrfs driver.
We'll probably endup replacing this with a common metadata store, as
these relationships are generally identical between implementations.

A small bug was discovered in the delete implementation in the course
testing, so the btrfs package has been updated with a fix.

The overlay driver was modified accordingly with the btrfs driver to use
`Driver` as the exported type of each driver packge.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-03 15:54:11 -08:00
Stephen Day
127882fca7 Merge pull request #483 from dmcgowan/snapshot-diff
Add fs package
2017-02-03 12:14:33 -08:00
Derek McGowan
aaf18b5962 Rename CopyDirectory to CopyDir
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-02-03 12:08:53 -08:00
Derek McGowan
d96e6e3952 Refactor changes and test functions
Remove change type in favor of explicit change function.
Using change function makes it more difficult to unnecessarily
add to the change interface.

Update test apply functions to use an interface rather
than a function type.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-02-03 11:28:06 -08:00
Derek McGowan
65e8c07847 Update diff interface to use callback
The change computation will be done on the callers thread
and use callbacks rather than running a goroutine and
returning a channel.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-02-03 11:28:06 -08:00
Derek McGowan
aa3be3b0fe Vendor errgroup
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-02-03 11:28:06 -08:00
Derek McGowan
572dbcdbd4 Use copy file range from sysx
Use pooled buffers for copy

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-02-03 11:28:06 -08:00
Derek McGowan
bb9f6b568d Update continuity to use sysx branch
Pulls in changes for copy file range

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-02-03 11:28:06 -08:00
Derek McGowan
245495d54e Use full mode for making devices node
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-02-03 11:28:06 -08:00
Derek McGowan
bf8f37ba78 Remove incorrect and unused timespec check
Compare is using its own time check comparison and
doing byte comparison when ambiguous rather than
ignoring it like this function does.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-02-03 11:28:06 -08:00
Derek McGowan
f78105d832 Add support for devices
Address code comments from previous commit

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-02-03 11:28:06 -08:00
Derek McGowan
200cd6e877 Update snapshots to use fs package
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-02-03 11:28:06 -08:00
Derek McGowan
574862fd89 Add fs package
Add diff comparison with support for double walking
two trees for comparison or single walking a diff
tree. Single walking requires further implementation
for specific mount types.

Add directory copy function which is intended to provide
fastest possible local copy of file system directories
without hardlinking.

Add test package to make creating filesystems for
test easy and comparisons deep and informative.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-02-03 11:27:40 -08:00
Derek McGowan
5f08e609c0 Vendor continuity
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-02-03 11:27:40 -08:00