Commit Graph

28 Commits

Author SHA1 Message Date
Stephen J Day 9c4b235954
snapshot/btrfs: update btrfs to snappshotter
Updates the btrfs snapshotter to meet the interface and current tests.
Mostly, we merge the keyspace into a common index. Like with the overlay
driver, we will still need to do more verification work to ensure
idempotence of key collisions.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-07 17:21:19 -08:00
Stephen J Day e6c1bb0ff2
snapshot/overlay: port overlay driver to Snapshotter
With the change to the snapshotter interface, we've now updated the
overlay driver to follow the conventions of the current test suite. To
support key unification, an hashed index was added to active and
committed directories. We still need to do some testing around
collisions, but we'll leave that for a future PR.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-07 17:20:21 -08:00
Stephen J Day ab08944aa7
snapshot: clarify active and committed snapshots
After receiving feedback on the `snapshot.Driver` interface, it was
found that the behavior of active and committed snapshots was confusing.
We attempt to clean this up by doing the following:

1. Define the concept of "active" and "committed" snapshots and their
lifecycle relationship. Active snapshots can be created from a parent.
Committed snapshots can only be created from active snapshots.
2. Only committed snapshots can be a parent.
3. Unify the keyspace of snapshots. For common operations, such as
removal and stat, we only have a single method that works for both
active and committed snapshots. For methods that take one or the other,
the restriction is called out. `Remove` and `Delete` are consolidated
for this purpose.
4. Define the `Info` data type to include name, parent, kind and
readonly state. This allows us to collect `Exists` and `Parent` into a
single method `Stat` and simplifies the `Walk` method, eliding `Active`.
5. The `Driver` has been renamed to `Snapshotter` due to the overuse of
the term `Driver`.

Effectively, we now have snapshots that are either active or committed.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-07 17:18:14 -08:00
Stephen J Day 74949be676
btrfs: calculate hash key only once
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-06 17:09:11 -08:00
Stephen J Day aeffd4f92c
btrfs: test btrfs snapshots with driver suite
We now include btrfs in the snapshot driver test suite. This includes
the addition of parent links and name hashing into the btrfs driver.
We'll probably endup replacing this with a common metadata store, as
these relationships are generally identical between implementations.

A small bug was discovered in the delete implementation in the course
testing, so the btrfs package has been updated with a fix.

The overlay driver was modified accordingly with the btrfs driver to use
`Driver` as the exported type of each driver packge.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-03 15:54:11 -08:00
Derek McGowan aaf18b5962 Rename CopyDirectory to CopyDir
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-02-03 12:08:53 -08:00
Derek McGowan 200cd6e877 Update snapshots to use fs package
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-02-03 11:28:06 -08:00
Stephen J Day d0b4ce8d17
snapshot/overlay: add snapshot test suite to overlay
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-02 18:55:53 -08:00
Stephen J Day 68fd25221d
snapshot: define the snapshot driver interface
We now define the `snapshot.Driver` interface based on earlier work.
Many details of the model are worked out, such as snapshot lifecycle and
parentage of commits against "Active" snapshots.

The impetus of this change is to provide a snapshot POC that does a
complete push/pull workflow. The beginnings of a test suite for snapshot
drivers is included that we can use to verify the assumptions of
drivers. The intent is to port the existing tests over to this test
suite and start scaling contributions and test to the snapshot driver
subsystem.

There are still some details that need to be worked out, such as listing
and metadata access. We can do this activity as we further integrate
with tooling.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-02 18:06:02 -08:00
Samuel Karp 3c37c9e6b1 snapshot: Remove mistakenly-added binary file
Signed-off-by: Samuel Karp <skarp@amazon.com>
2017-01-30 13:57:58 -08:00
Samuel Karp cacda40317 snapshot: Separate tests using root from non-root
Signed-off-by: Samuel Karp <skarp@amazon.com>
2017-01-25 17:13:29 -08:00
Akihiro Suda 1f763301a6 snapshot: fix terminology inconsistency
LayerManipulator, SnapshotManipulator -> SnapshotManager

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-01-20 02:02:29 +00:00
Aaron.L.Xu 08bcbddb32 fix typo I found in this repo
Signed-off-by: Aaron.L.Xu <likexu@harmonycloud.cn>
2017-01-20 01:18:26 +08:00
Michael Crosby a7a6270f2a Remove shelling out to mount
Also remove the target from the Mount struct because it should not be
used at all.  The target can be variable and set by a caller, not by the
snapshot drivers.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-01-18 11:10:31 -08:00
Stephen J Day e3f83fd53d snapshots: separate implementations into packages
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-01-13 15:32:41 -08:00
Samuel Karp 2c36c387de snapshot: Fix leaked mounts and temp directories
Signed-off-by: Samuel Karp <skarp@amazon.com>
2017-01-12 01:12:37 +00:00
Samuel Karp dbe117cb7e snapshot: automate btrfs test setup
Signed-off-by: Samuel Karp <skarp@amazon.com>
2017-01-10 00:50:33 +00:00
Gábor Lipták 102fb10fde Correct vet warnings
Signed-off-by: Gábor Lipták <gliptak@gmail.com>
2016-12-20 17:10:01 -05:00
Qiang Huang 3b8eee7522 Fix some typos
Signed-off-by: Qiang Huang <h.huangqiang@huawei.com>
2016-12-16 09:31:19 +08:00
Stephen J Day 030d5dde1e btrfs: add initial implementation of btrfs snapshot driver
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2016-12-12 21:53:48 -08:00
Stephen J Day bdc065a1df snapshot: follow dst-first order convention in commit
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2016-12-12 21:10:09 -08:00
Michael Crosby 934940a96c Add parent cache to overlayfs storage driver
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-12-12 13:35:02 -08:00
Michael Crosby 0808a5c485 Implement overlay fs snapshot driver
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-12-09 13:36:22 -08:00
Stephen J Day aee6045292
snapshot: provide naive/vfs POC implementation
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2016-12-07 19:59:34 -08:00
Stephen J Day 65e668a6b8
snapshot: ensure tests compile
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2016-12-07 19:59:03 -08:00
Stephen J Day f7850ccd68
snapshot: specify View function on snapshot manager
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2016-12-07 19:58:26 -08:00
Michael Crosby c2a57e2b00 Fix build because of imports in package
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-12-01 10:47:47 -08:00
Stephen J Day 3110cf37dc
snapshot: rename layer manipulator to snapshot
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2016-11-28 18:58:25 -08:00