Merge pull request #566 from samuelkarp/snapshot-design

design: Update snapshots.md with current design

design/snapshots.md

@@ -21,7 +21,7 @@ minimal API simplifies behavior without sacrificing power. This makes the
 surface area for driver implementations smaller, ensuring that behavior is more
 consistent between implementations.
 
-These differ from the concept of the graphdriver in that the _Snapshot Manager_
+These differ from the concept of the graphdriver in that the _Snapshotter_
 has no knowledge of images or containers. Users simply prepare and commit
 directories. We also avoid the integration between graph drivers and the tar
 format used to represent the changesets.
@@ -35,7 +35,7 @@ In the past, the `graphdriver` component has provided quite a lot of
 functionality in Docker. This includes serialization, hashing, unpacking,
 packing, mounting.
 
-The _Snapshot Manager_ will only provide mount-oriented snapshot
+The _Snapshotter_ will only provide mount-oriented snapshot
 access with minimal metadata. Serialization, hashing, unpacking, packing and
 mounting are not included in this design, opting for common implementations
 between graphdrivers, rather than specialized ones. This is less of a problem
@@ -44,102 +44,106 @@ interface.
 
 ## Architecture
 
-The _Snapshot Manager_ provides an API for allocating, snapshotting and mounting
+The _Snapshotter_ provides an API for allocating, snapshotting and mounting
 abstract, layer-based filesystems. The model works by building up sets of
 directories with parent-child relationships, known as _Snapshots_.
 
-Every snapshot is represented by an opaque `diff` directory, which acts as a
+A _Snapshot_ represents a filesystem state.  Every snapshot has a parent,
-handle to the snapshot. It may contain driver specific data, including changeset
+where the empty parent is represented by the empty string.  A diff can be taken
-data, parent information and arbitrary metadata.
+between a parent and its snapshot to create a classic layer.
 
-The `diff` directory for a _snapshot_ is created with a transactional
+Snapshots are best understood by their lifecycle.  _Active_ snapshots are always
-operation. Each _snapshot_ may have one parent snapshot. When one starts a
+created with `Prepare` or `View` from a _Committed_ snapshot (including the
-transaction on an existing snapshot, the result may only be used as a parent
+empty snapshot).  _Committed_ snapshots are always created with
-_after_ being committed.  The empty string `diff` directory is a handle to the
+`Commit` from an _Active_ snapshot.  Active snapshots never become committed
-empty snapshot, which is the ancestor of all snapshots.
+snapshots and vice versa. All snapshots may be removed.
 
-The `target` directory represents the active snapshot location. The driver may
+After mounting an _Active_ snapshot, changes can be made to the snapshot.  The
-maintain internal metadata associated with the `target` but the contents is
+act of committing creates a _Committed_ snapshot.  The committed snapshot will
-generally manipulated by the client.
+inherit the parent of the active snapshot.  The committed snapshot can then be
+used as a parent.  Active snapshots can never be used as a parent.
+
+The following diagram demonstrates the relationships of snapshots:
+
+![snapshot model diagram, showing active snapshots on the left and
+committed snapshots on the right](snapshot_model.png)
+
+In this diagram, you can see that the active snapshot _a_ is created by calling
+`Prepare` with the committed snapshot _P<sub>0</sub>_.  After modification, _a_
+becomes _a'_ and a committed snapshot _P<sub>1</sub>_ is created by calling
+`Commit`.  _a'_ can be further modified as _a''_ and a second committed snapshot
+can be created as _P<sub>2</sub>_ by calling `Commit` again.  Note here that
+_P<sub>2</sub>_'s parent is _P<sub>0</sub>_ and not _P<sub>1</sub>_.
 
 ### Operations
 
-The manifestation of _snapshots_ is facilitated by the _mount_ object and
+The manifestation of _snapshots_ is facilitated by the `Mount` object and
 user-defined directories used for opaque data storage. When creating a new
-snapshot, the caller provides a directory where they would like the _snapshot_
+active snapshot, the caller provides an identifier called the _key_. This
-to be mounted, called the _target_. This operation returns a list of mounts
+operation returns a list of mounts that, if mounted, will have the fully
-that, if mounted, will have the fully prepared snapshot at the requested path.
+prepared snapshot at the mounted path. We call this the _prepare_ operation.
-We call this the _prepare_ operation.
 
-Once a path is _prepared_ and mounted, the caller may write new data to the
+Once a snapshot is _prepared_ and mounted, the caller may write new data to the
-snapshot. Depending on the application, a user may want to capture these changes or
+snapshot. Depending on the application, a user may want to capture these changes
-not.
+or not.
+
+For a read-only view of a snapshot, the _view_ operation can be used. Like
+_prepare_, _view_ will return a list of mounts that, if mounted, will have the
+fully prepared snapshot at the mounted path.
 
 If the user wants to keep the changes, the _commit_ operation is employed. The
-_commit_ operation takes the `target` directory, which represents an open
+_commit_ operation takes the _key_ identifier, which represents an active
-transaction, and a `diff` directory. A successful result will provide the
+snapshot, and a _name_ identifier. A successful result will create a _committed_
-difference between the parent and the snapshot in the `diff` directory, which
+snapshot that can be used as the parent of new _active_ snapshots when
-should be treated as opaque by the caller. This new `diff` directory can then
+referenced by the _name_.
-be used as the `parent` in calls to future _prepare_ operations.
 
-If the user wants to discard the changes, the _rollback_ operation will release
+If the user wants to discard the changes in an active snapshot, the _remove_
-any resources associated with the snapshot. While rollback may be a rare operation
+operation will release any resources associated with the snapshot.  The mounts
-in other transactional systems, this is a common operation for containers.
+provided by _prepare_ or _view_ should be unmounted before calling this method.
-After removal, most containers will utilize the _rollback_ operation.
 
-For both _rollback_ and _commit_ the mounts provided by _prepare_ should be
+If the user wants to discard committed snapshots, the _remove_ operation can
-unmounted before calling these methods.
+also be used, but any children must be removed before proceeding.
+
+For detailed usage information, see the
+[GoDoc](https://godoc.org/github.com/docker/containerd/snapshot#Snapshotter).
 
 ### Graph metadata
 
 As snapshots are imported into the container system, a "graph" of snapshots and
 their parents will form. Queries over this graph must be a supported operation.
-Subsequently, each snapshot ends up representing 
-
-### Path Management
-
-No path layout for snapshot locations is imposed on the caller. The paths used
-by the snapshot drivers are largely under the control of the caller. This provides
-the most flexibility in using the snapshot system but requires discipline when
-deciding which paths to use and which ones to avoid.
-
-We may provide a helper component to manage `diff` path layout when working
-with OCI and docker images.
 
 ## How snapshots work
 
 To flesh out the _Snapshots_ terminology, we are going to demonstrate the use of
-the _Snapshot Manager_ from the perspective of importing layers. We'll use a Go API
+the _Snapshotter_ from the perspective of importing layers. We'll use a Go API
 to represent the process.
 
 ### Importing a Layer
 
-To import a layer, we simply have the _Snapshot Manager_ provide a list of
+To import a layer, we simply have the _Snapshotter_ provide a list of
 mounts to be applied such that our destination will capture a changeset. We start
 out by getting a path to the layer tar file and creating a temp location to
 unpack it to:
 
-	layerPath, tmpLocation := getLayerPath(), mkTmpDir() // just a path to the layer tar file.
+	layerPath, tmpDir := getLayerPath(), mkTmpDir() // just a path to layer tar file.
 
-Per the terminology above, `tmpLocation` is known as the `target`. `layerPath`
+We start by using a _Snapshotter_ to _Prepare_ a new snapshot transaction, using
-is simply a tar file, representing a changset. We start by using
+a _key_ and descending from the empty parent "":
-`SnapshotManager` to prepare the temporary location as a snapshot point:
 
-	sm := SnapshotManager()
+	mounts, err := snapshotter.Prepare(key, "")
-	mounts, err := sm.Prepare(tmpLocation, "")
 	if err != nil { ... }
 
-Note that we provide "" as the `parent`, since we are applying the diff to an
+We get back a list of mounts from `Snapshotter.Prepare`, with the `key`
-empty directory. We get back a list of mounts from `SnapshotManager.Prepare`.
+identifying the active snapshot. Mount this to the temporary location with the
-Before proceeding, we perform all these mounts:
+following:
 
-	if err := MountAll(mounts); err != nil { ... }
+	if err := MountAll(mounts, tmpDir); err != nil { ... }
 
 Once the mounts are performed, our temporary location is ready to capture
-a diff. In practice, this works similarly to a filesystem transaction. The
+a diff. In practice, this works similar to a filesystem transaction. The
-next step is to unpack the layer. We have a special function, `unpackLayer`
+next step is to unpack the layer. We have a special function `unpackLayer`
 that applies the contents of the layer to target location and calculates the
-DiffID of the unpacked layer (this is a requirement for the docker
+`DiffID` of the unpacked layer (this is a requirement for docker
 implementation):
 
 	layer, err := os.Open(layerPath)
@@ -149,49 +153,46 @@ implementation):
 
 When the above completes, we should have a filesystem the represents the
 contents of the layer. Careful implementations should verify that digest
-matches the expected DiffID. When completed, we unmount the mounts:
+matches the expected `DiffID`. When completed, we unmount the mounts:
 
 	unmount(mounts) // optional, for now
 
-Now that we've verified and unpacked our layer, we create a location to commit
+Now that we've verified and unpacked our layer, we commit the active
-the actual diff. For this example, we are just going to use the layer `digest`,
+snapshot to a _name_. For this example, we are just going to use the layer
-but in practice, this will probably be the `ChainID`:
+digest, but in practice, this will probably be the `ChainID`:
 
-	diffPath := filepath.Join("/layers", digest) // name location for the uncompressed layer digest
+	if err := snapshotter.Commit(digest.String(), key); err != nil { ... }
-	if err := sm.Commit(diffPath, tmpLocation); err != nil { ... }
 
-The new layer has been imported as a _snapshot_ into the `SnapshotManager`
+Now, we have a layer in the _Snapshotter_ that can be accessed with the digest
-under the name `diffPath`. `diffPath`, which is a user opaque directory
+provided during commit. Once you have committed the snapshot, the active
-location, can then be used as a parent in later snapshots.
+snapshot can be removed with the following:
+
+	snapshotter.Remove(key)
 
 ### Importing the Next Layer
 
 Making a layer depend on the above is identical to the process described
-above except that the parent is provided as diffPath when calling
+above except that the parent is provided as `parent` when calling
-`SnapshotManager.Prepare`:
+`Snapshotter.Prepare`, assuming a clean `tmpLocation`:
 
-	mounts, err := sm.Prepare(tmpLocation, parentDiffPath)
+	mounts, err := snapshotter.Prepare(tmpLocation, parentDigest)
 
-Because have a provided a `parent`, the resulting `tmpLocation`, after
+We then mount, apply and commit, as we did above. The new snapshot will be
-mounting, will have the changes from above. Any new changes will be isolated to
+based on the content of the previous one.
-the snapshot `target`.
-
-We run the same unpacking process and commit as before to get the new `diff`.
 
 ### Running a Container
 
-To run a container, we simply provide `SnapshotManager.Prepare` the `diff` of
+To run a container, we simply provide `Snapshotter.Prepare` the committed image
-the image we want to start the container from. After mounting, the prepared
+snapshot as the parent. After mounting, the prepared path can
-path can be used directly as the container's filesystem:
+be used directly as the container's filesystem:
 
-	mounts, err := sm.Prepare(containerRootFS, imageDiffPath)
+	mounts, err := snapshotter.Prepare(containerKey, imageRootFSChainID)
 
 The returned mounts can then be passed directly to the container runtime. If
-one would like to create a new image from the filesystem,
+one would like to create a new image from the filesystem, `Snapshotter.Commit`
-`SnapshotManager.Commit` is called:
+is called:
 
-	if err := sm.Commit(newImageDiff, containerRootFS); err != nil { ... }
+	if err := snapshotter.Commit(newImageSnapshot, containerKey); err != nil { ... }
 
-Alternatively, in the majority of cases, `SnapshotManager.Rollback` will be
+Alternatively, for most container runs, `Snapshotter.Remove` will be called to
-called to signal `SnapshotManager` to abandon the changes after a container
+signal the Snapshotter to abandon the changes.
-runtime process has completed.

snapshot/snapshotter.go

@@ -67,7 +67,7 @@ type Info struct {
 //
 // Importing a Layer
 //
-// To import a layer, we simply have the Manager provide a list of
+// To import a layer, we simply have the Snapshotter provide a list of
 // mounts to be applied such that our dst will capture a changeset. We start
 // out by getting a path to the layer tar file and creating a temp location to
 // unpack it to:
@@ -122,7 +122,7 @@ type Info struct {
 // above except that the parent is provided as parent when calling
 // Manager.Prepare, assuming a clean tmpLocation:
 //
-// 	mounts, err := sm.Prepare(tmpLocation, parentDigest)
+// 	mounts, err := snapshotter.Prepare(tmpLocation, parentDigest)
 //
 // We then mount, apply and commit, as we did above. The new snapshot will be
 // based on the content of the previous one.
@@ -133,15 +133,15 @@ type Info struct {
 // snapshot as the parent. After mounting, the prepared path can
 // be used directly as the container's filesystem:
 //
-// 	mounts, err := sm.Prepare(containerKey, imageRootFSChainID)
+// 	mounts, err := snapshotter.Prepare(containerKey, imageRootFSChainID)
 //
 // The returned mounts can then be passed directly to the container runtime. If
 // one would like to create a new image from the filesystem, Manager.Commit is
 // called:
 //
-// 	if err := sm.Commit(newImageSnapshot, containerKey); err != nil { ... }
+// 	if err := snapshotter.Commit(newImageSnapshot, containerKey); err != nil { ... }
 //
-// Alternatively, for most container runs, Manager.Remove will be called to
+// Alternatively, for most container runs, Snapshotter.Remove will be called to
 // signal the Snapshotter to abandon the changes.
 type Snapshotter interface {
 	// Stat returns the info for an active or committed snapshot by name or