Storage Driver: Ceph Object Storage (RADOS)

This driver implements the storagedriver.StorageDriver interface and
uses Ceph Object Storage as storage backend.

Since RADOS is an object storage and no hierarchy notion, the
following convention is used to keep the filesystem notions stored in
this backend:

* All the objects data are stored with opaque UUID names prefixed
  (e.g. "blob:d3d232ff-ab3a-4046-9ab7-930228d4c164).
* All the hierarchy information are stored in rados omaps, where the
  omap object identifier is the virtual directory name, the keys in
  a specific are the relative filenames and the values the blob
  object identifier (or empty value for a sub directory).

  e.g. For the following hierarchy:

     /directory1
     /directory1/object1
     /directory1/object2
     /directory1/directory2/object3

  The omap "/directory1" will contains the following key / values:
    - "object1" "blob:d3d232ff-ab3a-4046-9ab7-930228d4c164"
    - "object2" "blob:db2e359d-4af0-4bfb-ba1d-d2fd029866a0"
    - "directory2" ""

  The omap "/directory1/directory2" will contains:
    - "object3" "blob:9ae2371c-81fc-4945-80ac-8bf7f566a5d9"

* The MOVE is implemented by changing the reference to a specific
  blob in its parent virtual directory omap.

This driver stripes rados objects to a fixed size (e.g. 4M). The idea
is to keep small objects (as done by RBD on the top of RADOS) that
will be easily synchronized accross OSDs. The information of the
original object (i.e total size of the chunks) is stored as a Xattr
in the first chunk object.

Signed-off-by: Vincent Giersch <vincent.giersch@ovh.net>

docs/building.md

@@ -121,3 +121,15 @@ $ make
 
 If that is successful, standard `go` commands, such as `go test` should work,
 per package, without issue.
+
+### Optional build tags
+
+Optional [build tags](http://golang.org/pkg/go/build/) can be provided using
+the environment variable `DOCKER_BUILDTAGS`.
+
+To enable the [Ceph RADOS storage driver](storage-drivers/rados.md)
+(librados-dev and librbd-dev will be required to build the bindings):
+
+```sh
+export DOCKER_BUILDTAGS='include_rados'
+```

docs/configuration.md

@@ -43,6 +43,10 @@ storage:
 		v4auth: true
 		chunksize: 5242880
 		rootdirectory: /s3/object/name/prefix
+	rados:
+		poolname: radospool
+		username: radosuser
+		chunksize: 4194304
 	cache:
 		blobdescriptor: redis
 	maintenance:
@@ -261,6 +265,10 @@ storage:
 		v4auth: true
 		chunksize: 5242880
 		rootdirectory: /s3/object/name/prefix
+	rados:
+		poolname: radospool
+		username: radosuser
+		chunksize: 4194304
 	cache:
 		blobdescriptor: inmemory
 	maintenance:
@@ -348,6 +356,51 @@ This storage backend uses Microsoft's Azure Storage platform.
 </table>
 
 
+### rados
+
+This storage backend uses [Ceph Object Storage](http://ceph.com/docs/master/rados/).
+
+<table>
+  <tr>
+    <th>Parameter</th>
+    <th>Required</th>
+    <th>Description</th>
+  </tr>
+  <tr>
+    <td>
+      <code>poolname</code>
+    </td>
+    <td>
+      yes
+    </td>
+    <td>
+      Ceph pool name.
+    </td>
+  </tr>
+   <tr>
+    <td>
+      <code>username</code>
+    </td>
+    <td>
+      no
+    </td>
+    <td>
+      Ceph cluster user to connect as (i.e. admin, not client.admin).
+    </td>
+  </tr>
+   <tr>
+    <td>
+      <code>chunksize</code>
+    </td>
+    <td>
+      no
+    </td>
+    <td>
+      Size of the written RADOS objects. Default value is 4MB (4194304).
+    </td>
+  </tr>
+</table>
+
 
 ### S3
 

docs/mkdocs.yml

@@ -10,4 +10,5 @@
 - ['registry/storage-drivers/azure.md', '**HIDDEN**' ]
 - ['registry/storage-drivers/filesystem.md', '**HIDDEN**' ]
 - ['registry/storage-drivers/inmemory.md', '**HIDDEN**' ]
+- ['registry/storage-drivers/rados.md', '**HIDDEN**' ]
 - ['registry/storage-drivers/s3.md','**HIDDEN**' ]

docs/storage-drivers/rados.md

@@ -0,0 +1,37 @@
+<!--GITHUB
+page_title: Ceph RADOS storage driver
+page_description: Explains how to use the Ceph RADOS storage driver
+page_keywords: registry, service, driver, images, storage, ceph, rados
+IGNORES-->
+
+# Ceph RADOS storage driver
+
+An implementation of the `storagedriver.StorageDriver` interface which uses
+[Ceph RADOS Object Storage][rados] for storage backend.
+
+## Parameters
+
+The following parameters must be used to configure the storage driver
+(case-sensitive):
+
+* `poolname`: Name of the Ceph pool
+* `username` *optional*: The user to connect as (i.e. admin, not client.admin)
+* `chunksize` *optional*: Size of the written RADOS objects. Default value is
+4MB (4194304).
+
+This drivers loads the [Ceph client configuration][rados-config] from the
+following regular paths (the first found is used):
+
+* `$CEPH_CONF` (environment variable)
+* `/etc/ceph/ceph.conf`
+* `~/.ceph/config`
+* `ceph.conf` (in the current working directory)
+
+## Developing
+
+To include this driver when building Docker Distribution, use the build tag
+`include_rados`. Please see the [building documentation][building] for details.
+
+[rados]: http://ceph.com/docs/master/rados/
+[rados-config]: http://ceph.com/docs/master/rados/configuration/ceph-conf/
+[building]: https://github.com/docker/distribution/blob/master/docs/building.md#optional-build-tags

docs/storagedrivers.md

@@ -16,6 +16,7 @@ This storage driver package comes bundled with several drivers:
 - [filesystem](storage-drivers/filesystem.md): A local storage driver configured to use a directory tree in the local filesystem.
 - [s3](storage-drivers/s3.md): A driver storing objects in an Amazon Simple Storage Solution (S3) bucket.
 - [azure](storage-drivers/azure.md): A driver storing objects in [Microsoft Azure Blob Storage](http://azure.microsoft.com/en-us/services/storage/).
+- [rados](storage-drivers/rados.md): A driver storing objects in a [Ceph Object Storage](http://ceph.com/docs/master/rados/) pool.
 
 ## Storage Driver API