Add description of digests to API specification
Signed-off-by: Stephen J Day <stephen.day@docker.com>
This commit is contained in:
Stephen J Day
parent
c54adb667f
commit
7e6b4e8c52
2 changed files with 168 additions and 0 deletions
|
|
@ -116,6 +116,12 @@ indicating what is different. Optionally, we may start marking parts of the
|
||||||
specification to correspond with the versions enumerated here.
|
specification to correspond with the versions enumerated here.
|
||||||
|
|
||||||
<dl>
|
<dl>
|
||||||
|
<dt>2.0.2</dt>
|
||||||
|
<dd>
|
||||||
|
<li>Added section covering digest format.</li>
|
||||||
|
<li>Added more clarification that manifest cannot be deleted by tag.</li>
|
||||||
|
</dd>
|
||||||
|
|
||||||
<dt>2.0.1</dt>
|
<dt>2.0.1</dt>
|
||||||
<dd>
|
<dd>
|
||||||
<ul>
|
<ul>
|
||||||
|
|
@ -238,6 +244,84 @@ When a `200 OK` or `401 Unauthorized` response is returned, the
|
||||||
Clients may require this header value to determine if the endpoint serves this
|
Clients may require this header value to determine if the endpoint serves this
|
||||||
API. When this header is omitted, clients may fallback to an older API version.
|
API. When this header is omitted, clients may fallback to an older API version.
|
||||||
|
|
||||||
|
### Content Digests
|
||||||
|
|
||||||
|
This API design is driven heavily by [content addressability](http://en.wikipedia.org/wiki/Content-addressable_storage).
|
||||||
|
The core of this design is the concept of a content addressable identifier. It
|
||||||
|
uniquely identifies content by taking a collision-resistent hash of the bytes.
|
||||||
|
Such an identifier can be independently calculated and verified by selection
|
||||||
|
of a common _algorithm_. If such an identifier can be communicated in a secure
|
||||||
|
manner, one can retrieve the content from an insecure source, calculate it
|
||||||
|
independently and be certain that the correct content was obtained. Put simply,
|
||||||
|
the identifier is a property of the content.
|
||||||
|
|
||||||
|
To disambiguate from other concepts, we call this identifier a _digest_. A
|
||||||
|
_digest_ is a serialized hash result, consisting of a _algorithm_ and _hex_
|
||||||
|
portion. The _algorithm_ identifies the methodology used to calculate the
|
||||||
|
digest. The _hex_ portion is the hex-encoded result of the hash.
|
||||||
|
|
||||||
|
We define a _digest_ string to match the following grammar:
|
||||||
|
|
||||||
|
digest := algorithm ":" hex
|
||||||
|
algorithm := /[A-Fa-f0-9_+.-]+/
|
||||||
|
hex := /[A-Fa-f0-9]+/
|
||||||
|
|
||||||
|
Some examples of _digests_ include the following:
|
||||||
|
|
||||||
|
digest | description |
|
||||||
|
----------------------------------------------------------------------------------|------------------------------------------------
|
||||||
|
sha256:6c3c624b58dbbcd3c0dd82b4c53f04194d1247c6eebdaab7c610cf7d66709b3b | Common sha256 based digest |
|
||||||
|
tarsum.v1+sha256:6c3c624b58dbbcd3c0dd82b4c53f04194d1247c6eebdaab7c610cf7d66709b3b | Tarsum digest, used for legacy layer digests. |
|
||||||
|
|
||||||
|
> __NOTE:__ While we show an example of using a `tarsum` digest, the security
|
||||||
|
> of tarsum has not been verified. It is recommended that most implementations
|
||||||
|
> use sha256 for interoperability.
|
||||||
|
|
||||||
|
While the _algorithm_ does allow one to implement a wide variety of
|
||||||
|
algorithms, compliant implementations should use sha256. Heavy processing of
|
||||||
|
input before calculating a hash is discouraged to avoid degrading the
|
||||||
|
uniqueness of the _digest_ but some canonicalization may be performed to
|
||||||
|
ensure consistent identifiers.
|
||||||
|
|
||||||
|
Let's use a simple example in pseudo-code to demonstrate a digest calculation:
|
||||||
|
|
||||||
|
```
|
||||||
|
let C = 'a small string'
|
||||||
|
let B = sha256(C)
|
||||||
|
let D = 'sha256:' + EncodeHex(B)
|
||||||
|
let ID(C) = D
|
||||||
|
```
|
||||||
|
|
||||||
|
Above, we have bytestring _C_ passed into a function, _SHA256_, that returns a
|
||||||
|
bytestring B, which is the hash of _C_. _D_ gets the algorithm concatenated
|
||||||
|
with the hex encoding of _B_. We then define the identifier of _C_ to _ID(C)_
|
||||||
|
as equal to _D_. A digest can be verified by independently calculating _D_ and
|
||||||
|
comparing it with identifier _ID(C)_
|
||||||
|
|
||||||
|
#### Digest Header
|
||||||
|
|
||||||
|
To provide verification of http content, any response may include a `Docker-
|
||||||
|
Content-Digest` header. This will include the digest of the target entity
|
||||||
|
returned in the response. For blobs, this is the entire blob content. For
|
||||||
|
manifests, this is the manifest body without the signature content, also known
|
||||||
|
as the JWS payload. Note that the commonly used canonicalization for digest
|
||||||
|
calculation may be dependent on the mediatype of the content, such as with
|
||||||
|
manifests.
|
||||||
|
|
||||||
|
The client may choose to ignore the header or may verify it to ensure content
|
||||||
|
integrity and transport security. This is most important when fetching by a
|
||||||
|
digest. To ensure security, the content should be verified against the digest
|
||||||
|
used to fetch the content. At times, the returned digest may differ from that
|
||||||
|
used to initiate a request. Such digests are considered to be from different
|
||||||
|
_domains_, meaning they have different values for _algorithm_. In such a case,
|
||||||
|
the client may choose to verify the digests in both domains or ignore the
|
||||||
|
server's digest. To maintain security, the client _must_ always verify the
|
||||||
|
content against the _digest_ used to fetch the content.
|
||||||
|
|
||||||
|
> __IMPORTANT:__ If a _digest_ is used to fetch content, the client should use
|
||||||
|
> the same digest used to fetch the content to verify it. The header `Docker-
|
||||||
|
> Content-Digest` should not be trusted over the "local" digest.
|
||||||
|
|
||||||
### Pulling An Image
|
### Pulling An Image
|
||||||
|
|
||||||
An "image" is a combination of a JSON manifest and individual layer files. The
|
An "image" is a combination of a JSON manifest and individual layer files. The
|
||||||
|
|
|
||||||
|
|
@ -116,6 +116,12 @@ indicating what is different. Optionally, we may start marking parts of the
|
||||||
specification to correspond with the versions enumerated here.
|
specification to correspond with the versions enumerated here.
|
||||||
|
|
||||||
<dl>
|
<dl>
|
||||||
|
<dt>2.0.2</dt>
|
||||||
|
<dd>
|
||||||
|
<li>Added section covering digest format.</li>
|
||||||
|
<li>Added more clarification that manifest cannot be deleted by tag.</li>
|
||||||
|
</dd>
|
||||||
|
|
||||||
<dt>2.0.1</dt>
|
<dt>2.0.1</dt>
|
||||||
<dd>
|
<dd>
|
||||||
<ul>
|
<ul>
|
||||||
|
|
@ -238,6 +244,84 @@ When a `200 OK` or `401 Unauthorized` response is returned, the
|
||||||
Clients may require this header value to determine if the endpoint serves this
|
Clients may require this header value to determine if the endpoint serves this
|
||||||
API. When this header is omitted, clients may fallback to an older API version.
|
API. When this header is omitted, clients may fallback to an older API version.
|
||||||
|
|
||||||
|
### Content Digests
|
||||||
|
|
||||||
|
This API design is driven heavily by [content addressability](http://en.wikipedia.org/wiki/Content-addressable_storage).
|
||||||
|
The core of this design is the concept of a content addressable identifier. It
|
||||||
|
uniquely identifies content by taking a collision-resistent hash of the bytes.
|
||||||
|
Such an identifier can be independently calculated and verified by selection
|
||||||
|
of a common _algorithm_. If such an identifier can be communicated in a secure
|
||||||
|
manner, one can retrieve the content from an insecure source, calculate it
|
||||||
|
independently and be certain that the correct content was obtained. Put simply,
|
||||||
|
the identifier is a property of the content.
|
||||||
|
|
||||||
|
To disambiguate from other concepts, we call this identifier a _digest_. A
|
||||||
|
_digest_ is a serialized hash result, consisting of a _algorithm_ and _hex_
|
||||||
|
portion. The _algorithm_ identifies the methodology used to calculate the
|
||||||
|
digest. The _hex_ portion is the hex-encoded result of the hash.
|
||||||
|
|
||||||
|
We define a _digest_ string to match the following grammar:
|
||||||
|
|
||||||
|
digest := algorithm ":" hex
|
||||||
|
algorithm := /[A-Fa-f0-9_+.-]+/
|
||||||
|
hex := /[A-Fa-f0-9]+/
|
||||||
|
|
||||||
|
Some examples of _digests_ include the following:
|
||||||
|
|
||||||
|
digest | description |
|
||||||
|
----------------------------------------------------------------------------------|------------------------------------------------
|
||||||
|
sha256:6c3c624b58dbbcd3c0dd82b4c53f04194d1247c6eebdaab7c610cf7d66709b3b | Common sha256 based digest |
|
||||||
|
tarsum.v1+sha256:6c3c624b58dbbcd3c0dd82b4c53f04194d1247c6eebdaab7c610cf7d66709b3b | Tarsum digest, used for legacy layer digests. |
|
||||||
|
|
||||||
|
> __NOTE:__ While we show an example of using a `tarsum` digest, the security
|
||||||
|
> of tarsum has not been verified. It is recommended that most implementations
|
||||||
|
> use sha256 for interoperability.
|
||||||
|
|
||||||
|
While the _algorithm_ does allow one to implement a wide variety of
|
||||||
|
algorithms, compliant implementations should use sha256. Heavy processing of
|
||||||
|
input before calculating a hash is discouraged to avoid degrading the
|
||||||
|
uniqueness of the _digest_ but some canonicalization may be performed to
|
||||||
|
ensure consistent identifiers.
|
||||||
|
|
||||||
|
Let's use a simple example in pseudo-code to demonstrate a digest calculation:
|
||||||
|
|
||||||
|
```
|
||||||
|
let C = 'a small string'
|
||||||
|
let B = sha256(C)
|
||||||
|
let D = 'sha256:' + EncodeHex(B)
|
||||||
|
let ID(C) = D
|
||||||
|
```
|
||||||
|
|
||||||
|
Above, we have bytestring _C_ passed into a function, _SHA256_, that returns a
|
||||||
|
bytestring B, which is the hash of _C_. _D_ gets the algorithm concatenated
|
||||||
|
with the hex encoding of _B_. We then define the identifier of _C_ to _ID(C)_
|
||||||
|
as equal to _D_. A digest can be verified by independently calculating _D_ and
|
||||||
|
comparing it with identifier _ID(C)_
|
||||||
|
|
||||||
|
#### Digest Header
|
||||||
|
|
||||||
|
To provide verification of http content, any response may include a `Docker-
|
||||||
|
Content-Digest` header. This will include the digest of the target entity
|
||||||
|
returned in the response. For blobs, this is the entire blob content. For
|
||||||
|
manifests, this is the manifest body without the signature content, also known
|
||||||
|
as the JWS payload. Note that the commonly used canonicalization for digest
|
||||||
|
calculation may be dependent on the mediatype of the content, such as with
|
||||||
|
manifests.
|
||||||
|
|
||||||
|
The client may choose to ignore the header or may verify it to ensure content
|
||||||
|
integrity and transport security. This is most important when fetching by a
|
||||||
|
digest. To ensure security, the content should be verified against the digest
|
||||||
|
used to fetch the content. At times, the returned digest may differ from that
|
||||||
|
used to initiate a request. Such digests are considered to be from different
|
||||||
|
_domains_, meaning they have different values for _algorithm_. In such a case,
|
||||||
|
the client may choose to verify the digests in both domains or ignore the
|
||||||
|
server's digest. To maintain security, the client _must_ always verify the
|
||||||
|
content against the _digest_ used to fetch the content.
|
||||||
|
|
||||||
|
> __IMPORTANT:__ If a _digest_ is used to fetch content, the client should use
|
||||||
|
> the same digest used to fetch the content to verify it. The header `Docker-
|
||||||
|
> Content-Digest` should not be trusted over the "local" digest.
|
||||||
|
|
||||||
### Pulling An Image
|
### Pulling An Image
|
||||||
|
|
||||||
An "image" is a combination of a JSON manifest and individual layer files. The
|
An "image" is a combination of a JSON manifest and individual layer files. The
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue