mirror of
https://github.com/vbatts/go-mtree.git
synced 2024-12-18 12:06:30 +00:00
README: formatting, links, updates
Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
This commit is contained in:
parent
e7a79ae8a7
commit
ab2b6ed31f
1 changed files with 50 additions and 52 deletions
102
README.md
102
README.md
|
@ -10,31 +10,29 @@ While the traditional `mtree` cli utility is primarily on BSDs (FreeBSD,
|
||||||
openBSD, etc), even broader support for the `mtree` specification format is
|
openBSD, etc), even broader support for the `mtree` specification format is
|
||||||
provided with libarchive ([libarchive-formats(5)][libarchive-formats(5)]).
|
provided with libarchive ([libarchive-formats(5)][libarchive-formats(5)]).
|
||||||
|
|
||||||
There is also an [mtree port for Linux][archiecobbs/mtree-port] though it is
|
There is also an [mtree port for Linux][archiecobbs/mtree-port] though it is not widely packaged for Linux distributions.
|
||||||
not widely packaged for Linux distributions.
|
|
||||||
|
There was a [Google Summer of Code project to create a portable library and parser for mtree](https://wiki.freebsd.org/SummerOfCode2015/mtreeParsingLibrary).
|
||||||
|
It is available at [github.com/mratajsky/libmtree](https://github.com/mratajsky/libmtree) and a [talk on it](https://papers.freebsd.org/2016/asiabsdcon/ratajsky-mtree-parsing/).
|
||||||
|
|
||||||
## Format
|
## Format
|
||||||
|
|
||||||
The format of hierarchy specification is consistent with the `# mtree v2.0`
|
The BSD mtree specification is published in [mtree(5)][mtree(5)].
|
||||||
format. Both the BSD `mtree` and libarchive ought to be interoperable with it
|
|
||||||
with only one definite caveat. On Linux, extended attributes (`xattr`) on
|
|
||||||
files are often a critical aspect of the file, holding ACLs, capabilities, etc.
|
|
||||||
While FreeBSD filesystem do support `extattr`, this feature has not made its
|
|
||||||
way into their `mtree`.
|
|
||||||
|
|
||||||
This implementation of mtree supports a few non-upstream "keyword"s, such as:
|
The format of hierarchy specification is consistent with the `# mtree v2.0` format.
|
||||||
`xattr` and `tar_time`. If you include these keywords, the FreeBSD `mtree`
|
Both the BSD `mtree` and libarchive ought to be interoperable with it with only one definite caveat.
|
||||||
will fail, as they are unknown keywords to that implementation.
|
On Linux, extended attributes (`xattr`) on files are often a critical aspect of the file, holding ACLs, capabilities, etc.
|
||||||
|
While FreeBSD filesystem do support `extattr`, this feature has not made its way into their `mtree`.
|
||||||
|
|
||||||
To have `go-mtree` produce specifications that will be
|
This implementation of mtree supports a few non-upstream "keyword"s, such as: `xattr` and `tar_time`.
|
||||||
strictly compatible with the BSD `mtree`, use the `-bsd-keywords` flag when
|
If you include these keywords, the FreeBSD `mtree` will fail, as they are unknown keywords to that implementation.
|
||||||
creating a manifest. This will make sure that only the keywords supported by
|
|
||||||
BSD `mtree` are used in the program.
|
To have `go-mtree` produce specifications that will be strictly compatible with the BSD `mtree`, use the `-bsd-keywords` flag when creating a manifest.
|
||||||
|
This will make sure that only the keywords supported by BSD `mtree` are used in the program.
|
||||||
|
|
||||||
### Typical form
|
### Typical form
|
||||||
|
|
||||||
With the standard keywords, plus say `sha256digest`, the hierarchy
|
With the standard keywords, plus say `sha256digest`, the hierarchy specification looks like:
|
||||||
specification looks like:
|
|
||||||
|
|
||||||
```mtree
|
```mtree
|
||||||
# .
|
# .
|
||||||
|
@ -46,9 +44,9 @@ specification looks like:
|
||||||
[...]
|
[...]
|
||||||
```
|
```
|
||||||
|
|
||||||
See the directory presently in, and the files present. Along with each
|
See the directory presently in, and the files present.
|
||||||
path, is provided the keywords and the unique values for each path. Any common
|
Along with each path, is provided the keywords and the unique values for each path.
|
||||||
keyword and values are established in the `/set` command.
|
Any common keyword and values are established in the `/set` command.
|
||||||
|
|
||||||
### Extended attributes form
|
### Extended attributes form
|
||||||
|
|
||||||
|
@ -62,16 +60,13 @@ keyword and values are established in the `/set` command.
|
||||||
[...]
|
[...]
|
||||||
```
|
```
|
||||||
|
|
||||||
See the keyword prefixed with `xattr.` followed by the extended attribute's
|
See the keyword prefixed with `xattr.` followed by the extended attribute's namespace and keyword.
|
||||||
namespace and keyword. This setup is consistent for use with Linux extended
|
This setup is consistent for use with Linux extended attributes as well as FreeBSD extended attributes.
|
||||||
attributes as well as FreeBSD extended attributes.
|
|
||||||
|
|
||||||
Since extended attributes are an unordered hashmap, this approach allows for
|
Since extended attributes are an unordered hashmap, this approach allows for checking each `<namespace>.<key>` individually.
|
||||||
checking each `<namespace>.<key>` individually.
|
|
||||||
|
|
||||||
The value is the [base64 encoded][base64] of the value of the particular
|
The value is the [base64 encoded][base64] of the value of the particular extended attribute.
|
||||||
extended attribute. Since the values themselves could be raw bytes, this
|
Since the values themselves could be raw bytes, this approach avoids issues with encoding.
|
||||||
approach avoids issues with encoding.
|
|
||||||
|
|
||||||
### Tar form
|
### Tar form
|
||||||
|
|
||||||
|
@ -88,21 +83,15 @@ samedir type=dir mode=0775 tar_time=1468000972.000000000
|
||||||
[...]
|
[...]
|
||||||
```
|
```
|
||||||
|
|
||||||
While `go-mtree` serves mainly as a library for upstream `mtree` support,
|
While `go-mtree` serves mainly as a library for upstream `mtree` support, `go-mtree` is also compatible with [tar archives][tar] (which is not an upstream feature).
|
||||||
`go-mtree` is also compatible with [tar archives][tar] (which is not an upstream feature).
|
|
||||||
This means that we can now create and validate a manifest by specifying a tar file.
|
This means that we can now create and validate a manifest by specifying a tar file.
|
||||||
More interestingly, this also means that we can create a manifest from an archive, and then
|
More interestingly, this also means that we can create a manifest from an archive, and then validate this manifest against a filesystem hierarchy that's on disk, and vice versa.
|
||||||
validate this manifest against a filesystem hierarchy that's on disk, and vice versa.
|
|
||||||
|
|
||||||
Notice that for the output of creating a validation manifest from a tar file, the default behavior
|
Notice that for the output of creating a validation manifest from a tar file, the default behavior for evaluating a notion of time is to use the `tar_time` keyword.
|
||||||
for evaluating a notion of time is to use the `tar_time` keyword. In the
|
In the "filesystem hierarchy" format of mtree, `time` is being evaluated with nanosecond precision.
|
||||||
"filesystem hierarchy" format of mtree, `time` is being evaluated with
|
However, GNU tar truncates a file's modification time to 1-second precision.
|
||||||
nanosecond precision. However, GNU tar truncates a file's modification time
|
That is, if a file's full modification time is 123456789.123456789, the "tar time" equivalent would be 123456789.000000000.
|
||||||
to 1-second precision. That is, if a file's full modification time is
|
This way, if you validate a manifest created using a tar file against an actual root directory, there will be no complaints from `go-mtree` so long as the 1-second precision time of a file in the root directory is the same.
|
||||||
123456789.123456789, the "tar time" equivalent would be 123456789.000000000.
|
|
||||||
This way, if you validate a manifest created using a tar file against an
|
|
||||||
actual root directory, there will be no complaints from `go-mtree` so long as the
|
|
||||||
1-second precision time of a file in the root directory is the same.
|
|
||||||
|
|
||||||
## Usage
|
## Usage
|
||||||
|
|
||||||
|
@ -114,31 +103,31 @@ To use the command line tool, first [build it](#Building), then the following.
|
||||||
|
|
||||||
This will also include the sha512 digest of the files.
|
This will also include the sha512 digest of the files.
|
||||||
|
|
||||||
```bash
|
```shell
|
||||||
gomtree -c -K sha512digest -p . > /tmp/root.mtree
|
gomtree -c -K sha512digest -p . > /tmp/root.mtree
|
||||||
```
|
```
|
||||||
|
|
||||||
With a tar file:
|
With a tar file:
|
||||||
|
|
||||||
```bash
|
```shell
|
||||||
gomtree -c -K sha512digest -T sometarfile.tar > /tmp/tar.mtree
|
gomtree -c -K sha512digest -T sometarfile.tar > /tmp/tar.mtree
|
||||||
```
|
```
|
||||||
|
|
||||||
### Validate a manifest
|
### Validate a manifest
|
||||||
|
|
||||||
```bash
|
```shell
|
||||||
gomtree -p . -f /tmp/root.mtree
|
gomtree -p . -f /tmp/root.mtree
|
||||||
```
|
```
|
||||||
|
|
||||||
With a tar file:
|
With a tar file:
|
||||||
|
|
||||||
```bash
|
```shell
|
||||||
gomtree -T sometarfile.tar -f /tmp/root.mtree
|
gomtree -T sometarfile.tar -f /tmp/root.mtree
|
||||||
```
|
```
|
||||||
|
|
||||||
### See the supported keywords
|
### See the supported keywords
|
||||||
|
|
||||||
```bash
|
```shell
|
||||||
gomtree -list-keywords
|
gomtree -list-keywords
|
||||||
Available keywords:
|
Available keywords:
|
||||||
uname
|
uname
|
||||||
|
@ -173,37 +162,46 @@ Available keywords:
|
||||||
|
|
||||||
Either:
|
Either:
|
||||||
|
|
||||||
```bash
|
```shell
|
||||||
go get github.com/vbatts/go-mtree/cmd/gomtree
|
go install github.com/vbatts/go-mtree/cmd/gomtree@latest
|
||||||
```
|
```
|
||||||
|
|
||||||
or
|
or
|
||||||
|
|
||||||
```bash
|
```shell
|
||||||
git clone git://github.com/vbatts/go-mtree.git $GOPATH/src/github.com/vbatts/go-mtree
|
git clone git://github.com/vbatts/go-mtree.git $GOPATH/src/github.com/vbatts/go-mtree
|
||||||
cd $GOPATH/src/github.com/vbatts/go-mtree
|
cd $GOPATH/src/github.com/vbatts/go-mtree
|
||||||
go build ./cmd/gomtree
|
go build ./cmd/gomtree
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Build for many OS/Arch
|
||||||
|
|
||||||
|
```shell
|
||||||
|
make build.arches
|
||||||
|
```
|
||||||
|
|
||||||
## Testing
|
## Testing
|
||||||
|
|
||||||
On Linux:
|
On Linux:
|
||||||
|
|
||||||
```bash
|
```shell
|
||||||
cd $GOPATH/src/github.com/vbatts/go-mtree
|
cd $GOPATH/src/github.com/vbatts/go-mtree
|
||||||
make
|
make
|
||||||
```
|
```
|
||||||
|
|
||||||
On FreeBSD:
|
On FreeBSD:
|
||||||
|
|
||||||
```bash
|
```shell
|
||||||
cd $GOPATH/src/github.com/vbatts/go-mtree
|
cd $GOPATH/src/github.com/vbatts/go-mtree
|
||||||
gmake
|
gmake
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## Related tools
|
||||||
|
|
||||||
|
[mtree(5)]: https://man.freebsd.org/cgi/man.cgi?query=mtree&sektion=5&format=html
|
||||||
[mtree(8)]: https://www.freebsd.org/cgi/man.cgi?mtree(8)
|
[mtree(8)]: https://www.freebsd.org/cgi/man.cgi?mtree(8)
|
||||||
[libarchive-formats(5)]: https://www.freebsd.org/cgi/man.cgi?query=libarchive-formats&sektion=5&n=1
|
[libarchive-formats(5)]: https://www.freebsd.org/cgi/man.cgi?query=libarchive-formats&sektion=5&n=1
|
||||||
[archiecobbs/mtree-port]: https://github.com/archiecobbs/mtree-port
|
[archiecobbs/mtree-port]: https://github.com/archiecobbs/mtree-port
|
||||||
[godoc]: https://godoc.org/github.com/vbatts/go-mtree
|
[godoc]: https://pkg.go.dev/github.com/vbatts/go-mtree
|
||||||
[tar]: http://man7.org/linux/man-pages/man1/tar.1.html
|
[tar]: http://man7.org/linux/man-pages/man1/tar.1.html
|
||||||
[base64]: https://tools.ietf.org/html/rfc4648
|
[base64]: https://tools.ietf.org/html/rfc4648
|
||||||
|
|
Loading…
Reference in a new issue