Merge pull request #195 from vbatts/readme_cleanup

README: formatting, links, updates
This commit is contained in:
Vincent Batts 2023-10-23 21:46:31 -04:00 committed by GitHub
commit efd1ad5ed5
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 50 additions and 52 deletions

102
README.md
View File

@ -10,31 +10,29 @@ While the traditional `mtree` cli utility is primarily on BSDs (FreeBSD,
openBSD, etc), even broader support for the `mtree` specification format is openBSD, etc), even broader support for the `mtree` specification format is
provided with libarchive ([libarchive-formats(5)][libarchive-formats(5)]). provided with libarchive ([libarchive-formats(5)][libarchive-formats(5)]).
There is also an [mtree port for Linux][archiecobbs/mtree-port] though it is There is also an [mtree port for Linux][archiecobbs/mtree-port] though it is not widely packaged for Linux distributions.
not widely packaged for Linux distributions.
There was a [Google Summer of Code project to create a portable library and parser for mtree](https://wiki.freebsd.org/SummerOfCode2015/mtreeParsingLibrary).
It is available at [github.com/mratajsky/libmtree](https://github.com/mratajsky/libmtree) and a [talk on it](https://papers.freebsd.org/2016/asiabsdcon/ratajsky-mtree-parsing/).
## Format ## Format
The format of hierarchy specification is consistent with the `# mtree v2.0` The BSD mtree specification is published in [mtree(5)][mtree(5)].
format. Both the BSD `mtree` and libarchive ought to be interoperable with it
with only one definite caveat. On Linux, extended attributes (`xattr`) on
files are often a critical aspect of the file, holding ACLs, capabilities, etc.
While FreeBSD filesystem do support `extattr`, this feature has not made its
way into their `mtree`.
This implementation of mtree supports a few non-upstream "keyword"s, such as: The format of hierarchy specification is consistent with the `# mtree v2.0` format.
`xattr` and `tar_time`. If you include these keywords, the FreeBSD `mtree` Both the BSD `mtree` and libarchive ought to be interoperable with it with only one definite caveat.
will fail, as they are unknown keywords to that implementation. On Linux, extended attributes (`xattr`) on files are often a critical aspect of the file, holding ACLs, capabilities, etc.
While FreeBSD filesystem do support `extattr`, this feature has not made its way into their `mtree`.
To have `go-mtree` produce specifications that will be This implementation of mtree supports a few non-upstream "keyword"s, such as: `xattr` and `tar_time`.
strictly compatible with the BSD `mtree`, use the `-bsd-keywords` flag when If you include these keywords, the FreeBSD `mtree` will fail, as they are unknown keywords to that implementation.
creating a manifest. This will make sure that only the keywords supported by
BSD `mtree` are used in the program. To have `go-mtree` produce specifications that will be strictly compatible with the BSD `mtree`, use the `-bsd-keywords` flag when creating a manifest.
This will make sure that only the keywords supported by BSD `mtree` are used in the program.
### Typical form ### Typical form
With the standard keywords, plus say `sha256digest`, the hierarchy With the standard keywords, plus say `sha256digest`, the hierarchy specification looks like:
specification looks like:
```mtree ```mtree
# . # .
@ -46,9 +44,9 @@ specification looks like:
[...] [...]
``` ```
See the directory presently in, and the files present. Along with each See the directory presently in, and the files present.
path, is provided the keywords and the unique values for each path. Any common Along with each path, is provided the keywords and the unique values for each path.
keyword and values are established in the `/set` command. Any common keyword and values are established in the `/set` command.
### Extended attributes form ### Extended attributes form
@ -62,16 +60,13 @@ keyword and values are established in the `/set` command.
[...] [...]
``` ```
See the keyword prefixed with `xattr.` followed by the extended attribute's See the keyword prefixed with `xattr.` followed by the extended attribute's namespace and keyword.
namespace and keyword. This setup is consistent for use with Linux extended This setup is consistent for use with Linux extended attributes as well as FreeBSD extended attributes.
attributes as well as FreeBSD extended attributes.
Since extended attributes are an unordered hashmap, this approach allows for Since extended attributes are an unordered hashmap, this approach allows for checking each `<namespace>.<key>` individually.
checking each `<namespace>.<key>` individually.
The value is the [base64 encoded][base64] of the value of the particular The value is the [base64 encoded][base64] of the value of the particular extended attribute.
extended attribute. Since the values themselves could be raw bytes, this Since the values themselves could be raw bytes, this approach avoids issues with encoding.
approach avoids issues with encoding.
### Tar form ### Tar form
@ -88,21 +83,15 @@ samedir type=dir mode=0775 tar_time=1468000972.000000000
[...] [...]
``` ```
While `go-mtree` serves mainly as a library for upstream `mtree` support, While `go-mtree` serves mainly as a library for upstream `mtree` support, `go-mtree` is also compatible with [tar archives][tar] (which is not an upstream feature).
`go-mtree` is also compatible with [tar archives][tar] (which is not an upstream feature).
This means that we can now create and validate a manifest by specifying a tar file. This means that we can now create and validate a manifest by specifying a tar file.
More interestingly, this also means that we can create a manifest from an archive, and then More interestingly, this also means that we can create a manifest from an archive, and then validate this manifest against a filesystem hierarchy that's on disk, and vice versa.
validate this manifest against a filesystem hierarchy that's on disk, and vice versa.
Notice that for the output of creating a validation manifest from a tar file, the default behavior Notice that for the output of creating a validation manifest from a tar file, the default behavior for evaluating a notion of time is to use the `tar_time` keyword.
for evaluating a notion of time is to use the `tar_time` keyword. In the In the "filesystem hierarchy" format of mtree, `time` is being evaluated with nanosecond precision.
"filesystem hierarchy" format of mtree, `time` is being evaluated with However, GNU tar truncates a file's modification time to 1-second precision.
nanosecond precision. However, GNU tar truncates a file's modification time That is, if a file's full modification time is 123456789.123456789, the "tar time" equivalent would be 123456789.000000000.
to 1-second precision. That is, if a file's full modification time is This way, if you validate a manifest created using a tar file against an actual root directory, there will be no complaints from `go-mtree` so long as the 1-second precision time of a file in the root directory is the same.
123456789.123456789, the "tar time" equivalent would be 123456789.000000000.
This way, if you validate a manifest created using a tar file against an
actual root directory, there will be no complaints from `go-mtree` so long as the
1-second precision time of a file in the root directory is the same.
## Usage ## Usage
@ -114,31 +103,31 @@ To use the command line tool, first [build it](#Building), then the following.
This will also include the sha512 digest of the files. This will also include the sha512 digest of the files.
```bash ```shell
gomtree -c -K sha512digest -p . > /tmp/root.mtree gomtree -c -K sha512digest -p . > /tmp/root.mtree
``` ```
With a tar file: With a tar file:
```bash ```shell
gomtree -c -K sha512digest -T sometarfile.tar > /tmp/tar.mtree gomtree -c -K sha512digest -T sometarfile.tar > /tmp/tar.mtree
``` ```
### Validate a manifest ### Validate a manifest
```bash ```shell
gomtree -p . -f /tmp/root.mtree gomtree -p . -f /tmp/root.mtree
``` ```
With a tar file: With a tar file:
```bash ```shell
gomtree -T sometarfile.tar -f /tmp/root.mtree gomtree -T sometarfile.tar -f /tmp/root.mtree
``` ```
### See the supported keywords ### See the supported keywords
```bash ```shell
gomtree -list-keywords gomtree -list-keywords
Available keywords: Available keywords:
uname uname
@ -173,37 +162,46 @@ Available keywords:
Either: Either:
```bash ```shell
go get github.com/vbatts/go-mtree/cmd/gomtree go install github.com/vbatts/go-mtree/cmd/gomtree@latest
``` ```
or or
```bash ```shell
git clone git://github.com/vbatts/go-mtree.git $GOPATH/src/github.com/vbatts/go-mtree git clone git://github.com/vbatts/go-mtree.git $GOPATH/src/github.com/vbatts/go-mtree
cd $GOPATH/src/github.com/vbatts/go-mtree cd $GOPATH/src/github.com/vbatts/go-mtree
go build ./cmd/gomtree go build ./cmd/gomtree
``` ```
### Build for many OS/Arch
```shell
make build.arches
```
## Testing ## Testing
On Linux: On Linux:
```bash ```shell
cd $GOPATH/src/github.com/vbatts/go-mtree cd $GOPATH/src/github.com/vbatts/go-mtree
make make
``` ```
On FreeBSD: On FreeBSD:
```bash ```shell
cd $GOPATH/src/github.com/vbatts/go-mtree cd $GOPATH/src/github.com/vbatts/go-mtree
gmake gmake
``` ```
## Related tools
[mtree(5)]: https://man.freebsd.org/cgi/man.cgi?query=mtree&sektion=5&format=html
[mtree(8)]: https://www.freebsd.org/cgi/man.cgi?mtree(8) [mtree(8)]: https://www.freebsd.org/cgi/man.cgi?mtree(8)
[libarchive-formats(5)]: https://www.freebsd.org/cgi/man.cgi?query=libarchive-formats&sektion=5&n=1 [libarchive-formats(5)]: https://www.freebsd.org/cgi/man.cgi?query=libarchive-formats&sektion=5&n=1
[archiecobbs/mtree-port]: https://github.com/archiecobbs/mtree-port [archiecobbs/mtree-port]: https://github.com/archiecobbs/mtree-port
[godoc]: https://godoc.org/github.com/vbatts/go-mtree [godoc]: https://pkg.go.dev/github.com/vbatts/go-mtree
[tar]: http://man7.org/linux/man-pages/man1/tar.1.html [tar]: http://man7.org/linux/man-pages/man1/tar.1.html
[base64]: https://tools.ietf.org/html/rfc4648 [base64]: https://tools.ietf.org/html/rfc4648