diff --git a/README.md b/README.md index 369ae07..262975d 100644 --- a/README.md +++ b/README.md @@ -10,31 +10,29 @@ While the traditional `mtree` cli utility is primarily on BSDs (FreeBSD, openBSD, etc), even broader support for the `mtree` specification format is provided with libarchive ([libarchive-formats(5)][libarchive-formats(5)]). -There is also an [mtree port for Linux][archiecobbs/mtree-port] though it is -not widely packaged for Linux distributions. +There is also an [mtree port for Linux][archiecobbs/mtree-port] though it is not widely packaged for Linux distributions. + +There was a [Google Summer of Code project to create a portable library and parser for mtree](https://wiki.freebsd.org/SummerOfCode2015/mtreeParsingLibrary). +It is available at [github.com/mratajsky/libmtree](https://github.com/mratajsky/libmtree) and a [talk on it](https://papers.freebsd.org/2016/asiabsdcon/ratajsky-mtree-parsing/). ## Format -The format of hierarchy specification is consistent with the `# mtree v2.0` -format. Both the BSD `mtree` and libarchive ought to be interoperable with it -with only one definite caveat. On Linux, extended attributes (`xattr`) on -files are often a critical aspect of the file, holding ACLs, capabilities, etc. -While FreeBSD filesystem do support `extattr`, this feature has not made its -way into their `mtree`. +The BSD mtree specification is published in [mtree(5)][mtree(5)]. -This implementation of mtree supports a few non-upstream "keyword"s, such as: -`xattr` and `tar_time`. If you include these keywords, the FreeBSD `mtree` -will fail, as they are unknown keywords to that implementation. +The format of hierarchy specification is consistent with the `# mtree v2.0` format. +Both the BSD `mtree` and libarchive ought to be interoperable with it with only one definite caveat. +On Linux, extended attributes (`xattr`) on files are often a critical aspect of the file, holding ACLs, capabilities, etc. +While FreeBSD filesystem do support `extattr`, this feature has not made its way into their `mtree`. -To have `go-mtree` produce specifications that will be -strictly compatible with the BSD `mtree`, use the `-bsd-keywords` flag when -creating a manifest. This will make sure that only the keywords supported by -BSD `mtree` are used in the program. +This implementation of mtree supports a few non-upstream "keyword"s, such as: `xattr` and `tar_time`. +If you include these keywords, the FreeBSD `mtree` will fail, as they are unknown keywords to that implementation. + +To have `go-mtree` produce specifications that will be strictly compatible with the BSD `mtree`, use the `-bsd-keywords` flag when creating a manifest. +This will make sure that only the keywords supported by BSD `mtree` are used in the program. ### Typical form -With the standard keywords, plus say `sha256digest`, the hierarchy -specification looks like: +With the standard keywords, plus say `sha256digest`, the hierarchy specification looks like: ```mtree # . @@ -46,9 +44,9 @@ specification looks like: [...] ``` -See the directory presently in, and the files present. Along with each -path, is provided the keywords and the unique values for each path. Any common -keyword and values are established in the `/set` command. +See the directory presently in, and the files present. +Along with each path, is provided the keywords and the unique values for each path. +Any common keyword and values are established in the `/set` command. ### Extended attributes form @@ -62,16 +60,13 @@ keyword and values are established in the `/set` command. [...] ``` -See the keyword prefixed with `xattr.` followed by the extended attribute's -namespace and keyword. This setup is consistent for use with Linux extended -attributes as well as FreeBSD extended attributes. +See the keyword prefixed with `xattr.` followed by the extended attribute's namespace and keyword. +This setup is consistent for use with Linux extended attributes as well as FreeBSD extended attributes. -Since extended attributes are an unordered hashmap, this approach allows for -checking each `.` individually. +Since extended attributes are an unordered hashmap, this approach allows for checking each `.` individually. -The value is the [base64 encoded][base64] of the value of the particular -extended attribute. Since the values themselves could be raw bytes, this -approach avoids issues with encoding. +The value is the [base64 encoded][base64] of the value of the particular extended attribute. +Since the values themselves could be raw bytes, this approach avoids issues with encoding. ### Tar form @@ -88,21 +83,15 @@ samedir type=dir mode=0775 tar_time=1468000972.000000000 [...] ``` -While `go-mtree` serves mainly as a library for upstream `mtree` support, -`go-mtree` is also compatible with [tar archives][tar] (which is not an upstream feature). +While `go-mtree` serves mainly as a library for upstream `mtree` support, `go-mtree` is also compatible with [tar archives][tar] (which is not an upstream feature). This means that we can now create and validate a manifest by specifying a tar file. -More interestingly, this also means that we can create a manifest from an archive, and then -validate this manifest against a filesystem hierarchy that's on disk, and vice versa. +More interestingly, this also means that we can create a manifest from an archive, and then validate this manifest against a filesystem hierarchy that's on disk, and vice versa. -Notice that for the output of creating a validation manifest from a tar file, the default behavior -for evaluating a notion of time is to use the `tar_time` keyword. In the -"filesystem hierarchy" format of mtree, `time` is being evaluated with -nanosecond precision. However, GNU tar truncates a file's modification time -to 1-second precision. That is, if a file's full modification time is -123456789.123456789, the "tar time" equivalent would be 123456789.000000000. -This way, if you validate a manifest created using a tar file against an -actual root directory, there will be no complaints from `go-mtree` so long as the -1-second precision time of a file in the root directory is the same. +Notice that for the output of creating a validation manifest from a tar file, the default behavior for evaluating a notion of time is to use the `tar_time` keyword. +In the "filesystem hierarchy" format of mtree, `time` is being evaluated with nanosecond precision. +However, GNU tar truncates a file's modification time to 1-second precision. +That is, if a file's full modification time is 123456789.123456789, the "tar time" equivalent would be 123456789.000000000. +This way, if you validate a manifest created using a tar file against an actual root directory, there will be no complaints from `go-mtree` so long as the 1-second precision time of a file in the root directory is the same. ## Usage @@ -114,31 +103,31 @@ To use the command line tool, first [build it](#Building), then the following. This will also include the sha512 digest of the files. -```bash +```shell gomtree -c -K sha512digest -p . > /tmp/root.mtree ``` With a tar file: -```bash +```shell gomtree -c -K sha512digest -T sometarfile.tar > /tmp/tar.mtree ``` ### Validate a manifest -```bash +```shell gomtree -p . -f /tmp/root.mtree ``` With a tar file: -```bash +```shell gomtree -T sometarfile.tar -f /tmp/root.mtree ``` ### See the supported keywords -```bash +```shell gomtree -list-keywords Available keywords: uname @@ -173,37 +162,46 @@ Available keywords: Either: -```bash -go get github.com/vbatts/go-mtree/cmd/gomtree +```shell +go install github.com/vbatts/go-mtree/cmd/gomtree@latest ``` or -```bash +```shell git clone git://github.com/vbatts/go-mtree.git $GOPATH/src/github.com/vbatts/go-mtree cd $GOPATH/src/github.com/vbatts/go-mtree go build ./cmd/gomtree ``` +### Build for many OS/Arch + +```shell +make build.arches +``` + ## Testing On Linux: -```bash +```shell cd $GOPATH/src/github.com/vbatts/go-mtree make ``` On FreeBSD: -```bash +```shell cd $GOPATH/src/github.com/vbatts/go-mtree gmake ``` +## Related tools + +[mtree(5)]: https://man.freebsd.org/cgi/man.cgi?query=mtree&sektion=5&format=html [mtree(8)]: https://www.freebsd.org/cgi/man.cgi?mtree(8) [libarchive-formats(5)]: https://www.freebsd.org/cgi/man.cgi?query=libarchive-formats&sektion=5&n=1 [archiecobbs/mtree-port]: https://github.com/archiecobbs/mtree-port -[godoc]: https://godoc.org/github.com/vbatts/go-mtree +[godoc]: https://pkg.go.dev/github.com/vbatts/go-mtree [tar]: http://man7.org/linux/man-pages/man1/tar.1.html [base64]: https://tools.ietf.org/html/rfc4648