mirrors
/
go-mtree
mirror of https://github.com/vbatts/go-mtree.git
1
0
Fork 0
Code Releases Activity

Merge pull request #195 from vbatts/readme_cleanup 

README: formatting, links, updates
This commit is contained in:
Vincent Batts 2023-10-23 21:46:31 -04:00 committed by GitHub
parent e7a79ae8a7 ab2b6ed31f
commit efd1ad5ed5
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 50 additions and 52 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

102
README.md
View File

@ -10,31 +10,29 @@ While the traditional `mtree` cli utility is primarily on BSDs (FreeBSD,
openBSD, etc), even broader support for the `mtree` specification format is
provided with libarchive ([libarchive-formats(5)][libarchive-formats(5)]).
There is also an [mtree port for Linux][archiecobbs/mtree-port] though it is
not widely packaged for Linux distributions.
There is also an [mtree port for Linux][archiecobbs/mtree-port] though it is not widely packaged for Linux distributions.
There was a [Google Summer of Code project to create a portable library and parser for mtree](https://wiki.freebsd.org/SummerOfCode2015/mtreeParsingLibrary).
It is available at [github.com/mratajsky/libmtree](https://github.com/mratajsky/libmtree) and a [talk on it](https://papers.freebsd.org/2016/asiabsdcon/ratajsky-mtree-parsing/).
## Format
The format of hierarchy specification is consistent with the `# mtree v2.0`
format. Both the BSD `mtree` and libarchive ought to be interoperable with it
with only one definite caveat. On Linux, extended attributes (`xattr`) on
files are often a critical aspect of the file, holding ACLs, capabilities, etc.
While FreeBSD filesystem do support `extattr`, this feature has not made its
way into their `mtree`.
The BSD mtree specification is published in [mtree(5)][mtree(5)].
This implementation of mtree supports a few non-upstream "keyword"s, such as:
`xattr` and `tar_time`. If you include these keywords, the FreeBSD `mtree`
will fail, as they are unknown keywords to that implementation.
The format of hierarchy specification is consistent with the `# mtree v2.0` format.
Both the BSD `mtree` and libarchive ought to be interoperable with it with only one definite caveat.
On Linux, extended attributes (`xattr`) on files are often a critical aspect of the file, holding ACLs, capabilities, etc.
While FreeBSD filesystem do support `extattr`, this feature has not made its way into their `mtree`.
To have `go-mtree` produce specifications that will be
strictly compatible with the BSD `mtree`, use the `-bsd-keywords` flag when
creating a manifest. This will make sure that only the keywords supported by
BSD `mtree` are used in the program.
This implementation of mtree supports a few non-upstream "keyword"s, such as: `xattr` and `tar_time`.
If you include these keywords, the FreeBSD `mtree` will fail, as they are unknown keywords to that implementation.
To have `go-mtree` produce specifications that will be strictly compatible with the BSD `mtree`, use the `-bsd-keywords` flag when creating a manifest.
This will make sure that only the keywords supported by BSD `mtree` are used in the program.
### Typical form
With the standard keywords, plus say `sha256digest`, the hierarchy
specification looks like:
With the standard keywords, plus say `sha256digest`, the hierarchy specification looks like:
```mtree
# .
@ -46,9 +44,9 @@ specification looks like:
[...]
```
See the directory presently in, and the files present. Along with each
path, is provided the keywords and the unique values for each path. Any common
keyword and values are established in the `/set` command.
See the directory presently in, and the files present.
Along with each path, is provided the keywords and the unique values for each path.
Any common keyword and values are established in the `/set` command.
### Extended attributes form
@ -62,16 +60,13 @@ keyword and values are established in the `/set` command.
[...]
```
See the keyword prefixed with `xattr.` followed by the extended attribute's
namespace and keyword. This setup is consistent for use with Linux extended
attributes as well as FreeBSD extended attributes.
See the keyword prefixed with `xattr.` followed by the extended attribute's namespace and keyword.
This setup is consistent for use with Linux extended attributes as well as FreeBSD extended attributes.
Since extended attributes are an unordered hashmap, this approach allows for
checking each `<namespace>.<key>` individually.
Since extended attributes are an unordered hashmap, this approach allows for checking each `<namespace>.<key>` individually.
The value is the [base64 encoded][base64] of the value of the particular
extended attribute. Since the values themselves could be raw bytes, this
approach avoids issues with encoding.
The value is the [base64 encoded][base64] of the value of the particular extended attribute.
Since the values themselves could be raw bytes, this approach avoids issues with encoding.
### Tar form
@ -88,21 +83,15 @@ samedir type=dir mode=0775 tar_time=1468000972.000000000
[...]
```
While `go-mtree` serves mainly as a library for upstream `mtree` support,
`go-mtree` is also compatible with [tar archives][tar] (which is not an upstream feature).
While `go-mtree` serves mainly as a library for upstream `mtree` support, `go-mtree` is also compatible with [tar archives][tar] (which is not an upstream feature).
This means that we can now create and validate a manifest by specifying a tar file.
More interestingly, this also means that we can create a manifest from an archive, and then
validate this manifest against a filesystem hierarchy that's on disk, and vice versa.
More interestingly, this also means that we can create a manifest from an archive, and then validate this manifest against a filesystem hierarchy that's on disk, and vice versa.
Notice that for the output of creating a validation manifest from a tar file, the default behavior
for evaluating a notion of time is to use the `tar_time` keyword. In the
"filesystem hierarchy" format of mtree, `time` is being evaluated with
nanosecond precision. However, GNU tar truncates a file's modification time
to 1-second precision. That is, if a file's full modification time is
123456789.123456789, the "tar time" equivalent would be 123456789.000000000.
This way, if you validate a manifest created using a tar file against an
actual root directory, there will be no complaints from `go-mtree` so long as the
1-second precision time of a file in the root directory is the same.
Notice that for the output of creating a validation manifest from a tar file, the default behavior for evaluating a notion of time is to use the `tar_time` keyword.
In the "filesystem hierarchy" format of mtree, `time` is being evaluated with nanosecond precision.
However, GNU tar truncates a file's modification time to 1-second precision.
That is, if a file's full modification time is 123456789.123456789, the "tar time" equivalent would be 123456789.000000000.
This way, if you validate a manifest created using a tar file against an actual root directory, there will be no complaints from `go-mtree` so long as the 1-second precision time of a file in the root directory is the same.
## Usage
@ -114,31 +103,31 @@ To use the command line tool, first [build it](#Building), then the following.
This will also include the sha512 digest of the files.
```bash
```shell
gomtree -c -K sha512digest -p . > /tmp/root.mtree
```
With a tar file:
```bash
```shell
gomtree -c -K sha512digest -T sometarfile.tar > /tmp/tar.mtree
```
### Validate a manifest
```bash
```shell
gomtree -p . -f /tmp/root.mtree
```
With a tar file:
```bash
```shell
gomtree -T sometarfile.tar -f /tmp/root.mtree
```
### See the supported keywords
```bash
```shell
gomtree -list-keywords
Available keywords:
uname
@ -173,37 +162,46 @@ Available keywords:
Either:
```bash
go get github.com/vbatts/go-mtree/cmd/gomtree
```shell
go install github.com/vbatts/go-mtree/cmd/gomtree@latest
```
or
```bash
```shell
git clone git://github.com/vbatts/go-mtree.git $GOPATH/src/github.com/vbatts/go-mtree
cd $GOPATH/src/github.com/vbatts/go-mtree
go build ./cmd/gomtree
```
### Build for many OS/Arch
```shell
make build.arches
```
## Testing
On Linux:
```bash
```shell
cd $GOPATH/src/github.com/vbatts/go-mtree
make
```
On FreeBSD:
```bash
```shell
cd $GOPATH/src/github.com/vbatts/go-mtree
gmake
```
## Related tools
[mtree(5)]: https://man.freebsd.org/cgi/man.cgi?query=mtree&sektion=5&format=html
[mtree(8)]: https://www.freebsd.org/cgi/man.cgi?mtree(8)
[libarchive-formats(5)]: https://www.freebsd.org/cgi/man.cgi?query=libarchive-formats&sektion=5&n=1
[archiecobbs/mtree-port]: https://github.com/archiecobbs/mtree-port
[godoc]: https://godoc.org/github.com/vbatts/go-mtree
[godoc]: https://pkg.go.dev/github.com/vbatts/go-mtree
[tar]: http://man7.org/linux/man-pages/man1/tar.1.html
[base64]: https://tools.ietf.org/html/rfc4648