mirror of https://github.com/vbatts/go-mtree.git
208 lines
7.0 KiB
Markdown
208 lines
7.0 KiB
Markdown
# go-mtree
|
|
|
|
[](https://github.com/vbatts/go-mtree/actions/workflows/go.yml)
|
|
[](https://goreportcard.com/report/github.com/vbatts/go-mtree)
|
|
|
|
`mtree` is a filesystem hierarchy validation tooling and format.
|
|
This is a library and simple cli tool for [mtree(8)][mtree(8)] support.
|
|
|
|
While the traditional `mtree` cli utility is primarily on BSDs (FreeBSD,
|
|
openBSD, etc), even broader support for the `mtree` specification format is
|
|
provided with libarchive ([libarchive-formats(5)][libarchive-formats(5)]).
|
|
|
|
There is also an [mtree port for Linux][archiecobbs/mtree-port] though it is not widely packaged for Linux distributions.
|
|
|
|
There was a [Google Summer of Code project to create a portable library and parser for mtree](https://wiki.freebsd.org/SummerOfCode2015/mtreeParsingLibrary).
|
|
It is available at [github.com/mratajsky/libmtree](https://github.com/mratajsky/libmtree) and a [talk on it](https://papers.freebsd.org/2016/asiabsdcon/ratajsky-mtree-parsing/).
|
|
|
|
## Format
|
|
|
|
The BSD mtree specification is published in [mtree(5)][mtree(5)].
|
|
|
|
The format of hierarchy specification is consistent with the `# mtree v2.0` format.
|
|
Both the BSD `mtree` and libarchive ought to be interoperable with it with only one definite caveat.
|
|
On Linux, extended attributes (`xattr`) on files are often a critical aspect of the file, holding ACLs, capabilities, etc.
|
|
While FreeBSD filesystem do support `extattr`, this feature has not made its way into their `mtree`.
|
|
|
|
This implementation of mtree supports a few non-upstream "keyword"s, such as: `xattr` and `tar_time`.
|
|
If you include these keywords, the FreeBSD `mtree` will fail, as they are unknown keywords to that implementation.
|
|
|
|
To have `go-mtree` produce specifications that will be strictly compatible with the BSD `mtree`, use the `-bsd-keywords` flag when creating a manifest.
|
|
This will make sure that only the keywords supported by BSD `mtree` are used in the program.
|
|
|
|
### Typical form
|
|
|
|
With the standard keywords, plus say `sha256digest`, the hierarchy specification looks like:
|
|
|
|
```mtree
|
|
# .
|
|
/set type=file nlink=1 mode=0664 uid=1000 gid=100
|
|
. size=4096 type=dir mode=0755 nlink=6 time=1459370393.273231538
|
|
LICENSE size=1502 mode=0644 time=1458851690.0 sha256digest=ef4e53d83096be56dc38dbf9bc8ba9e3068bec1ec37c179033d1e8f99a1c2a95
|
|
README.md size=2820 mode=0644 time=1459370256.316148361 sha256digest=d9b955134d99f84b17c0a711ce507515cc93cd7080a9dcd50400e3d993d876ac
|
|
|
|
[...]
|
|
```
|
|
|
|
See the directory presently in, and the files present.
|
|
Along with each path, is provided the keywords and the unique values for each path.
|
|
Any common keyword and values are established in the `/set` command.
|
|
|
|
### Extended attributes form
|
|
|
|
```mtree
|
|
# .
|
|
/set type=file nlink=1 mode=0664 uid=1000 gid=1000
|
|
. size=4096 type=dir mode=0775 nlink=6 time=1459370191.11179595 xattr.security.selinux=dW5jb25maW5lZF91Om9iamVjdF9yOnVzZXJfaG9tZV90OnMwAA==
|
|
LICENSE size=1502 time=1458851690.583562292 xattr.security.selinux=dW5jb25maW5lZF91Om9iamVjdF9yOnVzZXJfaG9tZV90OnMwAA==
|
|
README.md size=2366 mode=0644 time=1459369604.0 xattr.security.selinux=dW5jb25maW5lZF91Om9iamVjdF9yOnVzZXJfaG9tZV90OnMwAA==
|
|
|
|
[...]
|
|
```
|
|
|
|
See the keyword prefixed with `xattr.` followed by the extended attribute's namespace and keyword.
|
|
This setup is consistent for use with Linux extended attributes as well as FreeBSD extended attributes.
|
|
|
|
Since extended attributes are an unordered hashmap, this approach allows for checking each `<namespace>.<key>` individually.
|
|
|
|
The value is the [base64 encoded][base64] of the value of the particular extended attribute.
|
|
Since the values themselves could be raw bytes, this approach avoids issues with encoding.
|
|
|
|
### Tar form
|
|
|
|
```mtree
|
|
# .
|
|
/set type=file mode=0664 uid=1000 gid=1000
|
|
. type=dir mode=0775 tar_time=1468430408.000000000
|
|
|
|
# samedir
|
|
samedir type=dir mode=0775 tar_time=1468000972.000000000
|
|
file2 size=0 tar_time=1467999782.000000000
|
|
file1 size=0 tar_time=1467999781.000000000
|
|
|
|
[...]
|
|
```
|
|
|
|
While `go-mtree` serves mainly as a library for upstream `mtree` support, `go-mtree` is also compatible with [tar archives][tar] (which is not an upstream feature).
|
|
This means that we can now create and validate a manifest by specifying a tar file.
|
|
More interestingly, this also means that we can create a manifest from an archive, and then validate this manifest against a filesystem hierarchy that's on disk, and vice versa.
|
|
|
|
Notice that for the output of creating a validation manifest from a tar file, the default behavior for evaluating a notion of time is to use the `tar_time` keyword.
|
|
In the "filesystem hierarchy" format of mtree, `time` is being evaluated with nanosecond precision.
|
|
However, GNU tar truncates a file's modification time to 1-second precision.
|
|
That is, if a file's full modification time is 123456789.123456789, the "tar time" equivalent would be 123456789.000000000.
|
|
This way, if you validate a manifest created using a tar file against an actual root directory, there will be no complaints from `go-mtree` so long as the 1-second precision time of a file in the root directory is the same.
|
|
|
|
## Usage
|
|
|
|
To use the Go programming language library, see [the docs][godoc].
|
|
|
|
To use the command line tool, first [build it](#Building), then the following.
|
|
|
|
### Create a manifest
|
|
|
|
This will also include the sha512 digest of the files.
|
|
|
|
```shell
|
|
gomtree validate -c -K sha512digest -p . > /tmp/root.mtree
|
|
```
|
|
|
|
With a tar file:
|
|
|
|
```shell
|
|
gomtree validate -c -K sha512digest -T sometarfile.tar > /tmp/tar.mtree
|
|
```
|
|
|
|
### Validate a manifest
|
|
|
|
```shell
|
|
gomtree validate -p . -f /tmp/root.mtree
|
|
```
|
|
|
|
With a tar file:
|
|
|
|
```shell
|
|
gomtree validate -T sometarfile.tar -f /tmp/root.mtree
|
|
```
|
|
|
|
### See the supported keywords
|
|
|
|
```shell
|
|
gomtree validate -list-keywords
|
|
Available keywords:
|
|
uname
|
|
sha1
|
|
sha1digest
|
|
sha256digest
|
|
xattrs (not upstream)
|
|
link (default)
|
|
nlink (default)
|
|
md5digest
|
|
rmd160digest
|
|
mode (default)
|
|
cksum
|
|
md5
|
|
rmd160
|
|
type (default)
|
|
time (default)
|
|
uid (default)
|
|
gid (default)
|
|
sha256
|
|
sha384
|
|
sha512
|
|
xattr (not upstream)
|
|
tar_time (not upstream)
|
|
size (default)
|
|
ripemd160digest
|
|
sha384digest
|
|
sha512digest
|
|
```
|
|
|
|
## Building
|
|
|
|
Either:
|
|
|
|
```shell
|
|
go install github.com/vbatts/go-mtree/cmd/gomtree@latest
|
|
```
|
|
|
|
or
|
|
|
|
```shell
|
|
git clone git://github.com/vbatts/go-mtree.git $GOPATH/src/github.com/vbatts/go-mtree
|
|
cd $GOPATH/src/github.com/vbatts/go-mtree
|
|
go build ./cmd/gomtree
|
|
```
|
|
|
|
### Build for many OS/Arch
|
|
|
|
```shell
|
|
make build.arches
|
|
```
|
|
|
|
## Testing
|
|
|
|
On Linux:
|
|
|
|
```shell
|
|
cd $GOPATH/src/github.com/vbatts/go-mtree
|
|
make
|
|
```
|
|
|
|
On FreeBSD:
|
|
|
|
```shell
|
|
cd $GOPATH/src/github.com/vbatts/go-mtree
|
|
gmake
|
|
```
|
|
|
|
## Related tools
|
|
|
|
[mtree(5)]: https://man.freebsd.org/cgi/man.cgi?query=mtree&sektion=5&format=html
|
|
[mtree(8)]: https://www.freebsd.org/cgi/man.cgi?mtree(8)
|
|
[libarchive-formats(5)]: https://www.freebsd.org/cgi/man.cgi?query=libarchive-formats&sektion=5&n=1
|
|
[archiecobbs/mtree-port]: https://github.com/archiecobbs/mtree-port
|
|
[godoc]: https://pkg.go.dev/github.com/vbatts/go-mtree
|
|
[tar]: http://man7.org/linux/man-pages/man1/tar.1.html
|
|
[base64]: https://tools.ietf.org/html/rfc4648
|