vbatts/go-mtree
1
0
Fork 0
Code Issues Pull requests Releases Activity
File systems verification utility and library, in likeness of mtree(8) https://github.com/vbatts/go-mtree
62 commits 3 branches 12 tags 5.7 MiB
Go 94.8%
Shell 4%
Makefile 1.2%
Find a file
Stephen Chung 1c5b44f1df keywords: return "link" keyword along with its value 
Originally only returning `Sys.linkname` when evaluating
a tar header's link field. We want it to be in the form of
keyword=value instead.

Signed-off-by: Stephen Chung <schung@redhat.com>
2016-07-20 17:41:17 -04:00
cmd/gomtree *: refactoring to support streams 2016-07-20 17:41:17 -04:00
testdata *: refactoring to support streams 2016-07-20 17:41:17 -04:00
xattr travis: add travis check 2016-04-06 13:57:11 -04:00
.travis.yml travis: go1.5 and go1.6 have vet builtin 2016-04-13 15:34:50 -04:00
check.go *: refactoring to support streams 2016-07-20 17:41:17 -04:00
check_test.go Merge pull request #22 from stephen679/set_aliasing 2016-07-20 07:40:19 +09:00
cksum.go cksum: comment 2016-03-24 16:35:09 -04:00
cksum_test.go cksum: test is fine. commenting. 2016-04-05 16:47:36 -04:00
creator.go *: refactoring to support streams 2016-07-20 17:41:17 -04:00
entry.go *: refactoring to support streams 2016-07-20 17:41:17 -04:00
hierarchy.go *: refactoring to support streams 2016-07-20 17:41:17 -04:00
keywords.go keywords: return "link" keyword along with its value 2016-07-20 17:41:17 -04:00
keywords_linux.go *: refactoring to support streams 2016-07-20 17:41:17 -04:00
keywords_unsupported.go *: refactoring to support streams 2016-07-20 17:41:17 -04:00
LICENSE LICENSE: adding a license 2016-03-24 16:34:58 -04:00
mtree_test.go hierarchy: testing works 2016-04-05 17:16:44 -04:00
parse.go parse: ignore leading whitespace for comments 2016-07-17 12:21:52 -04:00
README.md main: change -l to -list-keywords 2016-07-20 11:39:32 -04:00
tar.go *: refactoring to support streams 2016-07-20 17:41:17 -04:00
tar_test.go *: refactoring to support streams 2016-07-20 17:41:17 -04:00
walk.go *: refactoring to support streams 2016-07-20 17:41:17 -04:00
walk_test.go walk: test passes and validates parse 2016-04-06 12:45:08 -04:00

README.md

go-mtree

mtree is a filesystem hierarchy validation tooling and format. This is a library and simple cli tool for mtree(8) support.

While the traditional mtree cli utility is primarily on BSDs (FreeBSD, openBSD, etc), but even broader support for the mtree specification format is provided with libarchive (libarchive-formats(5)).

There is also an mtree port for Linux though it is not widely packaged for Linux distributions.

Format

The format of hierarchy specification is consistent with the # mtree v2.0 format. Both the BSD mtree and libarchive ought to be interoperable with it with only one definite caveat. On Linux, extended attributes (xattr) on files are often a critical aspect of the file, holding ACLs, capabilities, etc. While FreeBSD filesystem do support extattr, this feature has not made its way into their mtree.

This implementation of mtree supports an additional "keyword" of xattr. If you include this keyword, then the FreeBSD mtree will fail as it is an unknown keyword to that implementation.

Typical form

With the standard keywords, plus say sha256digest, the hierarchy specification looks like:

# .
/set type=file nlink=1 mode=0664 uid=1000 gid=100
. size=4096 type=dir mode=0755 nlink=6 time=1459370393.273231538
    LICENSE size=1502 mode=0644 time=1458851690.0 sha256digest=ef4e53d83096be56dc38dbf9bc8ba9e3068bec1ec37c179033d1e8f99a1c2a95
    README.md size=2820 mode=0644 time=1459370256.316148361 sha256digest=d9b955134d99f84b17c0a711ce507515cc93cd7080a9dcd50400e3d993d876ac

[...]

See the directory presently in, and the files present. Along with each path, is provided the keywords and the unique values for each path. Any common keyword and values are established in the /set command.

Extended attributes form

# .
/set type=file nlink=1 mode=0664 uid=1000 gid=1000
. size=4096 type=dir mode=0775 nlink=6 time=1459370191.11179595 xattr.security.selinux=6b53fb56e2e61a6c6d672817791db03ebe693748
    LICENSE size=1502 time=1458851690.583562292 xattr.security.selinux=6b53fb56e2e61a6c6d672817791db03ebe693748
    README.md size=2366 mode=0644 time=1459369604.0 xattr.security.selinux=6b53fb56e2e61a6c6d672817791db03ebe693748

[...]

See the keyword prefixed with xattr. followed by the extended attribute's namespace and keyword. This setup is consistent for use with Linux extended attributes as well as FreeBSD extended attributes.

Since extended attributes are an unordered hashmap, this approach allows for checking each <namespace>.<key> individually.

The value is the SHA1 digest of the value of the particular extended attribute. Since the values themselves could be raw bytes, this approach both avoids issues with encoding, as well as issues of information leaking. The designation of SHA1 is arbitrary and seen as a general "good enough" assertion of the value.

Usage

To use the Go programming language library, see the docs.

To use the command line tool, first build it, then the following.

Create a manifest

This will also include the sha512 digest of the files.

gomtree -c -K sha512digest -p . > /tmp/mtree.txt

Validate a manifest

gomtree -p . -f /tmp/mtree.txt

See the supported keywords

gomtree -list-keywords
Available keywords:
  rmd160
  ripemd160digest
  type  (default)
  link  (default)
  cksum
  md5
  md5digest
  sha256digest
  sha512
  time  (default)
  gid  (default)
  mode  (default)
  sha1
  sha1digest
  size  (default)
  uid  (default)
  sha256
  sha384
  sha512digest
  nlink  (default)
  uname
  rmd160digest
  sha384digest
  xattr

Building

Either:

go get github.com/vbatts/go-mtree/cmd/gomtree

or

git clone git://github.com/vbatts/go-mtree.git
cd ./go-mtree/cmd/gomtree
go build .