File systems verification utility and library, in likeness of mtree(8) https://github.com/vbatts/go-mtree

Find a file

Stephen Chung 9d870829cb tar: create and validate a manifest from a tar stream This commit contains added features to go-mtree that allows user to create an mtree spec with '-T' option when specifying a tar archive. Users can also validate an mtree spec against a tar archive with an mtree spec. Also for the test archive, there is a mixture of files, and folders (empty & non-empty), and symlinks (broken & unbroken). Signed-off-by: Stephen Chung <schung@redhat.com>		2016-07-22 14:07:17 -04:00
cmd/gomtree	tar: create and validate a manifest from a tar stream	2016-07-22 14:07:17 -04:00
testdata	tar: create and validate a manifest from a tar stream	2016-07-22 14:07:17 -04:00
xattr	travis: add travis check	2016-04-06 13:57:11 -04:00
.travis.yml	travis: go1.5 and go1.6 have vet builtin	2016-04-13 15:34:50 -04:00
check.go	tar: create and validate a manifest from a tar stream	2016-07-22 14:07:17 -04:00
check_test.go	Merge pull request #22 from stephen679/set_aliasing	2016-07-20 07:40:19 +09:00
cksum.go	cksum: comment	2016-03-24 16:35:09 -04:00
cksum_test.go	cksum: test is fine. commenting.	2016-04-05 16:47:36 -04:00
creator.go	*: refactoring to support streams	2016-07-20 17:41:17 -04:00
entry.go	tar: create and validate a manifest from a tar stream	2016-07-22 14:07:17 -04:00
hierarchy.go	*: refactoring to support streams	2016-07-20 17:41:17 -04:00
keywords.go	keywords: return "link" keyword along with its value	2016-07-20 17:41:17 -04:00
keywords_linux.go	*: refactoring to support streams	2016-07-20 17:41:17 -04:00
keywords_unsupported.go	*: refactoring to support streams	2016-07-20 17:41:17 -04:00
LICENSE	LICENSE: adding a license	2016-03-24 16:34:58 -04:00
mtree_test.go	hierarchy: testing works	2016-04-05 17:16:44 -04:00
parse.go	tar: create and validate a manifest from a tar stream	2016-07-22 14:07:17 -04:00
README.md	main: change `-l` to `-list-keywords`	2016-07-20 11:39:32 -04:00
tar.go	tar: create and validate a manifest from a tar stream	2016-07-22 14:07:17 -04:00
tar_test.go	tar: create and validate a manifest from a tar stream	2016-07-22 14:07:17 -04:00
walk.go	*: refactoring to support streams	2016-07-20 17:41:17 -04:00
walk_test.go	walk: test passes and validates parse	2016-04-06 12:45:08 -04:00

README.md

go-mtree

mtree is a filesystem hierarchy validation tooling and format. This is a library and simple cli tool for mtree(8) support.

While the traditional mtree cli utility is primarily on BSDs (FreeBSD, openBSD, etc), but even broader support for the mtree specification format is provided with libarchive (libarchive-formats(5)).

There is also an mtree port for Linux though it is not widely packaged for Linux distributions.

Format

The format of hierarchy specification is consistent with the # mtree v2.0 format. Both the BSD mtree and libarchive ought to be interoperable with it with only one definite caveat. On Linux, extended attributes (xattr) on files are often a critical aspect of the file, holding ACLs, capabilities, etc. While FreeBSD filesystem do support extattr, this feature has not made its way into their mtree.

This implementation of mtree supports an additional "keyword" of xattr. If you include this keyword, then the FreeBSD mtree will fail as it is an unknown keyword to that implementation.

Typical form

With the standard keywords, plus say sha256digest, the hierarchy specification looks like:

# .
/set type=file nlink=1 mode=0664 uid=1000 gid=100
. size=4096 type=dir mode=0755 nlink=6 time=1459370393.273231538
    LICENSE size=1502 mode=0644 time=1458851690.0 sha256digest=ef4e53d83096be56dc38dbf9bc8ba9e3068bec1ec37c179033d1e8f99a1c2a95
    README.md size=2820 mode=0644 time=1459370256.316148361 sha256digest=d9b955134d99f84b17c0a711ce507515cc93cd7080a9dcd50400e3d993d876ac

[...]

See the directory presently in, and the files present. Along with each path, is provided the keywords and the unique values for each path. Any common keyword and values are established in the /set command.

Extended attributes form

# .
/set type=file nlink=1 mode=0664 uid=1000 gid=1000
. size=4096 type=dir mode=0775 nlink=6 time=1459370191.11179595 xattr.security.selinux=6b53fb56e2e61a6c6d672817791db03ebe693748
    LICENSE size=1502 time=1458851690.583562292 xattr.security.selinux=6b53fb56e2e61a6c6d672817791db03ebe693748
    README.md size=2366 mode=0644 time=1459369604.0 xattr.security.selinux=6b53fb56e2e61a6c6d672817791db03ebe693748

[...]

See the keyword prefixed with xattr. followed by the extended attribute's namespace and keyword. This setup is consistent for use with Linux extended attributes as well as FreeBSD extended attributes.

Since extended attributes are an unordered hashmap, this approach allows for checking each <namespace>.<key> individually.

The value is the SHA1 digest of the value of the particular extended attribute. Since the values themselves could be raw bytes, this approach both avoids issues with encoding, as well as issues of information leaking. The designation of SHA1 is arbitrary and seen as a general "good enough" assertion of the value.

Usage

To use the Go programming language library, see the docs.

To use the command line tool, first build it, then the following.

Create a manifest

This will also include the sha512 digest of the files.

gomtree -c -K sha512digest -p . > /tmp/mtree.txt

Validate a manifest

gomtree -p . -f /tmp/mtree.txt

See the supported keywords

gomtree -list-keywords
Available keywords:
  rmd160
  ripemd160digest
  type  (default)
  link  (default)
  cksum
  md5
  md5digest
  sha256digest
  sha512
  time  (default)
  gid  (default)
  mode  (default)
  sha1
  sha1digest
  size  (default)
  uid  (default)
  sha256
  sha384
  sha512digest
  nlink  (default)
  uname
  rmd160digest
  sha384digest
  xattr

Building

Either:

go get github.com/vbatts/go-mtree/cmd/gomtree

git clone git://github.com/vbatts/go-mtree.git
cd ./go-mtree/cmd/gomtree
go build .