Commit Graph

324 Commits

Author SHA1 Message Date
Stephen Chung 3973da9b81 tar: remove completed TODO's and dead code
minor clean-up of some comments and code that is not used
in tar functionality

Signed-off-by: Stephen Chung <spchung@andrew.cmu.edu>
2016-08-21 15:41:28 -04:00
Vincent Batts bc52946f8e Merge pull request #72 from vbatts/test_cleanup
test: cleanup testdir after testing
2016-08-17 10:33:39 -04:00
Vincent Batts fdf37fcebf test: cleanup testdir after testing
Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2016-08-17 10:25:15 -04:00
Vincent Batts bd52b623df Merge pull request #70 from vbatts/version
Add versioning
2016-08-16 11:42:19 -04:00
Vincent Batts 23a95a1c97 version: bump for a dev cycle
Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2016-08-16 11:37:47 -04:00
Vincent Batts 92b6fc1af3 main: expose the version with a flag
Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2016-08-16 11:37:11 -04:00
Vincent Batts 475c51cbd7 version: adding a version variable
Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2016-08-16 11:34:01 -04:00
Vincent Batts a63446dc6f Merge pull request #67 from stephen679/tar_test_extra_missing
tar_test: don't check for extra/missing when validating relative to "."
2016-08-12 09:04:06 -04:00
Stephen Chung 0e2cf67ac3 tar_test: don't check for extra/missing when validating relative to "."
right now, Check() doesn't check for missing/extra files. So, for the
sake of this test, we don't want to do this check for extra/missing files
when we validate the validation manifest produced from a tar against
the "." root directory (which would be /testdata).

Signed-off-by: Stephen Chung <schung@redhat.com>
2016-08-11 22:23:26 -04:00
Vincent Batts 52985ee3a5 Merge pull request #66 from vbatts/list_used_JSON
gomtree: `-list-used` can output JSON
2016-08-11 15:00:19 -04:00
Vincent Batts 6d8cd9fe4c gomtree: `-list-used` can output JSON
Piggybacking on `-result-format`:

```bash
$ tar c .git/ | gomtree -c -T - > git.mtree
$ gomtree -result-format=json -list-used -f ./git.mtree
{
  "./git.mtree": [
    "type",
    "mode",
    "uid",
    "gid",
    "tar_time",
    "size"
  ]
}
$ tar c .git/ | gomtree -c -T - -K sha512digest > git.mtree
$ gomtree -result-format=json -list-used -f ./git.mtree
{
  "./git.mtree": [
    "type",
    "mode",
    "uid",
    "gid",
    "tar_time",
    "size",
    "sha512digest"
  ]
}
```

Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2016-08-11 14:50:20 -04:00
Vincent Batts 97a2505b8b Merge pull request #65 from vbatts/stdin_tar
gomtree: allow tar from stdin
2016-08-11 14:12:10 -04:00
Vincent Batts 90bdfb6c29 gomtree: allow tar from stdin
Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2016-08-11 14:07:07 -04:00
Vincent Batts 232b80f320 Merge pull request #63 from stephen679/resolve_hardlinks
tar: resolve hardlinks when streaming archive
2016-08-11 13:39:28 -04:00
Stephen Chung 5837d00b07 tar: resolve hardlinks when streaming archive
Fill in the data of the Entry with the data of the
file that a hardlink actually represents.

Signed-off-by: Stephen Chung <schung@redhat.com>
2016-08-11 13:24:44 -04:00
Vincent Batts ea6c6eff1b Merge pull request #57 from stephen679/tar_symlink_size
keywords: return non-zero symlink size
2016-08-11 12:44:03 -04:00
Vincent Batts 7dbe113002 Merge pull request #62 from stephen679/evaluating_symlink
check: functionality for symlinks
2016-08-10 15:22:32 -04:00
Stephen Chung 2facedc401 check: functionality for symlinks
Default behavior (according to upstream mtree) for validating symlinks
is to just validate the link itself, not to follow it.

Signed-off-by: Stephen Chung <schung@redhat.com>
2016-08-10 15:01:07 -04:00
Vincent Batts d6b0881515 Merge pull request #61 from vbatts/clearer_branch
tar: more clear `if` predicate
2016-08-10 14:02:02 -04:00
Vincent Batts 2a99b062a7 tar: more clear `if` predicate
Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2016-08-10 13:46:14 -04:00
Vincent Batts 48e7ab8031 Merge pull request #54 from stephen679/keywords-used-list
hierarchy: provide option to list the used keywords in a spec
2016-08-10 13:35:46 -04:00
Vincent Batts 71abbd944f Merge pull request #55 from stephen679/refactor_tar
tar: refactoring and resolve race condition
2016-08-10 13:31:53 -04:00
Stephen Chung 00a4a2e674 tar: remove "time" keyword before tree creation
Instead of checking each time during keyword evaluation if
the keyword is "time", just remove it from the start and
replace it with "tar_time" (if necessary).

Signed-off-by: Stephen Chung <schung@redhat.com>
2016-08-10 13:17:16 -04:00
Stephen Chung 3d6b74d6f7 tar: populate Entry tree under a common root
Resolves #56. Now, the Entry tree will be populated
under a common root (if necessary), so that a manifest can
be accurately generate from a tar file that has been
created using multiple directories.

Signed-off-by: Stephen Chung <schung@redhat.com>
2016-08-09 12:09:36 -04:00
Stephen Chung 5372b5fc47 hierarchy: provide option to list the used keywords in a spec
To increase a user's control on how they validate a directory or
tar archive with a specification, it is helpful to know which
keywords are actually used in the spec provided. This way, the user
can see what keywords to use or not use with the '-k' or '-K' flags.

Signed-off-by: Stephen Chung <schung@redhat.com>
2016-08-05 10:50:54 -04:00
Stephen Chung 4dc5000106 keywords: return non-zero symlink size
Previously, the symlink size reported by a archive/tar header
was 0. This is incorrect, as the size of a symlink should be
the size of its contents, which is just the path to where
the symlink points to. Thus, the size of this file would just
be len(Linkname).

Signed-off-by: Stephen Chung <schung@redhat.com>
2016-08-02 15:39:54 -04:00
Stephen Chung 6a37331074 tar: flatten the pseudo-tree only after readHeaders() is done
Flattening within the readHeaders() function call was problematic
because readHeaders() is called as a goroutine; thus, as the
main program was calling `tdh.writeTo(os.Stdout)`, readHeaders() was
still in the process of flattening the tree structure. To get around this,
we now call flatten in ts.Hierarchy(), such that only when the main program
is ready to retrieve a "valid" DirectoryHierarchy from the archive, should
we flatten the tree.

Signed-off-by: Stephen Chung <schung@redhat.com>
2016-08-02 11:42:02 -04:00
Stephen Chung 822f319224 tar: minor refactoring of populateTree and flatten
Cleaned up some dead code, and made populateTree not
take in a *tar.Streamer argument, as the ts argument was
only used to set an error. The function now returns
an error (if any). Also made flatten not have to take in
a *tar.Streamer argument as well.

Signed-off-by: Stephen Chung <schung@redhat.com>
2016-07-31 22:26:55 -04:00
Vincent Batts aa6d7c2c6e Merge pull request #53 from vbatts/xattrs_base64
keywords: switch 'xattr' to base64
2016-07-27 18:40:15 -04:00
Vincent Batts 7e2ed0b70b keywords: switch 'xattr' to base64
Rather than an arbitrary election of hashing the xattr value with SHA1
to avoid leaking the value. By going with base64 encoding it allows for
the possibility of restoring a directory with the values stored in the
manifest.

Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2016-07-27 18:38:10 -04:00
Vincent Batts 27459241ae Merge pull request #51 from stephen679/readme-with-tar
readme: update details about go-mtree in readme
2016-07-27 18:34:39 -04:00
Stephen Chung a320d77a6f readme: update details about go-mtree in readme
Include details about tar compatibility, as well as
other aspects that have changed

Signed-off-by: Stephen Chung <schung@redhat.com>
2016-07-27 18:11:34 -04:00
Vincent Batts 94f9a10253 Merge pull request #49 from stephen679/full_digest_output
keywords: synonyms of keywords should produce same output
2016-07-27 15:24:39 -04:00
Stephen Chung 877272303b keywords: synonyms of keywords should produce same output
Currently, if you create a manifest with, say 'sha1', and another
manifest with 'sha1digest', gomtree will produce different output
with regard to the keyword name. I.e, `sha1=[...]` vs. `sha1digest=[...]`.
If we want to use synonyms for keywords, as well as align with upstream mtree
output, specifying 'sha1' and 'sha1digest' should impact gomtree's
in the same way.

Signed-off-by: Stephen Chung <schung@redhat.com>
2016-07-27 10:59:58 -04:00
Vincent Batts 3d7e24b48a Merge pull request #45 from vbatts/bsd_keywords
Bsd keywords
2016-07-26 16:48:49 -04:00
Vincent Batts 919d71c105 main: add -bsd-keywords
To support either producing or checking manifests that are compatible
with upstream FreeBSD mtree(8) keywords, this flag will only operate on
upstream keywords. Completely ignoring non-upstream keywords, though
printing out an INFO to stderr for information purposes.

Example:
```bash
INFO: ignoring "xattrs" as it is not an upstream keyword

/set type=file nlink=1 mode=0664 uid=1000 gid=100
. size=4096 type=dir mode=0755 nlink=2 time=1469556206.235575511

[...]
```

Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2016-07-26 14:31:39 -04:00
Vincent Batts b790afae01 keywords: include list of upstream keywords
Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2016-07-26 14:21:44 -04:00
Vincent Batts 664ad32ff6 Merge pull request #44 from vbatts/xattrs
fix xattr ERANGE
2016-07-26 14:03:16 -04:00
Vincent Batts 2685cd3cc5 xattr: fix the failure on empty list
Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2016-07-26 13:55:25 -04:00
Vincent Batts ed6b293839 debug: add an mtree.Debugf and -debug flag
Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2016-07-26 13:54:45 -04:00
Vincent Batts 39f68f5be2 Merge pull request #41 from stephen679/fix_too_many_files_open
[bug fix] tar: explicitly close files after populateTree
2016-07-26 11:56:57 -04:00
Vincent Batts 2324bb8b19 Merge pull request #42 from stephen679/fix_no_arguments
cmd: gomtree no arguments
2016-07-26 11:52:03 -04:00
Vincent Batts e31036f7e3 Merge pull request #39 from stephen679/no_duplicate_keywords
main: don't dupliate keywords
2016-07-26 11:35:43 -04:00
Stephen Chung 8e3e4c3651 cmd: gomtree no arguments
my tar_stream_tar_time PR accidentally put the functionality
for when gomtree has no arguments inside an unreachable block.

Signed-off-by: Stephen Chung <schung@redhat.com>
2016-07-25 20:35:38 -04:00
Stephen Chung 767d4bbb4b tar: explicitly close files after populateTree
Files don't close properly when `defer`ing inside
a for loop, since the surrounding function is still
iterating in a for loop. To fix this, just close the files
explicitly after `populateTree()` in `readHeaders()`

Signed-off-by: Stephen Chung <schung@redhat.com>
2016-07-25 20:04:55 -04:00
Vincent Batts b0c1606133 Merge pull request #37 from vbatts/no_null_json
check: omitempty rather than `null`
2016-07-25 18:51:38 -04:00
Stephen Chung 843517f136 main: don't dupliate keywords
When adding keywords with -K, we don't want to
duplicate keywords in the keywords slice.

Signed-off-by: Stephen Chung <schung@redhat.com>
2016-07-25 18:29:10 -04:00
Vincent Batts c74c2ed6d7 Merge pull request #38 from stephen679/vis
vis: refactor code to use Vis() and Unvis()
2016-07-25 17:59:48 -04:00
Stephen Chung 773763fb87 vis: refactored code to reflect using vis/unvis for file names
Added some more test cases for `vis`ing and `unvis`ing
strings, and a test case that walks/checks a directory with
filenames that require encoding. Had to change Path() to account
for possible errors Unvis() could return. Refactored Vis()/Unvis() into
go-mtree tar functionality as well.

Signed-off-by: Stephen Chung <schung@redhat.com>
2016-07-25 16:41:06 -04:00
Vincent Batts 898661f983 check: omitempty rather than `null`
When using `-result-format=json` flag, just show populated fields.

Before:
```bash
$ gomtree -result-format=json -p ./bin -f ./bin.mtree
{"failures":[{"path":"gomtree","keyword":"size","expected":"2646101","got":"2930231"}],"Missing":null,"Extra":null}
```

After:
```bash
$ gomtree -result-format=json -p ./bin -f ./bin.mtree
{"failures":[{"path":"gomtree","keyword":"size","expected":"2646101","got":"2930231"}]}
```

Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2016-07-25 15:39:55 -04:00