Commit Graph

150 Commits

Author SHA1 Message Date
Vincent Batts d50e5c9283 LICENSE: update LICENSE to BSD 3-clause
Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2015-12-03 15:45:57 -05:00
Vincent Batts 0de4e9db0c Merge pull request #27 from vbatts/bench_asm
tar/asm: basic benchmark on disasm/asm of testdata
2015-12-02 14:09:21 -06:00
Vincent Batts 1501fe6002 Merge pull request #22 from tonistiigi/stream-opt
Optimize tar stream generation
2015-12-02 14:09:08 -06:00
Vincent Batts 19b7e22058 tar/asm: basic benchmark on disasm/asm of testdata
```
PASS
BenchmarkAsm-4         5         238968475 ns/op        66841059 B/op       2449 allocs/op
ok      _/home/vbatts/src/vb/tar-split/tar/asm  2.267s
```

Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2015-12-02 14:36:02 -05:00
Vincent Batts 026e78012b Merge pull request #26 from vbatts/better_discard_in_test
tar/asm: remove unneeded Tee
2015-12-02 12:00:26 -06:00
Vincent Batts 2efe34695a tar/asm: remove unneeded Tee
Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2015-12-02 12:56:52 -05:00
Tonis Tiigi 23b6435e6b Optimize tar stream generation
- New writeTo method allows to avoid creating extra pipe.
- Copy with a pooled buffer instead of allocating new buffer for each file.
- Avoid extra object allocations inside the loop.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2015-12-01 14:08:53 -08:00
Vincent Batts 93666d5824 Merge pull request #25 from vbatts/bench
tar/storage: adding Getter Putter benchmark
2015-12-01 14:37:10 -06:00
Vincent Batts 11281e8c09 tar/storage: adding Getter Putter benchmark
Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2015-12-01 15:31:48 -05:00
Vincent Batts fc1e47e71d Merge pull request #24 from vbatts/drop_go1.2
travis: drop go1.2
2015-12-01 14:31:13 -06:00
Vincent Batts d80c6b3bb1 travis: drop go1.2
seems overly reasonable to support go1.3 and greater. :-)

Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2015-12-01 15:26:30 -05:00
Tonis Tiigi 8b20f9161d Optimize JSON decoding
This allows to avoid extra allocations on `ReadBytes` and
decoding buffers.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2015-11-30 09:52:44 -08:00
Vincent Batts bece0c7009 demo: docker layer checksums
Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2015-10-16 17:05:18 -04:00
Vincent Batts 7ea74e1c31 demo: basic command
Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2015-10-16 16:41:09 -04:00
Vincent Batts c955161e57 Merge pull request #20 from vbatts/unicode-utf8
remove common in favor of stdlib `unicode/utf8`
2015-09-25 14:38:49 -04:00
Vincent Batts 10250c25e0 tar/asm: remove useless test
The iso-8859-1 archive is already tested round trip, and this test did
not do anything really.
2015-09-25 14:35:12 -04:00
Vincent Batts 7e38cefd4b common: remove in favor of stdlib `unicode/utf8` 2015-09-25 14:33:24 -04:00
Vincent Batts 7ef16e6f67 Merge pull request #19 from LK4D4/go_143
Update travis to go1.4.3
2015-09-24 16:02:39 -04:00
Alexander Morozov 27876e49c2 Update travis to go1.4.3
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2015-09-24 12:24:31 -07:00
Vincent Batts 8a361ef0d8 tar/storage: Sprintf is unnecessary
fmt.Sprintf() vs string() for this []byte conversion is too much and
does not provide any further safety.

https://gist.github.com/vbatts/ab17181086aed558dd3a
2015-09-24 09:51:58 -04:00
Vincent Batts 7f56c08c48 Merge pull request #18 from vbatts/iso-8859-1
Iso 8859 1
2015-09-23 15:47:23 -04:00
Vincent Batts cde639172f tar/asm: work with non-utf8 entry names 2015-09-23 15:27:33 -04:00
Vincent Batts 032efafc29 tar/storage: work with raw (invalid utf8) names
When the entry name is not UTF-8, for example ISO-8859-1, then store the
raw bytes.
To accommodate this, we will have getters and setters for the entry's
name now. Since this most heavily affects the json marshalling, we'll
double check the sanity of the name before storing it in the JSONPacker.
2015-09-23 15:27:33 -04:00
Vincent Batts 39d06b9dc4 tar/common: get index of first invalid utf-8 char 2015-09-23 15:27:15 -04:00
Vincent Batts 2865353200 common: add a UTF-8 check helper 2015-09-23 15:27:13 -04:00
Vincent Batts 7384cf1827 Merge pull request #16 from LK4D4/go_15
Add go 1.5.1 to CI
2015-09-11 12:50:40 -04:00
Alexander Morozov 1148e7ee3b Add go 1.5.1 to CI
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2015-09-11 08:49:52 -07:00
Vincent Batts 414a687f83 README: usage 2015-09-03 15:01:25 -04:00
Vincent Batts b4d27b5426 Merge pull request #15 from vbatts/go1.5
Go1.5 updates for archive/tar
2015-08-20 21:25:41 -07:00
Vincent Batts 4d4b53c78b archive/tar: don't treat multiple file system links as a tar hardlink
Do not assume that if stat shows multiple links that we should mark the
file as a hardlink in the tar format.  If the hardlink link was not
referenced, this caused a link to "/".  On an overlay file system, all
files have multiple links.

The caller must keep the inode references and set TypeLink, Size = 0,
and LinkName themselves.

Change-Id: I873b8a235bc8f8fbb271db74ee54232da36ca013
Reviewed-on: https://go-review.googlesource.com/13045
Reviewed-by: Ian Lance Taylor <iant@golang.org>

Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2015-08-21 00:15:22 -04:00
Alex Brainman 3b34dbd368 archive/tar: move round-trip reading into common os file
Fixes #11426

Change-Id: I77368b0e852149ed4533e139cc43887508ac7f78
Reviewed-on: https://go-review.googlesource.com/11662
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>

Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2015-08-21 00:15:22 -04:00
Brad Fitzpatrick 27e18409b9 archive/tar: also skip header roundtrip test on nacl
Update #11426

Change-Id: I7abc4ed2241a7a3af6d57c934786f36de4f97b77
Reviewed-on: https://go-review.googlesource.com/11592
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2015-08-21 00:15:22 -04:00
Brad Fitzpatrick 8eee43d0df archive/tar: disable new failing test on windows and plan9
Update #11426

Change-Id: If406d2efcc81965825a63c76f5448d544ba2a740
Reviewed-on: https://go-review.googlesource.com/11590
Reviewed-by: Austin Clements <austin@google.com>

Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2015-08-21 00:15:22 -04:00
Vincent Batts b48c28014e archive/tar: fix round-trip attributes
The issue was identified while
working with round trip FileInfo of the headers of hardlinks. Also,
additional test cases for hard link handling.
(review carried over from http://golang.org/cl/165860043)

Fixes #9027

Change-Id: I9e3a724c8de72eb1b0fbe0751a7b488894911b76
Reviewed-on: https://go-review.googlesource.com/6790
Reviewed-by: Russ Cox <rsc@golang.org>

Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2015-08-21 00:15:22 -04:00
Michael Gehring 2e5698249c archive/tar: add missing error checks
Check for errors when reading the headers following the pax headers.

Fixes #11169.

Change-Id: Ifec4a949ec8df8b49fa7cb7a67eb826fe2282ad8
Reviewed-on: https://go-review.googlesource.com/11031
Reviewed-by: Russ Cox <rsc@golang.org>

Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2015-08-21 00:15:22 -04:00
Michael Gehring 69de764807 archive/tar: fix slice bounds out of range
Sanity check the pax-header size field before using it.

Fixes #11167.

Change-Id: I9d5d0210c3990e6fb9434c3fe333be0d507d5962
Reviewed-on: https://go-review.googlesource.com/10954
Reviewed-by: David Symonds <dsymonds@golang.org>

Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2015-08-21 00:15:22 -04:00
Håvard Haugen 55dceefe42 archive/tar: terminate when reading malformed sparse files
Fixes #10968.

Change-Id: I027bc571a71629ac49c2a0ff101b2950af6e7531
Reviewed-on: https://go-review.googlesource.com/10482
Reviewed-by: David Symonds <dsymonds@golang.org>
Run-TryBot: David Symonds <dsymonds@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2015-08-21 00:15:22 -04:00
Håvard Haugen 576b273762 archive/tar: don't panic on negative file size
Fixes #10959.
Fixes #10960.

Change-Id: I9a81a0e2b8275338d0d1c3f7f7265e0fd91f3de2
Reviewed-on: https://go-review.googlesource.com/10402
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Symonds <dsymonds@golang.org>

Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2015-08-21 00:15:22 -04:00
David du Colombier 6e38573de2 archive/tar: fix error message
Write should return ErrWriteAfterClose instead
of ErrWriteTooLong when called after Close.

Change-Id: If5ec4ef924e4c56489e0d426976f7e5fad79be9b
Reviewed-on: https://go-review.googlesource.com/9259
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2015-08-21 00:15:22 -04:00
Vincent Batts bf82db1f0d README: updates 2015-08-18 14:54:32 -04:00
Vincent Batts ea4426eee9 Merge pull request #13 from vbatts/travis-go-versions
travis: adding older and newer golang
2015-08-14 10:18:38 -04:00
Vincent Batts 3a88af2866 travis: adding older and newer golang 2015-08-14 10:15:26 -04:00
Vincent Batts 4f81319c22 *: adding some version magic 2015-08-14 10:02:46 -04:00
Vincent Batts c76e42010e tar/asm: additional GNU LongLink testcase
Adding a minimal test case for GNU @LongLink.
Tested that it fails on v0.9.5, but now passes on v0.9.6 and master.
2015-08-14 07:55:18 -04:00
Vincent Batts 44d93178df Merge pull request #11 from LK4D4/add_vet
Add vet check to travis
2015-08-13 15:37:30 -04:00
Vincent Batts 8f81a50860 Merge pull request #10 from LK4D4/fix_pipe_close
asm: Remove unreachable code
2015-08-13 15:36:42 -04:00
Vincent Batts e72b4959f9 Merge pull request #9 from LK4D4/fix_json_tags
storage: Fix syntax of json tags
2015-08-13 15:35:20 -04:00
Vincent Batts 4d66163297 archive/tar: a []byte copy needed for GNU LongLink 2015-08-13 15:32:17 -04:00
Vincent Batts 9b9df04f1f Merge pull request #12 from LK4D4/multi_for_tee
tar/storage: Replace TeeReader with MultiWriter
2015-08-13 15:12:22 -04:00
Alexander Morozov 45399711c2 tar/storage: Replace TeeReader with MultiWriter
It uses slightly less memory and more understandable.
Benchmar results:

benchmark             old ns/op     new ns/op     delta
BenchmarkPutter-4     57272         52375         -8.55%

benchmark             old allocs     new allocs     delta
BenchmarkPutter-4     21             19             -9.52%
benchmark             old bytes     new bytes     delta
BenchmarkPutter-4     19416         13336         -31.31%

Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2015-08-13 11:43:31 -07:00