mirrors
/
tar-split
mirror of https://github.com/vbatts/tar-split.git
1
0
Fork 1
Code Releases Activity
242 Commits 9 Branches 27 Tags 4.8 MiB
Commit Graph

30 Commits

Author SHA1 Message Date
guoguangwu 919f9abf38 chore: remove refs to deprecated io/ioutil 
Signed-off-by: guoguangwu <guoguangwu@magic-shield.com>
 2023-07-20 23:00:46 +08:00
Vincent Batts e4450847fb
tar/storage: remove TODO's on sailed shipped for changing the encoding 
this function is used widely and it's JSON. And it was not written in
such a way as to have exchangable codec.. per se
So, maybe I'll just kick out the idea of using https://github.com/ugorji/go

Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
 2023-03-26 14:10:16 -04:00
Vincent Batts 2b88967591
*.go: `gomft -s -w` 
Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
 2023-03-25 21:05:25 -04:00
Vincent Batts 516158dbfb
*.go: linting project specific code 
the pointer to the pool may be useful, but holding on that until I get
benchmarks of memory use to show the benefit.

Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
 2023-03-25 20:45:23 -04:00
Miloslav Trmač 8d76363085 Avoid a 32 kB file allocation on every bitBucketFilePutter.Put 
io.Copy usually allocates a 32kB buffer, and due to the large
number of files processed by tar-split, this shows up in Go profiles
as a very large alloc_space total.

It doesn't seem to actually be a measurable problem in any way,
but we can allocate the buffer only once per tar-split creation,
at no additional cost to existing allocations, so let's do so,
and remove the distraction.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
 2021-08-21 03:24:39 +02:00
Vincent Batts 1501fe6002 Merge pull request #22 from tonistiigi/stream-opt 
Optimize tar stream generation
 2015-12-02 14:09:08 -06:00
Vincent Batts 11281e8c09 tar/storage: adding Getter Putter benchmark 
Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
 2015-12-01 15:31:48 -05:00
Tonis Tiigi 8b20f9161d Optimize JSON decoding 
This allows to avoid extra allocations on `ReadBytes` and
decoding buffers.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2015-11-30 09:52:44 -08:00
Vincent Batts 7e38cefd4b common: remove in favor of stdlib `unicode/utf8` 2015-09-25 14:33:24 -04:00
Vincent Batts 8a361ef0d8 tar/storage: Sprintf is unnecessary 
fmt.Sprintf() vs string() for this []byte conversion is too much and
does not provide any further safety.

https://gist.github.com/vbatts/ab17181086aed558dd3a
 2015-09-24 09:51:58 -04:00
Vincent Batts 032efafc29 tar/storage: work with raw (invalid utf8) names 
When the entry name is not UTF-8, for example ISO-8859-1, then store the
raw bytes.
To accommodate this, we will have getters and setters for the entry's
name now. Since this most heavily affects the json marshalling, we'll
double check the sanity of the name before storing it in the JSONPacker.
 2015-09-23 15:27:33 -04:00
Vincent Batts e72b4959f9 Merge pull request #9 from LK4D4/fix_json_tags 
storage: Fix syntax of json tags
 2015-08-13 15:35:20 -04:00
Alexander Morozov 45399711c2 tar/storage: Replace TeeReader with MultiWriter 
It uses slightly less memory and more understandable.
Benchmar results:

benchmark             old ns/op     new ns/op     delta
BenchmarkPutter-4     57272         52375         -8.55%

benchmark             old allocs     new allocs     delta
BenchmarkPutter-4     21             19             -9.52%
benchmark             old bytes     new bytes     delta
BenchmarkPutter-4     19416         13336         -31.31%

Signed-off-by: Alexander Morozov <lk4d4@docker.com>
 2015-08-13 11:43:31 -07:00
Alexander Morozov ea73dc6f6f tar/storage: Benchmark for bufferFileGetPutter.Put 
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
 2015-08-13 11:42:14 -07:00
Alexander Morozov b1783bc86d storage: Fix syntax of json tags 
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
 2015-08-12 22:41:28 -07:00
Alexander Morozov e6df23162e Remove redundant TeeReader 
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
 2015-08-12 16:46:04 -07:00
Jonathan Boulle caf6a872c9 tar/storage: switch to map[string]struct{} for set 
Using an empty struct is more idiomatic/efficient for representing a
set-like container.
 2015-07-22 15:32:49 -04:00
Jonathan Boulle 002d19f0b0 *: clean up assorted spelling/grammar issues 
Various minor fixes noticed on walking through
 2015-07-22 15:32:49 -04:00
Vincent Batts c2c2dde4cb tar/storage: use `filepath` instead of `path` 2015-07-22 10:27:53 -04:00
Vincent Batts 46840c585a *: golint and docs 2015-03-09 14:11:11 -04:00
Vincent Batts d8ebf3c0a7 tar: mv the Getter to tar/storage 2015-03-09 13:20:26 -04:00
Vincent Batts e045daf0b0 tar/storage: package docs 2015-03-09 13:13:04 -04:00
Vincent Batts d55695834a tar/storage: deciding on crc64 for integrity hash 2015-03-03 14:25:09 -05:00
Vincent Batts cfd32ecbc4 tar/storage: do not accept duplicate paths 2015-02-27 16:54:14 -05:00
Vincent Batts 34a67dfed5 tar/storage: remove cruft Reader 2015-02-24 16:10:49 -05:00
Vincent Batts 2d70b98f6e tar/storage: show gzip usage in a test 2015-02-24 15:57:14 -05:00
Vincent Batts eafa1b9426 tar/storage: fix the test 2015-02-24 15:49:21 -05:00
Vincent Batts 8ff77b5aa0 tar/storage: add Packer TODO 2015-02-24 15:49:06 -05:00
Vincent Batts 89bd581749 tar/storage: comments 2015-02-24 15:36:21 -05:00
Vincent Batts b1284905d3 storage: working on packing and unpacking 2015-02-24 15:22:20 -05:00