1
0
Fork 1
mirror of https://github.com/vbatts/tar-split.git synced 2024-12-24 14:26:30 +00:00
Commit graph

26 commits

Author SHA1 Message Date
Miloslav Trmač
8d76363085 Avoid a 32 kB file allocation on every bitBucketFilePutter.Put
io.Copy usually allocates a 32kB buffer, and due to the large
number of files processed by tar-split, this shows up in Go profiles
as a very large alloc_space total.

It doesn't seem to actually be a measurable problem in any way,
but we can allocate the buffer only once per tar-split creation,
at no additional cost to existing allocations, so let's do so,
and remove the distraction.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2021-08-21 03:24:39 +02:00
1501fe6002 Merge pull request #22 from tonistiigi/stream-opt
Optimize tar stream generation
2015-12-02 14:09:08 -06:00
11281e8c09 tar/storage: adding Getter Putter benchmark
Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2015-12-01 15:31:48 -05:00
Tonis Tiigi
8b20f9161d Optimize JSON decoding
This allows to avoid extra allocations on `ReadBytes` and
decoding buffers.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2015-11-30 09:52:44 -08:00
7e38cefd4b common: remove in favor of stdlib unicode/utf8 2015-09-25 14:33:24 -04:00
8a361ef0d8 tar/storage: Sprintf is unnecessary
fmt.Sprintf() vs string() for this []byte conversion is too much and
does not provide any further safety.

https://gist.github.com/vbatts/ab17181086aed558dd3a
2015-09-24 09:51:58 -04:00
032efafc29 tar/storage: work with raw (invalid utf8) names
When the entry name is not UTF-8, for example ISO-8859-1, then store the
raw bytes.
To accommodate this, we will have getters and setters for the entry's
name now. Since this most heavily affects the json marshalling, we'll
double check the sanity of the name before storing it in the JSONPacker.
2015-09-23 15:27:33 -04:00
e72b4959f9 Merge pull request #9 from LK4D4/fix_json_tags
storage: Fix syntax of json tags
2015-08-13 15:35:20 -04:00
Alexander Morozov
45399711c2 tar/storage: Replace TeeReader with MultiWriter
It uses slightly less memory and more understandable.
Benchmar results:

benchmark             old ns/op     new ns/op     delta
BenchmarkPutter-4     57272         52375         -8.55%

benchmark             old allocs     new allocs     delta
BenchmarkPutter-4     21             19             -9.52%
benchmark             old bytes     new bytes     delta
BenchmarkPutter-4     19416         13336         -31.31%

Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2015-08-13 11:43:31 -07:00
Alexander Morozov
ea73dc6f6f tar/storage: Benchmark for bufferFileGetPutter.Put
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2015-08-13 11:42:14 -07:00
Alexander Morozov
b1783bc86d storage: Fix syntax of json tags
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2015-08-12 22:41:28 -07:00
Alexander Morozov
e6df23162e Remove redundant TeeReader
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2015-08-12 16:46:04 -07:00
Jonathan Boulle
caf6a872c9 tar/storage: switch to map[string]struct{} for set
Using an empty struct is more idiomatic/efficient for representing a
set-like container.
2015-07-22 15:32:49 -04:00
Jonathan Boulle
002d19f0b0 *: clean up assorted spelling/grammar issues
Various minor fixes noticed on walking through
2015-07-22 15:32:49 -04:00
c2c2dde4cb tar/storage: use filepath instead of path 2015-07-22 10:27:53 -04:00
46840c585a *: golint and docs 2015-03-09 14:11:11 -04:00
d8ebf3c0a7 tar: mv the Getter to tar/storage 2015-03-09 13:20:26 -04:00
e045daf0b0 tar/storage: package docs 2015-03-09 13:13:04 -04:00
d55695834a tar/storage: deciding on crc64 for integrity hash 2015-03-03 14:25:09 -05:00
cfd32ecbc4 tar/storage: do not accept duplicate paths 2015-02-27 16:54:14 -05:00
34a67dfed5 tar/storage: remove cruft Reader 2015-02-24 16:10:49 -05:00
2d70b98f6e tar/storage: show gzip usage in a test 2015-02-24 15:57:14 -05:00
eafa1b9426 tar/storage: fix the test 2015-02-24 15:49:21 -05:00
8ff77b5aa0 tar/storage: add Packer TODO 2015-02-24 15:49:06 -05:00
89bd581749 tar/storage: comments 2015-02-24 15:36:21 -05:00
b1284905d3 storage: working on packing and unpacking 2015-02-24 15:22:20 -05:00