Avoid a 32 kB file allocation on every bitBucketFilePutter.Put

io.Copy usually allocates a 32kB buffer, and due to the large
number of files processed by tar-split, this shows up in Go profiles
as a very large alloc_space total.

It doesn't seem to actually be a measurable problem in any way,
but we can allocate the buffer only once per tar-split creation,
at no additional cost to existing allocations, so let's do so,
and remove the distraction.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This commit is contained in:
Miloslav Trmač 2021-08-21 03:24:39 +02:00
parent 05e7c395fb
commit 8d76363085
1 changed files with 2 additions and 1 deletions

View File

@ -92,11 +92,12 @@ func NewDiscardFilePutter() FilePutter {
}
type bitBucketFilePutter struct {
buffer [32 * 1024]byte // 32 kB is the buffer size currently used by io.Copy, as of August 2021.
}
func (bbfp *bitBucketFilePutter) Put(name string, r io.Reader) (int64, []byte, error) {
c := crc64.New(CRCTable)
i, err := io.Copy(c, r)
i, err := io.CopyBuffer(c, r, bbfp.buffer[:])
return i, c.Sum(nil), err
}