mirrors
/
tar-split
mirror of https://github.com/vbatts/tar-split.git
1
0
Fork 1
Code Releases Activity
242 Commits 9 Branches 27 Tags 4.8 MiB
Commit Graph

242 Commits

Author SHA1 Message Date
Vincent Batts cef3176ff9
*: update modules 
Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
 2024-03-22 15:18:03 +00:00
Vincent Batts fe4605ae8b
Merge pull request #69 from testwill/typo 
fix: utility typo
 2023-08-26 07:26:36 -05:00
guoguangwu bd00927367 fix: utility typo 2023-08-26 16:23:45 +08:00
Vincent Batts 5ef0dd8243
Merge pull request #68 from mtrmac/read-0-nil 
Correctly handle Read returning (0, nil)
 2023-07-22 13:09:38 -04:00
Miloslav Trmač cd197d3076 Correctly handle Read returning (0, nil) 
It's not an EOF indication.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
 2023-07-22 02:35:45 +02:00
Vincent Batts 9982e470fa
Merge pull request #67 from vbatts/troubleshoot_65 
don't add a padding entry if it has no bytes
 2023-07-21 09:10:47 -04:00
Vincent Batts b6372414e5
tar/asm: don't add a padding entry if it has no bytes 
Fixes #65

if the read bytes is 0, then don't even create the entry for that
padding.
This sounds like the solution for the issue opened, but I haven't found
a reproducer for this issue yet. :-\

Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
 2023-07-21 09:02:43 -04:00
Vincent Batts cad1f451fd
tar/asm: troubleshooting padding EOF issue 
Reference #65

Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
 2023-07-21 09:02:29 -04:00
Vincent Batts db25994f5b
Merge pull request #66 from testwill/ioutil 
chore: remove refs to deprecated io/ioutil
 2023-07-20 11:41:06 -04:00
guoguangwu 919f9abf38 chore: remove refs to deprecated io/ioutil 
Signed-off-by: guoguangwu <guoguangwu@magic-shield.com>
 2023-07-20 23:00:46 +08:00
Vincent Batts f966b14096
magefile: attempting to recreate `make` file dependencies 
this is sloppy/clunky :-\

Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
 2023-04-27 14:19:40 -04:00
Vincent Batts 724d595c03
magefile: include benchmark in test 
Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
 2023-04-27 14:12:07 -04:00
Vincent Batts 58175ba396
Merge pull request #64 from vbatts/mage 
Housekeeping!
 2023-03-26 14:20:38 -04:00
Vincent Batts e4450847fb
tar/storage: remove TODO's on sailed shipped for changing the encoding 
this function is used widely and it's JSON. And it was not written in
such a way as to have exchangable codec.. per se
So, maybe I'll just kick out the idea of using https://github.com/ugorji/go

Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
 2023-03-26 14:10:16 -04:00
Vincent Batts 075c33cadf
*: mage, drop go1.1{5,6}, module updates, drop vendor 
Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
 2023-03-26 14:01:33 -04:00
Vincent Batts 16d1376be9
README: update the doc URLs 
Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
 2023-03-25 22:53:36 -04:00
Vincent Batts 95fc9b62ed
version: move version into the app main 
ssssss-off-by: Vincent Batts <vbatts@hashbangbash.com>
 2023-03-25 22:49:08 -04:00
Vincent Batts 3cdf99b6d4
Merge pull request #63 from vbatts/go_updates 
*go: update all modules to the latest still compatible with go1.15
 2023-03-25 22:25:25 -04:00
Vincent Batts de24f79d5d
*go: update all modules to the latest still compatible with go1.15 
Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
 2023-03-25 22:20:56 -04:00
Vincent Batts dea500a137
Merge pull request #62 from vbatts/gofmt 
*.go: `gomft -s -w`
 2023-03-25 21:08:59 -04:00
Vincent Batts 32b8c61be1
Merge pull request #61 from vbatts/badge 
README: switch to the github action badges
 2023-03-25 21:08:06 -04:00
Vincent Batts 2b88967591
*.go: `gomft -s -w` 
Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
 2023-03-25 21:05:25 -04:00
Vincent Batts 679a7948ae
README: switch to the github action badges 
Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
 2023-03-25 21:00:42 -04:00
Vincent Batts da105eb683
Merge pull request #60 from vbatts/workflows 
Workflows
 2023-03-25 20:56:13 -04:00
Vincent Batts bc1624cbfc
archive/tar: linting errors 
I intend to not make changes to this `archive/tar` that aren't from
upstream, or are not directly related to the usage by this project...

Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
 2023-03-25 20:45:37 -04:00
Vincent Batts 516158dbfb
*.go: linting project specific code 
the pointer to the pool may be useful, but holding on that until I get
benchmarks of memory use to show the benefit.

Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
 2023-03-25 20:45:23 -04:00
Vincent Batts 19fa6f3d1e
github/workflow: first pass 
May add magefile/mage next, but it seems to require go1.17? So, I'm
holding off for a minute.

Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
 2023-03-25 20:39:31 -04:00
Vincent Batts 3c599ed534
travis: be gone with you! 
Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
 2023-03-25 20:39:06 -04:00
Vincent Batts 70fb294a9b
tar/asm: go vet fixes 
on go1.19.7

Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
 2023-03-25 20:38:36 -04:00
Vincent Batts 80a436fd61
Merge pull request #54 from mtrmac/allocations 
Avoid a 32 kB file allocation on every bitBucketFilePutter.Put
 2021-08-23 06:18:35 -04:00
Miloslav Trmač 8d76363085 Avoid a 32 kB file allocation on every bitBucketFilePutter.Put 
io.Copy usually allocates a 32kB buffer, and due to the large
number of files processed by tar-split, this shows up in Go profiles
as a very large alloc_space total.

It doesn't seem to actually be a measurable problem in any way,
but we can allocate the buffer only once per tar-split creation,
at no additional cost to existing allocations, so let's do so,
and remove the distraction.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
 2021-08-21 03:24:39 +02:00
Vincent Batts 05e7c395fb
Merge pull request #53 from asellappen/master 
Add poweron architecture ppc64le to travis build
 2020-10-28 07:32:45 -04:00
asellappen 6b914ec7a4
Add poweron architecture ppc64le to travis build 
travis build to support
 2020-10-28 11:46:55 +05:30
Vincent Batts 4b50e3822d
Merge pull request #52 from vbatts/vendor 
go mod and vendoring
 2020-10-27 20:19:30 -04:00
Vincent Batts f78d1a4af1
travis: test from go1.10 - go1.15 
Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
 2020-10-27 12:10:43 -04:00
Vincent Batts ec6b1ae20e
vendor the dependencies 
There is a discrepancy of behavior of `github.com/urfave/cli` between
using go1.12 and go1.15, when the dependency is not present as vendored
source. Now this builds fine with go1.12

There are users of tar-split as a package. It is the hope that by adding
this vendored source it does not impact them depending on tar-split
itself.

Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
 2020-10-27 12:10:40 -04:00
Vincent Batts a62c8017af
go mod 
closer coupling to versions of depencies

Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
 2020-10-27 12:10:36 -04:00
Vincent Batts 620714a4c5
Merge pull request #49 from steveb/asm-compress 
Add --compress flag to tar-split asm
 2018-10-22 15:49:18 +01:00
Steve Baker dbab9ab4fb Add --compress flag to tar-split asm 
The Go implementation of gzip is the only known to produce compressed
layers with the expected digest hashes.

This change allows compressed tar layer files to be produced, which is
useful for exporting layers from non-Go tools.
 2018-10-18 12:57:54 +13:00
Vincent Batts 61b49a82bb
Merge pull request #48 from kolyshkin/go111 
rebase to go-1.11's archive/tar
 2018-09-06 08:34:40 -04:00
Kir Kolyshkin 3a386a2750 travis: update golang versions 
Now when golang 1.11 is out, 1.9 and older versions are no longer
supported. More to say, since the archive/tar is from go-1.11, it
uses some features from new Go versions (strings.Builder and sync.Map)
not supported by anything older than Go 1.10.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
 2018-09-05 17:13:21 -07:00
Kir Kolyshkin a555806af9 README.md: update 
The fork of archive/tar is now from go-1.11.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
 2018-09-05 17:13:18 -07:00
Kir Kolyshkin 9a95e02602 archive/tar: port RawHeader() changes 
This is a port of commits adding RawHeader() to go-1.11 archive/tar.

In addition:
* simplify the rawBytes.Write() code in readHeader()
* ignore errors from rawBytes.Write(), as (at least for go-1.11)
  it never returns an error, only panics (if the buffer grew too large)

Also, remove the internal/testenv from tar_tar.go to enable go test.
As working symlink detection is non-trivial on Windows, just skip
the test on that platform.

In addition to `go test`, I did some minimal manual testing, and
it seems this code creates tar-data.json.gz which is identical
to the one made by the old version.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
 2018-09-05 17:12:53 -07:00
Kir Kolyshkin 73fdb78c36 archive/tar: replace with one from go-1.11 
The RawAccounting changes are to be ported on top.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
 2018-09-05 14:04:10 -07:00
Vincent Batts e489928272
Merge pull request #44 from nolith/sirupsen 
rename github.com/Sirupsen/logrus -> github.com/sirupsen/logrus
 2018-02-23 09:46:38 -05:00
Alessio Caiazza a9de541771
rename github.com/Sirupsen/logrus -> github.com/sirupsen/logrus 
Since version 1.0.0 sirupsen changed his github username to downcase.
 2018-02-23 14:42:06 +01:00
Vincent Batts 38ec4ddb06
Merge pull request #42 from cyphar/chunked-padding 
tar: asm: store padding in chunks to avoid memory exhaustion
 2017-11-07 10:39:49 -05:00
Aleksa Sarai 99430a8454
tar: asm: add an excess padding test case 
To ensure we don't have regressions in our padding fix, add a test case
that attempts to crash the test by creating 20GB of random junk padding.

Signed-off-by: Aleksa Sarai <asarai@suse.de>
 2017-11-08 02:35:01 +11:00
Aleksa Sarai 3d9db48dbe
tar: asm: store padding in chunks to avoid memory exhaustion 
Previously, we would read the entire padding in a given archive into
memory in order to store it in the packer. This would cause memory
exhaustion if a malicious archive was crafted with very large amounts of
padding. Since a given SegmentType is reconstructed losslessly, we can
simply chunk up any padding into large segments to avoid this problem.
Use a reasonable default of 1MiB to avoid changing the tar-split.json of
existing archives that are not malformed.

Fixes: CVE-2017-14992
Signed-off-by: Aleksa Sarai <asarai@suse.de>
 2017-11-08 02:34:56 +11:00
Aleksa Sarai b9775006bf
*: move tar_benchmark to cmd/tar-split/ 
This fixes a new go-vet(1) error which has surfaced in Go HEAD.

  $ go vet ./...
  go build github.com/vbatts/tar-split: no non-test Go files in
  /home/travis/gopath/src/github.com/vbatts/tar-split

Signed-off-by: Aleksa Sarai <asarai@suse.de>
 2017-11-08 02:20:30 +11:00