Commit graph

1085 commits

Author SHA1 Message Date
unclejack
14803cda50 pkg/graphdb: use transactions for transactions
Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com>
2014-12-19 13:57:21 +02:00
Michael Crosby
f4ca80df36 Merge pull request #9233 from inatatsu/fix-pkg-units-size-for-gccgo
Fix to avoid a compile error due to float to int truncation with GCCGO
2014-12-18 18:33:01 -08:00
unclejack
b8e50e81c1 pkg/tarsum: delete the logging code
Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com>
2014-12-18 23:38:26 +02:00
Josh Hawn
c57c03f841 Refactor to optimize storage driver ApplyDiff()
To avoid an expensive call to archive.ChangesDirs() which walks two directory
trees and compares every entry, archive.ApplyLayer() has been extended to
also return the size of the layer changes.

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
2014-12-17 21:54:23 -08:00
Srini Brahmaroutu
52679b2b7f Fix to avoid a compile error due to float to int truncation with GCCGO
Signed-off-by: Srini Brahmaroutu <srbrahma@us.ibm.com>
2014-12-17 22:44:55 +00:00
Álex González
d27d1dfcbe Log when truncindex.Get returns >1 container
When the user is not using the full has to retrieve a container it's
possible that we find conflicts with the ids of other containers.

At the moment it's just failing saying that it can not find a container,
but it doesn't say why. Adding a small log saying that duplicates where
found is going to help the user.

Closes #8098

Signed-off-by: Alex Gonzalez <agonzalezro@gmail.com>
2014-12-13 17:43:05 +00:00
Alexander Morozov
4d20c0cb80 Fix vet errors about formatting directives
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2014-12-12 10:58:56 -08:00
Alexander Morozov
6b772e6532 Fix vet errors about unkeyed fields
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2014-12-12 10:44:59 -08:00
Jessica Frazelle
e4b13e4efb Merge branch master into bump_v1.4.0
Docker-DCO-1.1-Signed-off-by: Jessica Frazelle <jess@docker.com> (github: jfrazelle)
2014-12-11 15:00:03 -08:00
Michael Crosby
6454d26740 Decompress archive before streaming the unpack in a chroot
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

Conflicts:
	pkg/archive/archive.go
	pkg/chrootarchive/archive.go

Conflicts:
	pkg/archive/archive.go
2014-12-11 16:29:13 -05:00
Michael Crosby
eab5117c5c Update chroot apply layer to handle decompression outside chroot
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

Conflicts:
	pkg/archive/diff.go
	pkg/chrootarchive/archive.go

Conflicts:
	pkg/archive/diff.go
	pkg/chrootarchive/diff.go
2014-12-11 16:28:24 -05:00
Tibor Vass
2a81dcd671 symlink: add more tests
Signed-off-by: Tibor Vass <teabee89@gmail.com>
2014-12-11 16:26:01 -05:00
Tibor Vass
21026f47e6 symlink: cleanup names and break big test into multiple smaller ones
Signed-off-by: Tibor Vass <teabee89@gmail.com>
2014-12-11 16:26:01 -05:00
Tibor Vass
9bd8acc9b7 Refactor of symlink tests to remove testdata dir
Signed-off-by: Tibor Vass <teabee89@gmail.com>
2014-12-11 16:26:01 -05:00
Tianon Gravi
96678a759b Simplify FollowSymlinkInScope based on Go 1.3.3's EvalSymlinks
Signed-off-by: Andrew Page <admwiggin@gmail.com>
2014-12-11 16:26:01 -05:00
ab77e1f2dd Merge pull request #9258 from rhvgoyal/transaction-id-improvements
devmapper fix usage of pool transaction id
2014-12-11 12:58:18 -05:00
Michael Crosby
9888ff1fc6 Merge pull request #9099 from vbatts/vbatts-tarsum_spec
pkg/tarsum: specification on TarSum checksum algorithm
2014-12-10 10:24:48 -08:00
Alexandr Morozov
5ce91364b3 Change path breakout detection logic in archive package
Fixes #9375

Signed-off-by: Alexandr Morozov <lk4d4@docker.com>

Conflicts:
	integration-cli/docker_cli_cp_test.go
		removed extra test
2014-12-09 23:16:03 +02:00
Michael Crosby
cd74be68b8 Flush stdin from within chroot archive
This makes sure that we don't buffer in memory and that we also flush
stdin from diff as well as untar.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

Conflicts:
	pkg/chrootarchive/diff.go
2014-12-09 14:56:17 -05:00
Lewis Marshall
a8124280d2 Fix chroot untar for zero padded archive from slow reader
Signed-off-by: Lewis Marshall <lewis@lmars.net>
2014-12-09 14:56:16 -05:00
Michael Crosby
1036313f03 Decompress archive before streaming the unpack in a chroot
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

Conflicts:
	pkg/archive/archive.go
	pkg/chrootarchive/archive.go
2014-12-09 14:56:16 -05:00
Michael Crosby
79b559bc05 Update chroot apply layer to handle decompression outside chroot
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

Conflicts:
	pkg/archive/diff.go
	pkg/chrootarchive/archive.go
2014-12-09 14:56:16 -05:00
Tibor Vass
abc58c3bfd symlink: add more tests
Signed-off-by: Tibor Vass <teabee89@gmail.com>
2014-12-09 14:56:16 -05:00
Tibor Vass
b96f5922bb symlink: cleanup names and break big test into multiple smaller ones
Signed-off-by: Tibor Vass <teabee89@gmail.com>
2014-12-09 14:56:16 -05:00
Tibor Vass
7235750de8 Refactor of symlink tests to remove testdata dir
Signed-off-by: Tibor Vass <teabee89@gmail.com>
2014-12-09 14:56:16 -05:00
Tianon Gravi
ae3c4f8582 Simplify FollowSymlinkInScope based on Go 1.3.3's EvalSymlinks
Signed-off-by: Andrew Page <admwiggin@gmail.com>
2014-12-09 14:56:16 -05:00
Michael Crosby
6bd7943697 Flush stdin from within chroot archive
This makes sure that we don't buffer in memory and that we also flush
stdin from diff as well as untar.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2014-12-08 15:41:07 -08:00
Tibor Vass
aec3164411 Merge pull request #9539 from flynn/chroot-untar-empty-archive-slow
Fix chroot untar for zero padded archive from slow reader
2014-12-08 17:59:21 -05:00
Erik Dubbelboer
ad11e1392c Removed race condition
If two interrupts were fired really quickly interruptCount could have been incremented twice before the LoadUint32 making cleanup not being called at all.

Signed-off-by: Erik Dubbelboer <erik@dubbelboer.com>
2014-12-06 22:52:19 +08:00
Lewis Marshall
fec996b35d Fix chroot untar for zero padded archive from slow reader
Signed-off-by: Lewis Marshall <lewis@lmars.net>
2014-12-06 02:44:31 +00:00
Alexander Morozov
04fb504442 Merge pull request #9479 from jlhawn/tarsum_test_fix
Correct TarSum benchmarks: 9kTar and 9kTarGzip
2014-12-05 14:47:54 -08:00
Josh Hawn
d98b6f38ba Correctly close generated benchmark archives
Another update to TarSum tests, this patch fixes an issue where
the benchmarks were generating archives incorrectly by not closing
the tarWriter.

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
2014-12-03 22:26:06 -08:00
Jessica Frazelle
267e518231 Fix output format where no variable specified in mount pkg
Docker-DCO-1.1-Signed-off-by: Jessica Frazelle <jess@docker.com> (github: jfrazelle)
2014-12-03 19:02:51 -08:00
Jessie Frazelle
7a025f3fdf Merge pull request #9451 from erikh/fix-sqlite
graphdb: initialize the database semi-idempotently on every connection, not just new installs.
2014-12-03 17:17:20 -08:00
Andy Goldstein
5e3e0f129e Fix invalid argument error on push
With 32ba6ab from #9261, TempArchive now closes the underlying file and
cleans it up as soon as the file's contents have been read. When pushing
an image, PushImageLayerRegistry attempts to call Close() on the layer,
which is a TempArchive that has already been closed. In this situation,
Close() returns an "invalid argument" error.

Add a Close method to TempArchive that does a no-op if the underlying
file has already been closed.

Signed-off-by: Andy Goldstein <agoldste@redhat.com>
2014-12-03 16:12:24 -05:00
Josh Hawn
985d3bd404 Correct TarSum benchmarks: 9kTar and 9kTarGzip
These two cases did not actually read the same content with each iteration
of the benchmark. After the first read, the buffer was consumed. This patch
corrects this by using a bytes.Reader and seeking to the beginning of the
buffer at the beginning of each iteration.

Unfortunately, this benchmark was not actually as fast as we believed. But
the new results do bring its results closer to those of the other benchmarks.

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
2014-12-03 10:44:05 -08:00
Vivek Goyal
12524d5657 devmapper: Use device id as specified by caller
Currently devicemapper CreateDevice and CreateSnapDevice keep on retrying
device creation till a suitable device id is found. 

With new transaction mechanism we need to store device id in transaction
before it has been created.

So change the logic in such a way that caller decides the devices Id to
use. If that device Id is not available, caller bumps up the device Id
and retries.

That way caller can update transaciton too when it tries a new Id. Transaction
related patches will come later in the series.
  
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2014-12-03 13:06:43 -05:00
Josh Hawn
2b403ab360 Fix TarSum iteration test
I noticed that 3 of the tarsum test cases had expected a tarsum with
a sha256 hash of
  e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
As I've been working with sha256 quite a bit lately, it struck me that
this is the initial digest value for sha256, which means that no data
was processed. However, these tests *do* process data. It turns out that
there was a bug in the test handling code which did not wait for tarsum
to end completely. This patch corrects these test cases.

I'm unaware of anywhere else in the code base where this would be an issue,
though we definitily need to look out in the future to ensure we are
completing tarsum reads (waiting for EOF).

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
2014-12-02 15:28:13 -08:00
Erik Hollensbe
12efe03ef0 graphdb: initialize the database semi-idempotently on every connection.
Docker-DCO-1.1-Signed-off-by: Erik Hollensbe <github@hollensbe.org> (github: erikh)
2014-12-02 03:02:25 -08:00
Michael Crosby
8c02df4c6c Merge pull request #9407 from hqhq/hq_fix_comments
flag: fix the comments
2014-12-01 11:27:06 -08:00
Qiang Huang
16130e775f flag: fix the comments
Signed-off-by: Qiang Huang <h.huangqiang@huawei.com>
2014-11-30 01:58:16 +08:00
Alexandr Morozov
2ebf95a81e Change path breakout detection logic in archive package
Fixes #9375

Signed-off-by: Alexandr Morozov <lk4d4@docker.com>
2014-11-28 09:44:47 -08:00
4e3563a9b6 Merge pull request #9006 from snitm/thin-pool-improvements
Thin pool improvements
2014-11-26 14:44:09 -05:00
1867fe24fe Merge pull request #9279 from yoheiueda/termios-via-cgo
Use termios via CGO
2014-11-25 20:04:26 -05:00
Brian Goff
32fb3913b5 Cleanup ParseHost
Current implementation is comingling things that ought not be together.
There are _some_ similarities between parsing for the different proto
types, but they are more different than alike, making the code extremely
difficult to reason about.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2014-11-25 18:55:40 -05:00
unclejack
f47ea48cfe pkg/tarsum: add maintainers & add missing s
Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com>
2014-11-26 00:44:16 +02:00
Yohei Ueda
c1532556b2 Use termios via CGO only on Linux
Signed-off-by: Yohei Ueda <yohei@jp.ibm.com>
2014-11-25 20:49:01 +09:00
Tatsushi Inagaki
e5b1c19379 Fix to avoid a compilation error of size_test.go with GCCGO due to float to int truncation
Signed-off-by: Tatsushi Inagaki <e29253@jp.ibm.com>
2014-11-25 18:12:22 +09:00
Tatsushi Inagaki
7d92b1230e Revert "Fix to avoid a compile error due to float to int truncation with GCCGO"
This reverts commit 967a42f116d23051c862a3b4983925de2016f83c.

Signed-off-by: Tatsushi Inagaki <e29253@jp.ibm.com>

Roll back the change to fix the parameter of HumanSize from int64 to float64
2014-11-25 16:48:09 +09:00
bd9c676bb7 tarsum: updates for jamtur01 comments
Signed-off-by: Vincent Batts <vbatts@redhat.com>
2014-11-24 23:18:48 -05:00