Commit graph

81 commits

Author SHA1 Message Date
Vincent Batts
ecf82746a8 pkg/archive: test that confirms hardlink ordering
This test was written against
master(abdfb21e3a761efdd70614de42905ff7911c5372) as a failing test, but
works with this patch set.

Signed-off-by: Vincent Batts <vbatts@redhat.com>
2015-02-06 18:14:38 +01:00
Vincent Batts
c059e6d158 pkg/archive: sort files
sort changes found and exported.

Sorting the files before appending them to the tar archive
would mean a dependable ordering for types like hardlinks.

Also, combine sort logic used

Signed-off-by: Vincent Batts <vbatts@redhat.com>
2015-02-06 18:13:57 +01:00
unclejack
4c55c3aa45 pkg/archive: remove tar autodetection log line
Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com>
2015-01-24 08:43:03 +02:00
Doug Davis
f8ec68ef67 Have .dockerignore support Dockerfile/.dockerignore
If .dockerignore mentions either then the client will send them to the
daemon but the daemon will erase them after the Dockerfile has been parsed
to simulate them never being sent in the first place.

an events test kept failing for me so I tried to fix that too

Closes #8330

Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-01-06 10:57:48 -08:00
Josh Hawn
c57c03f841 Refactor to optimize storage driver ApplyDiff()
To avoid an expensive call to archive.ChangesDirs() which walks two directory
trees and compares every entry, archive.ApplyLayer() has been extended to
also return the size of the layer changes.

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
2014-12-17 21:54:23 -08:00
Michael Crosby
6454d26740 Decompress archive before streaming the unpack in a chroot
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

Conflicts:
	pkg/archive/archive.go
	pkg/chrootarchive/archive.go

Conflicts:
	pkg/archive/archive.go
2014-12-11 16:29:13 -05:00
Michael Crosby
eab5117c5c Update chroot apply layer to handle decompression outside chroot
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

Conflicts:
	pkg/archive/diff.go
	pkg/chrootarchive/archive.go

Conflicts:
	pkg/archive/diff.go
	pkg/chrootarchive/diff.go
2014-12-11 16:28:24 -05:00
Andy Goldstein
5e3e0f129e Fix invalid argument error on push
With 32ba6ab from #9261, TempArchive now closes the underlying file and
cleans it up as soon as the file's contents have been read. When pushing
an image, PushImageLayerRegistry attempts to call Close() on the layer,
which is a TempArchive that has already been closed. In this situation,
Close() returns an "invalid argument" error.

Add a Close method to TempArchive that does a no-op if the underlying
file has already been closed.

Signed-off-by: Andy Goldstein <agoldste@redhat.com>
2014-12-03 16:12:24 -05:00
Alexandr Morozov
2ebf95a81e Change path breakout detection logic in archive package
Fixes #9375

Signed-off-by: Alexandr Morozov <lk4d4@docker.com>
2014-11-28 09:44:47 -08:00
Tibor Vass
bdff6d8011 archive: do not call FollowSymlinkInScope in createTarFile
Signed-off-by: Tibor Vass <teabee89@gmail.com>
2014-11-25 01:03:41 +02:00
Tibor Vass
a80a838e6f archive: prevent breakout in ApplyLayer
Signed-off-by: Tibor Vass <teabee89@gmail.com>
2014-11-25 01:03:41 +02:00
Tibor Vass
aa62eca940 archive: prevent breakout in Untar
Signed-off-by: Tibor Vass <teabee89@gmail.com>
2014-11-25 01:03:41 +02:00
Tibor Vass
1752a203af archive: add breakout tests
Signed-off-by: Tibor Vass <teabee89@gmail.com>

Conflicts:
	pkg/archive/archive.go
		fixed conflict which git couldn't fix with the added BreakoutError

Conflicts:
	pkg/archive/archive_test.go
		fixed conflict in imports
2014-11-25 01:03:41 +02:00
unclejack
a590874f19 pkg/archive: fix TempArchive cleanup w/ one read
This fixes the removal of TempArchives which can read with only one
read. Such archives weren't getting removed because EOF wasn't being
triggered.

Docker-DCO-1.1-Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com> (github: unclejack)
2014-11-20 19:34:35 +02:00
Ahmet Alp Balkan
718066ad61 Refactor pkg/archive with a platform-independent stat struct
pkg/archive contains code both invoked from cli (cross platform) and
daemon (linux only) and Unix-specific dependencies break compilation on
Windows. We extracted those stat-related funcs into platform specific
implementations at pkg/system and added unit tests.

Signed-off-by: Ahmet Alp Balkan <ahmetb@microsoft.com>
2014-11-14 18:20:54 -08:00
Ahmet Alp Balkan
515e7481de Extract mknod, umask, lstat to pkg/system
Some parts of pkg/archive is called on both client/daemon code. To get
it compiling on Windows, these funcs are extracted into files with
build tags.

Signed-off-by: Ahmet Alp Balkan <ahmetb@microsoft.com>
2014-11-14 18:20:54 -08:00
Steven Burgess
cabd5528e2 Fix comment to match the arg name
Signed-off-by: Steven Burgess <steven.a.burgess@hotmail.com>
2014-11-11 18:01:08 -05:00
unclejack
513934f6b1 pkg/archive: add interface for Untar
Docker-DCO-1.1-Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com> (github: unclejack)
2014-11-11 18:28:33 +02:00
Tibor Vass
bf0a0e9ca7 Merge pull request #8813 from jlhawn/aufs_exclude_on_tar_layer
Exclude `.wh..wh.*` AUFS metadata on layer export
2014-11-04 11:38:12 -05:00
Vincent Batts
b4b52c87b2 ./pkg/archive: clean up Stat_t assertion
inspired by @tonistiigi comment
(https://github.com/docker/docker/pull/8046/files#r19579960)

Signed-off-by: Vincent Batts <vbatts@redhat.com>
2014-10-30 16:50:33 -04:00
74b38deaa9 archive: cleanup and more information
Signed-off-by: Vincent Batts <vbatts@redhat.com>
Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2014-10-29 14:52:59 -04:00
4a2fb0ab3d archive: example app for diffing directories
By default is a demo of file differences, but can be used to create a
tar of changes between an old and new path.

Signed-off-by: Vincent Batts <vbatts@redhat.com>
Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2014-10-29 14:52:59 -04:00
Vincent Batts
b17f754fff archive: preserve hardlinks in Tar and Untar
* integration test for preserving hardlinks

Signed-off-by: Vincent Batts <vbatts@redhat.com>
Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2014-10-29 14:52:52 -04:00
Josh Hawn
f70214084c Exclude .wh..wh.* AUFS metadata on layer export
In an effort to make layer content 'stable' between import
and export from two different graph drivers, we must resolve
an issue where AUFS produces metadata files in its layers
which other drivers explicitly ignore when importing.

The issue presents itself like this:

    - Generate a layer using AUFS
    - On commit of that container, the new stored layer contains
      AUFS metadata files/dirs. The stored layer content has some
      tarsum value: '1234567'
    - `docker save` that image to a USB drive and `docker load`
      into another docker engine instance which uses another
      graph driver, say 'btrfs'
    - On load, this graph driver explicitly ignores any AUFS metadata
      that it encounters. The stored layer content now has some
      different tarsum value: 'abcdefg'.

The only (apparent) useful aufs metadata to keep are the psuedo link
files located at `/.wh..wh.plink/`. Thes files hold information at the
RW layer about hard linked files between this layer and another layer.
The other graph drivers make sure to copy up these psuedo linked files
but I've tested out a few different situations and it seems that this
is unnecessary (In my test, AUFS already copies up the other hard linked
files to the RW layer).

This changeset adds explicit exclusion of the AUFS metadata files and
directories (NOTE: not the whiteout files!) on commit of a container
using the AUFS storage driver.

Also included is a change to the archive package. It now explicitly
ignores the root directory from being included in the resulting tar archive
for 2 reasons: 1) it's unnecessary. 2) It's another difference between
what other graph drivers produce when exporting a layer to a tar archive.

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
2014-10-28 10:14:05 -07:00
Jessie Frazelle
89f1e164e4 Merge pull request #8770 from LK4D4/logrus_support
Logrus support
2014-10-27 09:05:24 -07:00
Tibor Vass
0e52ddae8f Merge pull request #8641 from vbatts/vbatts-archive_test_and_benchmark
archive: tests and benchmarks for hardlinks
2014-10-27 09:55:19 -04:00
Alexandr Morozov
015f966a1b Mass gofmt
Signed-off-by: Alexandr Morozov <lk4d4@docker.com>
2014-10-24 15:11:48 -07:00
Alexandr Morozov
712a6554ce Use logrus everywhere for logging
Fixed #8761

Signed-off-by: Alexandr Morozov <lk4d4@docker.com>
2014-10-24 15:03:06 -07:00
Vincent Batts
396f1dd125 archive: tests and benchmarks for hardlinks
Adding moar information, so benchmark comparisons can be moar
comparative.

Signed-off-by: Vincent Batts <vbatts@redhat.com>
2014-10-24 08:58:31 -04:00
Brian Goff
b81a28fa8b Make container.Copy support volumes
Fixes #1992

Right now when you `docker cp` a path which is in a volume, the cp
itself works, however you end up getting files that are in the
container's fs rather than the files in the volume (which is not in the
container's fs).
This makes it so when you `docker cp` a path that is in a volume it
follows the volume to the real path on the host.

archive.go has been modified so that when you do `docker cp mydata:/foo
.`, and /foo is the volume, the outputed folder is called "foo" instead
of the volume ID (because we are telling it to tar up
`/var/lib/docker/vfs/dir/<some id>` and not "foo", but the user would be
expecting "foo", not the ID

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2014-10-20 20:23:01 -04:00
Rafe Colton
bd898dfb55 Move archive package into pkg/archive
Now that the archive package does not depend on any docker-specific
packages, only those in pkg and vendor, it can be safely moved into pkg.

Signed-off-by: Rafe Colton <rafael.colton@gmail.com>
2014-09-29 23:23:36 -07:00