Commit graph

1591 commits

Author SHA1 Message Date
Phil Estes
e118299052 Add user namespace (mapping) support to the Docker engine
Adds support for the daemon to handle user namespace maps as a
per-daemon setting.

Support for handling uid/gid mapping is added to the builder,
archive/unarchive packages and functions, all graphdrivers (except
Windows), and the test suite is updated to handle user namespace daemon
rootgraph changes.

Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
2015-10-09 17:47:37 -04:00
Phil Estes
5e8012caca Add utility/support package for user namespace support
The `pkg/idtools` package supports the creation of user(s) for
retrieving /etc/sub{u,g}id ranges and creation of the UID/GID mappings
provided to clone() to add support for user namespaces in Docker.

Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
2015-10-09 17:44:47 -04:00
Jess Frazelle
dade93d00f Merge pull request #16644 from dhiltgen/discovery_tls
Add TLS support for discovery backend
2015-10-08 14:21:06 -07:00
Jess Frazelle
07e78dc3cc Merge pull request #16803 from tiborvass/pkg-broadcaster
Move types from progressreader and broadcastwriter to broadcaster
2015-10-08 13:51:08 -07:00
Jess Frazelle
790b9791fd Merge pull request #13777 from cpuguy83/graphdriver_extpoints
Create extpoint for graphdrivers
2015-10-08 13:46:42 -07:00
Vincent Demeester
54a199952b Merge pull request #16781 from brahmaroutu/pkg_integration_timeout
timeouts causing tests to fail differently. GCCGO CI.
2015-10-08 22:02:22 +02:00
Jess Frazelle
880844e210 Merge pull request #16581 from fgimenez/11584-stdcopy-test-coverage
Added test coverage to StdCopy closes #11584
2015-10-08 12:53:25 -07:00
Srini Brahmaroutu
0a9741c6cb Timeouts are causing tests to fail differently. GCCGO CI.
Signed-off-by: Srini Brahmaroutu <srbrahma@us.ibm.com>
2015-10-08 15:14:56 +00:00
Daniel Hiltgen
125510e009 Add TLS support for discovery backend
This leverages recent additions to libkv enabling client
authentication via TLS so the discovery back-end can be locked
down with mutual TLS.  Example usage:

    docker daemon [other args] \
        --cluster-advertise 192.168.122.168:2376 \
        --cluster-store etcd://192.168.122.168:2379 \
        --cluster-store-opt kv.cacertfile=/path/to/ca.pem \
        --cluster-store-opt kv.certfile=/path/to/cert.pem \
        --cluster-store-opt kv.keyfile=/path/to/key.pem

Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
2015-10-07 16:01:00 -07:00
Alexander Morozov
2861b2e52f Merge pull request #16818 from runcom/rmi-perf
graph: add parent img refcount for faster rmi
2015-10-07 10:45:49 -07:00
Antonio Murdaca
8f009df876 graph: add parent img refcount for faster rmi
also fix a typo in pkg/truncindex package comment

Signed-off-by: Antonio Murdaca <amurdaca@redhat.com>
2015-10-07 19:09:44 +02:00
Alexander Morozov
d3aed6260a Merge pull request #16381 from rhvgoyal/deferred_deletion
devicemapper: Implement deferred deletion capability
2015-10-07 09:27:41 -07:00
Tibor Vass
2d6672b0e1 Move types from progressreader and broadcastwriter to broadcaster
progressreader.Broadcaster becomes broadcaster.Buffered and
broadcastwriter.Writer becomes broadcaster.Unbuffered.

The package broadcastwriter is thus renamed to broadcaster.

Signed-off-by: Tibor Vass <tibor@docker.com>
2015-10-06 22:20:07 -04:00
Brian Goff
7ddde13c00 Merge pull request #16147 from tiborvass/refactor-builder
Refactor builder with new Go interfaces
2015-10-06 20:36:07 -04:00
Tibor Vass
e5a4257fa7 Abstract builder and implement server-side dockerfile builder
This patch creates interfaces in builder/ for building Docker images.
It is a first step in a series of patches to remove the daemon
dependency on builder and later allow a client-side Dockerfile builder
as well as potential builder plugins.

It is needed because we cannot remove the /build API endpoint, so we
need to keep the server-side Dockerfile builder, but we also want to
reuse the same Dockerfile parser and evaluator for both server-side and
client-side.

builder/dockerfile/ and api/server/builder.go contain implementations
of those interfaces as a refactoring of the current code.

Signed-off-by: Tibor Vass <tibor@docker.com>
2015-10-06 19:10:19 -04:00
Vivek Goyal
30047b6365 devmapper: Implement deferred deletion functionality
Finally here is the patch to implement deferred deletion functionality.
Deferred deleted devices are marked as "Deleted" in device meta file. 

First we try to delete the device and only if deletion fails and user has
enabled deferred deletion, device is marked for deferred deletion.

When docker starts up again, we go through list of deleted devices and
try to delete these again.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2015-10-06 17:37:21 -04:00
Antonio Murdaca
4ebc358c68 bump libcontainer to 902c012e85cdae6bb68d8c7a0df69a42f818ce96
Signed-off-by: Antonio Murdaca <amurdaca@redhat.com>
2015-10-06 17:55:09 +02:00
Vincent Demeester
6cf9826537 Remove use of testify mock and testify vendored lib
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2015-10-05 08:43:34 +02:00
Doug Davis
fdc3d2821e Merge pull request #16582 from vdemeester/16360-dockerCmd-raceytests
Fix TestDockerCmd*Timeout racey tests
2015-10-02 20:45:41 -04:00
Jess Frazelle
0d6442725c Merge pull request #16733 from MHBauer/perjury
remove testify asserts from pkg/discovery
2015-10-02 16:04:56 -07:00
Morgan Bauer
890fa18dc5 remove testify asserts from pkg/discovery
Signed-off-by: Morgan Bauer <mbauer@us.ibm.com>
2015-10-02 13:05:15 -07:00
Jess Frazelle
911628c5ae Merge pull request #16490 from Microsoft/10662-mtimefix
Fixed file modified time not changing on windows
2015-10-02 12:06:03 -07:00
Jess Frazelle
2de02bc741 Merge pull request #16159 from runcom/validate-cpuset-cpus
Validate --cpuset-cpus, --cpuset-mems
2015-10-02 11:30:46 -07:00
Jess Frazelle
5978f60a59 Merge pull request #16367 from Morgy93/names-generator
Added some adjectives and names
2015-10-02 11:29:09 -07:00
Jess Frazelle
34f6a8d666 Merge pull request #16470 from tonistiigi/fix-aufs-opq
Add basic support for .wh..wh..opq
2015-10-02 11:28:33 -07:00
Jess Frazelle
edd768c3a8 Merge pull request #16594 from Microsoft/sjw/unc-build-fix
Windows: Fixing longpath hanlding of UNC paths.
2015-10-02 11:24:27 -07:00
Darren Stahl
c7478a100c Fixed file modified time not changing on Windows
Signed-off-by: Darren Stahl <darst@microsoft.com>
2015-10-01 10:45:32 -07:00
Stefan J. Wernli
09a9d216d9 Windows: Fixing longpath hanlding of UNC paths.
Signed-off-by: Stefan J. Wernli <swernli@microsoft.com>
2015-09-30 19:36:08 -07:00
Tobias Klauser
61a438196e Add more amazing people to the names generator
Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2015-09-30 16:13:51 +02:00
Brian Goff
0a8a14a5fc Merge pull request #16642 from vdemeester/remove-question-make-in-pkg-devicemapper
Remove "(?)" from comments in pkg/devicemapper
2015-09-30 09:39:23 -04:00
Vincent Demeester
8cef102997 Remove (?) from comments in pkg/devicemapper
Got merged with it, removing it as it doesn't add anything.

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2015-09-30 10:35:02 +02:00
Tibor Vass
b454c14cbd Revert "Merge pull request #16228 from duglin/ContextualizeEvents"
Although having a request ID available throughout the codebase is very
valuable, the impact of requiring a Context as an argument to every
function in the codepath of an API request, is too significant and was
not properly understood at the time of the review.

Furthermore, mixing API-layer code with non-API-layer code makes the
latter usable only by API-layer code (one that has a notion of Context).

This reverts commit de4164043546d2b9ee3bf323dbc41f4979c84480, reversing
changes made to 7daeecd42d7bb112bfe01532c8c9a962bb0c7967.

Signed-off-by: Tibor Vass <tibor@docker.com>

Conflicts:
	api/server/container.go
	builder/internals.go
	daemon/container_unix.go
	daemon/create.go
2015-09-29 14:26:51 -04:00
Tonis Tiigi
bd780a0bb0 Add constants for AUFS whiteout files
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2015-09-29 11:21:45 -07:00
Brian Goff
92c1310f61 Merge pull request #16570 from duglin/ReaderFix
Make Close() on simpleReaderCloser actually close the reader
2015-09-29 12:55:03 -04:00
Tonis Tiigi
4c53da8f9a Add basic support for .wh..wh..opq
This fixes the case where directory is removed in
aufs and then the same layer is imported to a
different graphdriver.

Currently when you do `rm -rf /foo && mkdir /foo`
in a layer in aufs the files under `foo` would
only be be hidden on aufs.

The problems with this fix:

1) When a new diff is recreated from non-aufs driver
the `opq` files would not be there. This should not
mean layer differences for the user but still
different content in the tar (one would have one
`opq` file, the others would have `.wh.*` for every
file inside that folder). This difference also only
happens if the tar-split file isn’t stored for the
layer.

2) New files that have the filenames before `.wh..wh..opq`
when they are sorted do not get picked up by non-aufs
graphdrivers. Fixing this would require a bigger
refactoring that is planned in the future.


Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2015-09-29 08:22:25 -07:00
Vincent Demeester
0f1eca7265 Fix TestDockerCmd*Timeout racey tests
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2015-09-29 14:50:22 +02:00
Antonio Murdaca
4c50b33ed5 Validate --cpuset-cpus, --cpuset-mems
Before this patch libcontainer badly errored out with `invalid
argument` or `numerical result out of range` while trying to write
to cpuset.cpus or cpuset.mems with an invalid value provided.
This patch adds validation to --cpuset-cpus and --cpuset-mems flag along with
validation based on system's available cpus/mems before starting a container.

Signed-off-by: Antonio Murdaca <runcom@linux.com>
2015-09-27 16:38:58 +02:00
Arnaud Porterie
35c086fa6b Add pkg/discovery for nodes discovery
Absorb Swarm's discovery package in order to provide a common node
discovery mechanism to be used by both Swarm and networking code.

Signed-off-by: Arnaud Porterie <arnaud.porterie@docker.com>
2015-09-25 13:33:23 -07:00
Jess Frazelle
fdd5ab2fc3 Merge pull request #16569 from brahmaroutu/ioutil_hang_gccgo
goroutine fairness is not guaranteed causing the hang with GCCGO (x86 GCCGO CI)
2015-09-25 10:24:13 -07:00
Srini Brahmaroutu
2d0927a08b goroutine fairness is not guaranteed causing the hang with GCCGO
Signed-off-by: Srini Brahmaroutu <srbrahma@us.ibm.com>
2015-09-25 14:40:16 +00:00
Federico Gimenez
a67d5db216 Added test coverage to StdCopy closes #11584
Some corner cases and error conditions are covered while reading
and writing

Signed-off-by: Federico Gimenez <fgimenez@coit.es>
2015-09-25 11:15:45 +02:00
Doug Davis
83d0df9cdc Make Close() on simpleReaderCloser actually close the reader
Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-09-24 20:22:59 -07:00
David Calavera
c0bd79f3f8 Merge pull request #16228 from duglin/ContextualizeEvents
Add context.RequestID to event stream
2015-09-24 14:16:22 -07:00
Jess Frazelle
ce0cf62ce0 Merge pull request #16559 from Microsoft/10662-fix16556
Fixes 16556 CI failures
2015-09-24 12:31:36 -07:00
Jess Frazelle
ad3c33e73d Merge pull request #14579 from hqhq/hq_add_softlimit
Add support for memory reservation
2015-09-24 12:11:36 -07:00
Doug Davis
cd4326596a Add context.RequestID to event stream
This PR adds a "request ID" to each event generated, the 'docker events'
stream now looks like this:

```
2015-09-10T15:02:50.000000000-07:00 [reqid: c01e3534ddca] de7c5d4ca927253cf4e978ee9c4545161e406e9b5a14617efb52c658b249174a: (from ubuntu) create
```
Note the `[reqID: c01e3534ddca]` part, that's new.

Each HTTP request will generate its own unique ID. So, if you do a
`docker build` you'll see a series of events all with the same reqID.
This allow for log processing tools to determine which events are all related
to the same http request.

I didn't propigate the context to all possible funcs in the daemon,
I decided to just do the ones that needed it in order to get the reqID
into the events. I'd like to have people review this direction first, and
if we're ok with it then I'll make sure we're consistent about when
we pass around the context - IOW, make sure that all funcs at the same level
have a context passed in even if they don't call the log funcs - this will
ensure we're consistent w/o passing it around for all calls unnecessarily.

ping @icecrime @calavera @crosbymichael

Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-09-24 11:56:37 -07:00
John Howard
ec75a8049c Fixes 16556 CI failures
Signed-off-by: John Howard <jhoward@microsoft.com>
2015-09-24 11:19:00 -07:00
John Howard
b5e2a90aa3 TestRandomUnixTmpDirPath platform agnostic
Signed-off-by: John Howard <jhoward@microsoft.com>
2015-09-24 09:37:07 -07:00
Morgy93
c8becca3e4 Added some names
Signed-off-by: Thomas Hauschild <thomas@ulfertsprygoda.de>
2015-09-24 17:15:47 +02:00
Jess Frazelle
f09705e6aa Merge pull request #16190 from LK4D4/drain_refactor
Refactoring of bufReader
2015-09-24 01:54:40 -07:00