Commit graph

1756 commits

Author SHA1 Message Date
David Calavera
17792e996c Merge pull request #18651 from vbatts/dm-cleanup
loopback (and devicemapper) cleanup
2015-12-18 15:13:28 -08:00
Vincent Batts
74e35aace0 loopback: separate loop logic from devicemapper
The loopback logic is not technically exclusive to the devicemapper
driver. This reorganizes the code such that the loopback code is usable
outside of the devicemapper package and driver.

Signed-off-by: Vincent Batts <vbatts@redhat.com>
2015-12-18 10:57:43 -05:00
Vincent Batts
c28b794f70 devicemapper: remove unused type mapping
Signed-off-by: Vincent Batts <vbatts@redhat.com>
2015-12-18 10:44:06 -05:00
Antonio Murdaca
bc21007445 authZ: more fixes
- fix naming and formatting
- provide more context when erroring auth
- do not capitalize errors
- fix wrong documentation
- remove ugly remoteError{}

Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2015-12-18 16:29:01 +01:00
Alexander Morozov
7051dc2661 Merge pull request #18744 from runcom/plugins-deadcode
pkg: plugins: remove dead code
2015-12-17 10:50:04 -08:00
Tibor Vass
ab40ebacb7 Merge pull request #18745 from runcom/pkg-version-String
pkg: version: add String method
2015-12-17 16:58:41 +01:00
Antonio Murdaca
41af5c35eb pkg: version: add String method
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2015-12-17 11:52:23 +01:00
Antonio Murdaca
16d022c482 pkg: authorization: add Err to tweak response status code
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2015-12-17 11:08:47 +01:00
Antonio Murdaca
a82080a992 pkg: plugins: remove dead code
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2015-12-17 11:05:50 +01:00
Brian Goff
16870807fe Merge pull request #18695 from dnephin/move_parse_link
Move ParseLink and validators into runconfig.parse where they are used
2015-12-16 16:15:52 -05:00
Vincent Demeester
d0b9fc9422 Merge pull request #18722 from calavera/remove_unused_packages
Remove timeoutconn package, it's not used anywhere
2015-12-16 20:56:43 +01:00
Vincent Demeester
172ba92d25 Merge pull request #18518 from MHBauer/spurious-timing
adjust test sleep timing to avoid spurious failure
2015-12-16 20:45:55 +01:00
Daniel Nephin
8e98c98e52 Move ParseLink and validators into runconfig.parse where they are used.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-12-16 14:22:54 -05:00
David Calavera
11a6db3391 Remove timeoutconn package.
It's not used anywhere.

Signed-off-by: David Calavera <david.calavera@gmail.com>
2015-12-16 13:51:56 -05:00
David Calavera
d88419060e Merge pull request #18682 from calavera/replace_units_package
Replace pkg/units with docker/go-units.
2015-12-16 10:48:59 -08:00
Antonio Murdaca
e0a1a648d8 Merge pull request #18693 from dnephin/move_parse_docker_host
Move ParseDockerDaemonHost to opts/ package
2015-12-16 19:05:07 +01:00
David Calavera
a076a297ac Replace pkg/units with docker/go-units.
Signed-off-by: David Calavera <david.calavera@gmail.com>
2015-12-16 12:26:49 -05:00
Antonio Murdaca
fb77ffd682 pkg: authorization: cleanup
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2015-12-16 12:01:04 +01:00
Daniel Nephin
91d25997f1 Move ParseDockerDaemonHost to opts/ package.
This function was only being used from a single place opts/opts.go. This
change moves it from a incohesive package (parsers) to the single place it
is used.

Also made a bunch of the helper methods private because they are not used
by any external modules.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-12-15 20:53:17 -05:00
David Calavera
207e5e455a Move filters package to the API.
These filters are only use to interchange data between clients and daemons.
They don't belong to the parsers package.

Signed-off-by: David Calavera <david.calavera@gmail.com>
2015-12-15 18:13:26 -05:00
David Calavera
0b059a5246 Merge pull request #18685 from calavera/remove_timeutils
Move timeutils functions to the only places where they are used.
2015-12-15 15:11:18 -08:00
David Calavera
54ce12da10 Merge pull request #18677 from runcom/fix-plugins-error
pkg: plugins: fix and better handle errors
2015-12-15 14:17:52 -08:00
Morgan Bauer
995814da2d adjust test sleep timing to avoid spurious failure
- refactor ConsumeWithSpeed
 - documentation

Signed-off-by: Morgan Bauer <mbauer@us.ibm.com>
2015-12-15 13:27:24 -08:00
David Calavera
0585b88aee Move timeutils functions to the only places where they are used.
- Move time json marshaling to the jsonlog package: this is a docker
  internal hack that we should not promote as a library.
- Move Timestamp encoding/decoding functions to the API types: This is
  only used there. It could be a standalone library but I don't this
it's worth having a separated repo for this. It could introduce more
complexity than it solves.

Signed-off-by: David Calavera <david.calavera@gmail.com>
2015-12-15 14:56:14 -05:00
Antonio Murdaca
b171259dde pkg: plugins: fix and better handle errors
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2015-12-15 17:51:48 +01:00
Tibor Vass
202e73c7ad builder: remove container package dependency
Signed-off-by: Tibor Vass <tibor@docker.com>
2015-12-15 17:24:07 +01:00
Chris Dituri
583754b67c Make pkg/devicemapper/ log messages with a common, consistent prefix.
Closes #16667

Uses the prefix "devicemapper:" for all the fmt and logrus error, debug, and info messages.

Signed-off-by: Chris Dituri <csdituri@gmail.com>
2015-12-14 21:35:15 -06:00
Tibor Vass
c999f3afa3 utils: move git functions to pkg/gitutils
Signed-off-by: Tibor Vass <tibor@docker.com>
2015-12-14 14:59:52 +01:00
Justas Brazauskas
244162287a Fix typos found across repository
Signed-off-by: Justas Brazauskas <brazauskasjustas@gmail.com>
2015-12-13 18:04:12 +02:00
Antonio Murdaca
df5dc52c05 Merge pull request #15365 from twistlock/14674-docker-authz
Docker authorization plug-in infrastructure
2015-12-12 12:30:33 +01:00
Alexander Morozov
ffa3493afc Merge pull request #18353 from aaronlehmann/transfer-manager
Improved push and pull with upload manager and download manager
2015-12-10 14:52:48 -08:00
Aaron Lehmann
41193db82e Improved push and pull with upload manager and download manager
This commit adds a transfer manager which deduplicates and schedules
transfers, and also an upload manager and download manager that build on
top of the transfer manager to provide high-level interfaces for uploads
and downloads. The push and pull code is modified to use these building
blocks.

Some benefits of the changes:

- Simplification of push/pull code
- Pushes can upload layers concurrently
- Failed downloads and uploads are retried after backoff delays
- Cancellation is supported, but individual transfers will only be
  cancelled if all pushes or pulls using them are cancelled.
- The distribution code is decoupled from Docker Engine packages and API
  conventions (i.e. streamformatter), which will make it easier to split
  out.

This commit also includes unit tests for the new distribution/xfer
package. The tests cover 87.8% of the statements in the package.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-12-09 19:13:35 -08:00
Brian Goff
f576cf770a Merge pull request #18538 from estesp/aufs-whiteout-userns-fix
Skip aufs whiteout files on userns translation to container
2015-12-09 14:47:01 -05:00
Sebastiaan van Stijn
17aa7f8b20 Remove redundant check
jm.ID is already checked in the outer "if",
so theres no reason to check it again here.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2015-12-09 17:38:06 +01:00
Phil Estes
aa06163678 Skip aufs whiteout files on userns translation to container
aufs kernel module creates whiteout files on upper layer delete (and
other situations) and those files already are 'translated' regarding
ownership in host terms (e.g. they are already "0:0" owned), so when
these layers are copied around with pkg/archive we don't want to try and
translate these files regarding ownership.

Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
2015-12-09 10:11:32 -05:00
Dima Stopel
c39dff3ecd Fixing documentation according to comments by @moxiegirl and @thaJeztah
Signed-off-by: Dima Stopel <dima@twistlock.com>
2015-12-08 17:34:15 +02:00
Liron Levin
ccdb366aa5 Docker authorization plug-in infrastructure enables extending the functionality of the Docker daemon with respect to user authorization. The infrastructure enables registering a set of external authorization plug-in. Each plug-in receives information about the user and the request and decides whether to allow or deny the request. Only in case all plug-ins allow accessing the resource the access is granted.
Each plug-in operates as a separate service, and registers with Docker
through general (plug-ins API)
[https://blog.docker.com/2015/06/extending-docker-with-plugins/]. No
Docker daemon recompilation is required in order to add / remove an
authentication plug-in. Each plug-in is notified twice for each
operation: 1) before the operation is performed and, 2) before the
response is returned to the client. The plug-ins can modify the response
that is returned to the client.

The authorization depends on the authorization effort that takes place
in parallel [https://github.com/docker/docker/issues/13697].

This is the official issue of the authorization effort:
https://github.com/docker/docker/issues/14674

(Here)[https://github.com/rhatdan/docker-rbac] you can find an open
document that discusses a default RBAC plug-in for Docker.

Signed-off-by: Liron Levin <liron@twistlock.com>
Added container create flow test and extended the verification for ps
2015-12-08 17:34:15 +02:00
Aaron Lehmann
6e9a3453f7 Fix the scoping of "diff" so its value doesn't leak between loop iterations
In the existing code, "diff" has function scope and the value from the
previous iteration may be used if it is not reset. This appears to be an
oversight. This commit changes its scope to the for loop body.

One confusing point is that the cursor movement escape sequences appear
to be necessary even if the requested movement is 0. I haven't been able
to figure out why this makes a difference.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-12-07 17:01:47 -08:00
Aaron Lehmann
5821bbd01d Don't update lines on the terminal from a previous operation
When we handle a message that isn't tracked in the "line" map (for
example, one with no ID), clear the line map. This means we won't update
lines that were part of a previous, completed set of operations when
doing something like pull -a. It also has the beneficial side effect
of avoiding terminal glitching in these types of situations, since
messages that don't get tracked in the "line" map cause the count of the
number of lines to get out of sync.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-12-07 16:04:42 -08:00
Sebastiaan van Stijn
cd655d8b48 Merge pull request #18449 from coolljt0725/plugins_cleanup
pkg/plugins/client.go: don't try to encode os decode if it's nil
2015-12-07 16:03:28 +01:00
Tibor Vass
cf5299f8bc Merge pull request #18123 from aidanhs/aphs-fail-on-broken-tar
Ensure adding a broken tar doesn't silently fail
2015-12-07 14:38:21 +01:00
Lei Jitang
a1a4101b73 pkg/plugins/client.go: don't try to encode os decode if it's nil
When user call the `Call()` method, they don't always want to sent
some args or get the return value, so they use `nil` when call `Call()`
method and this will casue an error. It's better to not trying to
encode or decode if it's nil.

Signed-off-by: Lei Jitang <leijitang@huawei.com>
2015-12-05 02:55:50 -05:00
Sebastiaan van Stijn
92079d14e1 Merge pull request #14466 from Mashimiao/add-support-blkio_throtte_bps
Add support for blkio read/write bps device
2015-12-04 12:29:58 +01:00
Alexander Morozov
b4547e38b1 Merge pull request #18412 from aaronlehmann/runcommand-race
Fix race in RunCommandWithOutputForDuration
2015-12-03 19:35:04 -08:00
Ma Shimiao
6986fd5e3d Add support for blkio read/write bps device
Signed-off-by: Ma Shimiao <mashimiao.fnst@cn.fujitsu.com>
2015-12-04 09:26:03 +08:00
Alexander Morozov
f9baa50548 Merge pull request #18411 from aaronlehmann/unit-test-timing
Adjust TestDockerCmdWithTimeout timeout to improve reliability
2015-12-03 16:34:36 -08:00
Aaron Lehmann
eb7916eb53 Fix race in RunCommandWithOutputForDuration
This function was starting a goroutine that modifies one of its return
values. The intent is for the goroutine to only influence the return
value when it's causing the function to return, but it's racy and can
also modify the return value when the function is returning due to the
timeout. Fix the goroutine to not modify return values directly.

Also, give the channel a buffer so that the goroutine doesn't block
forever after a timeout.

Fixes #18305

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-12-03 16:34:01 -08:00
Aaron Lehmann
6ba6058477 Adjust TestDockerCmdWithTimeout timeout to improve reliability
I saw a failure of TestDockerCmdWithTimeout. This test starts a command
that produces output after 10 ms, but uses a 5 ms timeout, so normally
the command will be killed before the output. The time intervals are so
small that the timeout may not reliably trigger before the output, which
can cause the test to fail.

This commit changes the test to only fail if the process is still alive
after 10 seconds. This means the test will confirm that the timeouts are
happening, but not attempt to gauge that the timeouts are happening
within milliseconds of when they are expected (which can't be done
reliably).

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-12-03 14:42:32 -08:00
Tibor Vass
de5a23ebff Merge pull request #18266 from calavera/events_pub_sub
Event PubSub topics + linear filtering.
2015-12-03 17:11:40 +01:00
David Calavera
a72c316bf4 Add PubSub topics.
A TopicFunc is an interface to let the pubisher decide whether it needs
to send a message to a subscriber or not. It returns true if the
publisher must send the message and false otherwise.

Users of the pubsub package can create a subscriber with a topic
function by calling `pubsub.SubscribeTopic`.

Message delivery has also been modified to use concurrent channels per
subscriber. That way, topic verification and message delivery is not
o(N+M) anymore, based on the number of subscribers and topic verification
complexity.

Using pubsub topics, the API stops controlling the message delivery,
delegating that function to a topic generated with the filtering
provided by the user. The publisher sends every message to the
subscriber if there is no filter, but the api doesn't have to select
messages to return anymore.

Signed-off-by: David Calavera <david.calavera@gmail.com>
2015-12-02 16:43:49 -05:00