Commit graph

55 commits

Author SHA1 Message Date
Nalin Dahyabhai
2783450a92 Rename: cow/cowman -> storage/oci-storage
Rename the library module and CLI wrapper.
Rename daemon/graphdriver to drivers.
Catch up vendoring to match modules we've pruned.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2016-07-18 17:35:50 -04:00
Nalin Dahyabhai
f191bf69ff Remove more code
Based on a patch by Grant Seltzer <grantseltzer@gmail.com>

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2016-07-18 17:35:49 -04:00
Tibor Vass
59c8eda724 plugins: experimental support for new plugin management
This patch introduces a new experimental engine-level plugin management
with a new API and command line. Plugins can be distributed via a Docker
registry, and their lifecycle is managed by the engine.
This makes plugins a first-class construct.

For more background, have a look at issue #20363.

Documentation is in a separate commit. If you want to understand how the
new plugin system works, you can start by reading the documentation.

Note: backwards compatibility with existing plugins is maintained,
albeit they won't benefit from the advantages of the new system.

Signed-off-by: Tibor Vass <tibor@docker.com>
Signed-off-by: Anusha Ragunathan <anusha@docker.com>
2016-06-14 14:20:27 -07:00
Antonio Murdaca
bcc185702c authz: cleanups
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-06-12 17:23:19 +02:00
Brian Goff
7ee212504d Add support for volume scopes
This is similar to network scopes where a volume can either be `local`
or `global`. A `global` volume is one that exists across the entire
cluster where as a `local` volume exists on a single engine.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2016-06-05 15:37:15 -04:00
Brian Goff
f636807065 Enhance pluginrpc-gen parser
Now handles `package.Type` and `*package.Type`
Fixes parsing issues with slice and map types.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2016-06-05 15:37:15 -04:00
Brian Goff
ac615c51ba When calling volume driver Mount, send opaque ID
This generates an ID string for calls to Mount/Unmount, allowing drivers
to differentiate between two callers of `Mount` and `Unmount`.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2016-04-29 09:37:02 -04:00
Madhu Venugopal
81b695d5c4 Initialize activateWait for plugins activated by json spec
Signed-off-by: Madhu Venugopal <madhu@docker.com>
2016-04-20 05:59:19 -07:00
Victor Vieux
86abb0ae42 fix typo in comment
Signed-off-by: Victor Vieux <vieux@docker.com>
2016-04-08 00:05:31 -07:00
Brian Goff
b290cff825 Fix panic in loading plugins
When a plugin is first found, it is loaded into the available plugins
even though it's not activated yet.
If activation fails it is taken out of the list.
While it is in the list, other callers may see it and try to check it's
manifest. If it is not fully activated yet, the manifest will be nil and
cause a panic.

This is especially problematic for drivers that are down and have not
been activated yet.

We could just not load the plugin into the available list until it's
fully active, however that will just cause multiple of the same plugin
to attemp to be loaded.

We could check if the manifest is nil and return early (instead of
panicing on a nil manifest), but this will cause a 2nd caller to receive
a response while the first caller is still waiting, which can be
awkward.

This change uses a condition variable to handle activation (instead of
sync.Once). If the plugin is not activated, callers will all wait until
it is activated and receive a broadcast from the condition variable
signaling that it's ok to proceed, in which case we'll check if their
was an error in activation and proceed accordingly.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2016-03-23 15:34:15 -04:00
Antonio Murdaca
a4b96d76f7 *: fix response body leaks
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-16 17:15:42 +01:00
David Calavera
44005e59d4 Call plugins with custom transports.
Small refactor to be able to use custom transports
to call remote plugins.

Signed-off-by: David Calavera <david.calavera@gmail.com>
2016-03-02 12:54:53 -05:00
David Calavera
3b1fcb4db2 Merge pull request #20686 from clintonskitson/bugfix/plugin_desc_leak
Fix plugin file descriptor leaks
2016-02-29 10:44:36 -08:00
Clinton Kitson
f08caca8a0 Fixes plugin file descriptor leak on plugin discovery
Signed-off-by: Clinton Kitson <clintonskitson@gmail.com>
2016-02-26 19:43:50 -08:00
Brian Goff
f5003987da Close resp body on plugin call error
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2016-02-24 20:09:51 -05:00
Stefan Weil
8cb9b57bc2 Fix some typos in comments and strings
Most of them were found and fixed by codespell.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2016-02-22 20:27:15 +01:00
Alexander Morozov
54bafc26b7 Add support for forwarding Docker client through SOCKS proxy
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2016-02-16 11:09:28 -08:00
Victor Vieux
209fd53a0c fix common misspell
Signed-off-by: Victor Vieux <vieux@docker.com>
2016-02-11 15:49:36 -08:00
John Howard
ab793c4490 Windows CI: test-unit on pkg\plugins
Signed-off-by: John Howard <jhoward@microsoft.com>
2016-02-10 19:28:30 -08:00
David Calavera
52470a41b8 Apply context changes to the client.
Signed-off-by: David Calavera <david.calavera@gmail.com>
2016-02-04 13:59:57 -05:00
Brian Goff
1cb28d7bbe Add back compat for volume drivers Get and Ls
Use a back-compat struct to handle listing volumes for volumes we know
about (because, presumably, they are being used by a container) for
volume drivers which don't yet support `List`.

Adds a fall-back for the volume driver `Get` call, which will use
`Create` when the driver returns a `404` for `Get`. The old behavior was
to always use `Create` to get a volume reference.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2016-01-22 22:48:17 -05:00
Brian Goff
d41cd11fcb Bump plugin API version
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2016-01-21 11:41:32 -05:00
Brian Goff
486bca6d94 Don't error out on plugin err with json
We don't want to error out when there is a json unmarshal error since
the `old way` will cause this to error.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2016-01-06 11:34:14 -05:00
Brian Goff
390876ba18 Move responsibility of ls/inspect to volume driver
Makes `docker volume ls` and `docker volume inspect` ask the volume
drivers rather than only using what is cached locally.

Previously in order to use a volume from an external driver, one would
either have to use `docker volume create` or have a container that is
already using that volume for it to be visible to the other volume
API's.

For keeping uniqueness of volume names in the daemon, names are bound to
a driver on a first come first serve basis. If two drivers have a volume
with the same name, the first one is chosen, and a warning is logged
about the second one.

Adds 2 new methods to the plugin API, `List` and `Get`.
If a plugin does not implement these endpoints, a user will not be able
to find the specified volumes as well requests go through the drivers.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2016-01-05 16:28:38 -05:00
David Calavera
562ec005a6 Remove usage of pkg sockets and tlsconfig.
- Use the ones provided by docker/go-connections, they are a drop in replacement.
- Remove pkg/sockets from docker.
- Keep pkg/tlsconfig because libnetwork still needs it and there is a
  circular dependency issue.

Signed-off-by: David Calavera <david.calavera@gmail.com>
2015-12-29 19:27:12 -05:00
Antonio Murdaca
f35a0f3210 pkg: authorization: do not register the same plugin
This patches avoids registering (and calling) the same plugin more than
once. Using an helper map which indexes by name guarantees this and keeps
the order.
The behavior of overriding the same name in a flag is consistent with,
for instance, the `docker run -v /test -v /test` flag which register
the volume just once.
Adds integration tests.

Without this patch:
```
Dec 20 19:34:52 localhost.localdomain docker[9988]:
time="2015-12-20T19:34:52.080901676+01:00" level=debug msg="Calling
GET
/v1.22/info"
Dec 20 19:34:52 localhost.localdomain docker[9988]:
time="2015-12-20T19:34:52.081213202+01:00" level=debug msg="AuthZ
request using plugin docker-novolume-plugin"
Dec 20 19:34:52 localhost.localdomain docker[9988]:
time="2015-12-20T19:34:52.081268132+01:00" level=debug
msg="docker-novolume-plugin implements: authz"
Dec 20 19:34:52 localhost.localdomain docker[9988]:
time="2015-12-20T19:34:52.081699788+01:00" level=debug msg="AuthZ
request using plugin docker-novolume-plugin"
Dec 20 19:34:52 localhost.localdomain docker[9988]:
time="2015-12-20T19:34:52.081762507+01:00" level=debug
msg="docker-novolume-plugin implements: authz"
Dec 20 19:34:52 localhost.localdomain docker[9988]:
time="2015-12-20T19:34:52.082092480+01:00" level=debug msg="GET
/v1.22/info"
Dec 20 19:34:52 localhost.localdomain docker[9988]:
time="2015-12-20T19:34:52.628691038+01:00" level=debug msg="AuthZ
response using plugin docker-novolume-plugin"
Dec 20 19:34:52 localhost.localdomain docker[9988]:
time="2015-12-20T19:34:52.629880930+01:00" level=debug msg="AuthZ
response using plugin docker-novolume-plugin"
```

With this patch:
```
Dec 20 19:37:32 localhost.localdomain docker[16620]:
time="2015-12-20T19:37:32.376523958+01:00" level=debug msg="Calling
GET
/v1.22/info"
Dec 20 19:37:32 localhost.localdomain docker[16620]:
time="2015-12-20T19:37:32.376715483+01:00" level=debug msg="AuthZ
request using plugin docker-novolume-plugin"
Dec 20 19:37:32 localhost.localdomain docker[16620]:
time="2015-12-20T19:37:32.376771230+01:00" level=debug
msg="docker-novolume-plugin implements: authz"
Dec 20 19:37:32 localhost.localdomain docker[16620]:
time="2015-12-20T19:37:32.377698897+01:00" level=debug msg="GET
/v1.22/info"
Dec 20 19:37:32 localhost.localdomain docker[16620]:
time="2015-12-20T19:37:32.951016441+01:00" level=debug msg="AuthZ
response using plugin docker-novolume-plugin"
```

Also removes a somehow duplicate debug statement (leaving only the
second one as it's a loop of plugin's manifest):
```
Dec 20 19:52:30 localhost.localdomain docker[25767]:
time="2015-12-20T19:52:30.544090518+01:00" level=debug
msg="docker-novolume-plugin's manifest: &{[authz]}"
Dec 20 19:52:30 localhost.localdomain docker[25767]:
time="2015-12-20T19:52:30.544170677+01:00" level=debug
msg="docker-novolume-plugin implements: authz"
```

Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2015-12-23 21:08:40 +01:00
Antonio Murdaca
bc21007445 authZ: more fixes
- fix naming and formatting
- provide more context when erroring auth
- do not capitalize errors
- fix wrong documentation
- remove ugly remoteError{}

Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2015-12-18 16:29:01 +01:00
Antonio Murdaca
a82080a992 pkg: plugins: remove dead code
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2015-12-17 11:05:50 +01:00
Antonio Murdaca
b171259dde pkg: plugins: fix and better handle errors
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2015-12-15 17:51:48 +01:00
Lei Jitang
a1a4101b73 pkg/plugins/client.go: don't try to encode os decode if it's nil
When user call the `Call()` method, they don't always want to sent
some args or get the return value, so they use `nil` when call `Call()`
method and this will casue an error. It's better to not trying to
encode or decode if it's nil.

Signed-off-by: Lei Jitang <leijitang@huawei.com>
2015-12-05 02:55:50 -05:00
Phil Estes
e118299052 Add user namespace (mapping) support to the Docker engine
Adds support for the daemon to handle user namespace maps as a
per-daemon setting.

Support for handling uid/gid mapping is added to the builder,
archive/unarchive packages and functions, all graphdrivers (except
Windows), and the test suite is updated to handle user namespace daemon
rootgraph changes.

Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
2015-10-09 17:47:37 -04:00
Jess Frazelle
790b9791fd Merge pull request #13777 from cpuguy83/graphdriver_extpoints
Create extpoint for graphdrivers
2015-10-08 13:46:42 -07:00
Alexandre Beslic
9e0bd3aca0 Merge pull request #16494 from calavera/fix_plugin_url_scheme
Do not hardcode http as plugin URL scheme for secure connections.
2015-09-23 16:18:00 -07:00
Brian Goff
3b036bd029 Add README for pluginrpc-gen
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2015-09-22 19:34:44 -04:00
David Calavera
f624804e8f Do not hardcode http as plugin URL scheme for secure connections.
Signed-off-by: David Calavera <david.calavera@gmail.com>
2015-09-22 15:54:29 -04:00
Brian Goff
18649f7bf0 Create extpoint for graphdrivers
Allows people to create out-of-process graphdrivers that can be used
with Docker.

Extensions must be started before Docker otherwise Docker will fail to
start.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2015-09-09 20:24:35 -04:00
Vincent Demeester
721af25e56 Lint package pkg/plugins/pluginrpc-gen
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2015-09-05 23:15:13 +02:00
Stephen Rust
7eb8f570af Retry registering a volume driver
Signed-off-by: Stephen Rust <srust@blockbridge.com>
2015-09-01 14:56:29 -04:00
Brian Goff
1858b15498 Add volume API/CLI
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2015-08-26 13:37:52 -04:00
Darren Shepherd
bf81718eba Don't globally lock on driver initialization
This patch makes it such that plugin initialization is synchronized
based on the plugin name and not globally

Signed-off-by: Darren Shepherd <darren@rancher.com>
2015-08-18 22:25:30 -07:00
Vincent Demeester
9bcb3cba83 Lint on pkg/* packages
- pkg/useragent
- pkg/units
- pkg/ulimit
- pkg/truncindex
- pkg/timeoutconn
- pkg/term
- pkg/tarsum
- pkg/tailfile
- pkg/systemd
- pkg/stringutils
- pkg/stringid
- pkg/streamformatter
- pkg/sockets
- pkg/signal
- pkg/proxy
- pkg/progressreader
- pkg/pools
- pkg/plugins
- pkg/pidfile
- pkg/parsers
- pkg/parsers/filters
- pkg/parsers/kernel
- pkg/parsers/operatingsystem

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2015-07-27 21:26:21 +02:00
Sebastiaan van Stijn
8e5b6040fa Merge pull request #13951 from calavera/plugins_path
Separate plugin sockets and specs.
2015-07-17 21:11:31 +02:00
David Calavera
bee3972856 Separate plugin sockets and specs.
Check if there is a plugin socket first under `/run/docker/plugins/NAME.sock`.
If there is no socket for a plugin, check `/etc/docker/plugins/NAME.spec` and
`/usr/lib/docker/plugins/NAME.spec` for spec files.

Signed-off-by: David Calavera <david.calavera@gmail.com>
2015-07-16 14:20:07 -07:00
root
b220cdfe3b time duration should be nano seconds, gccgo treats it as zero
Signed-off-by: Srini Brahmaroutu <srbrahma@us.ibm.com>
2015-07-13 18:01:43 +00:00
Michael Crosby
dbb8849289 Merge pull request #13835 from cpuguy83/gen-prc
generate plugin clients via template
2015-07-07 17:17:39 -07:00
David Calavera
7312c4b8bb Plugins JSON spec.
Allow full configuration of external plugins via a JSON document.

Signed-off-by: David Calavera <david.calavera@gmail.com>
2015-06-29 10:32:18 -07:00
Brian Goff
f666eef0d4 generate plugin clients via template
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2015-06-10 11:41:40 -04:00
Zefan Li
2f287cc873 Don't forget to clenaup tmpdir in TestFileSpecPlugin()
Also remove redundant code.

Signed-off-by: Zefan Li <lizefan@huawei.com>
2015-06-04 21:26:15 +08:00
Phil Estes
93b26d601b Merge pull request #13577 from WeiZhang555/httpClose
bug fix: close http response body no longer in use
2015-05-29 16:59:17 -04:00
Zhang Wei
a0e7dc03a2 return error when failed to read http response body
Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
2015-05-29 12:52:51 +08:00