We encountered a situation where concurrent invocations of the docker daemon on a machine with an older version of iptables led to nondeterministic errors related to simultaenous invocations of iptables.
While this is best resolved by upgrading iptables itself, the particular situation would have been avoided if the docker daemon simply took care not to concurrently invoke iptables. Of course, external processes could also cause iptables to fail in this way, but invoking docker in parallel seems like a pretty common case.
Signed-off-by: Aaron Davidson <aaron@databricks.com>
If we tear through a few layers of abstraction, we can get at the inodes
contained in a directory without having to stat all the files. This
allows us to eliminate identical files much earlier in the changelist
generation process.
Signed-off-by: Burke Libbey <burke@libbey.me>
Add tests on:
- changes.go
- archive.go
- wrap.go
Should fix#11603 as the coverage is now 81.2% on the ``pkg/archive``
package. There is still room for improvement though :).
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
If memory cgroup is mounted, memory limit is always supported,
no need to check if these files are exist.
Signed-off-by: Qiang Huang <h.huangqiang@huawei.com>
Deferred reove functionality was added to library later. So in old version
of library it did not report deferred_remove field.
Create a new function which also gets deferred_remove field and it will be
called only on newer version of library.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
If a device has been scheduled for deferred deactivation and container
is started again and we need to activate device again, we need to cancel
the deferred deactivation which is already scheduled on the device.
Create a method for the same.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
A lot of time device mapper devices leak across mount namespace which docker
does not know about and when docker tries to deactivate/delete device,
operation fails as device is open in some mount namespace.
Create a mechanism where one can defer the device deactivation/deletion
so that docker operation does not fail and device automatically goes
away when last reference to it is dropped.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
When firewalld (or iptables service) restarts/reloads,
all previously added docker firewall rules are flushed.
With firewalld we can react to its Reloaded() [1]
D-Bus signal and recreate the firewall rules.
Also when firewalld gets restarted (stopped & started)
we can catch the NameOwnerChanged signal [2].
To specify which signals we want to react to we use AddMatch [3].
Libvirt has been doing this for quite a long time now.
Docker changes firewall rules on basically 3 places.
1) daemon/networkdriver/portmapper/mapper.go - port mappings
Portmapper fortunatelly keeps list of mapped ports,
so we can easily recreate firewall rules on firewalld restart/reload
New ReMapAll() function does that
2) daemon/networkdriver/bridge/driver.go
When setting a bridge, basic firewall rules are created.
This is done at once during start, it's parametrized and nowhere
tracked so how can one know what and how to set it again when
there's been firewalld restart/reload ?
The only solution that came to my mind is using of closures [4],
i.e. I keep list of references to closures (anonymous functions
together with a referencing environment) and when there's firewalld
restart/reload I re-call them in the same order.
3) links/links.go - linking containers
Link is added in Enable() and removed in Disable().
In Enable() we add a callback function, which creates the link,
that's OK so far.
It'd be ideal if we could remove the same function from
the list in Disable(). Unfortunatelly that's not possible AFAICT,
because we don't know the reference to that function
at that moment, so we can only add a reference to function,
which removes the link. That means that after creating and
removing a link there are 2 functions in the list,
one adding and one removing the link and after
firewalld restart/reload both are called.
It works, but it's far from ideal.
[1] https://jpopelka.fedorapeople.org/firewalld/doc/firewalld.dbus.html#FirewallD1.Signals.Reloaded
[2] http://dbus.freedesktop.org/doc/dbus-specification.html#bus-messages-name-owner-changed
[3] http://dbus.freedesktop.org/doc/dbus-specification.html#message-bus-routing-match-rules
[4] https://en.wikipedia.org/wiki/Closure_%28computer_programming%29
Signed-off-by: Jiri Popelka <jpopelka@redhat.com>
Firewalld [1] is a firewall managing daemon with D-Bus interface.
What sort of problem are we trying to solve with this ?
Firewalld internally also executes iptables/ip6tables to change firewall settings.
It might happen on systems where both docker and firewalld are running
concurrently, that both of them try to call iptables at the same time.
The result is that the second one fails because the first one is holding a xtables lock.
One workaround is to use --wait/-w option in both
docker & firewalld when calling iptables.
It's already been done in both upstreams:
b315c380f4b3b451d6f8
But it'd still be better if docker used firewalld when it's running.
Other problem the firewalld support would solve is that
iptables/firewalld service's restart flushes all firewall rules
previously added by docker.
See next patch for possible solution.
This patch utilizes firewalld's D-Bus interface.
If firewalld is running, we call direct.passthrough() [2] method instead
of executing iptables directly.
direct.passthrough() takes the same arguments as iptables tool itself
and passes them through to iptables tool.
It might be better to use other methods, like direct.addChain and
direct.addRule [3] so it'd be more intergrated with firewalld, but
that'd make the patch much bigger.
If firewalld is not running, everything works as before.
[1] http://www.firewalld.org/
[2] https://jpopelka.fedorapeople.org/firewalld/doc/firewalld.dbus.html#FirewallD1.direct.Methods.passthrough
[3] https://jpopelka.fedorapeople.org/firewalld/doc/firewalld.dbus.html#FirewallD1.direct.Methods.addChainhttps://jpopelka.fedorapeople.org/firewalld/doc/firewalld.dbus.html#FirewallD1.direct.Methods.addRule
Signed-off-by: Jiri Popelka <jpopelka@redhat.com>
Check test correctness of untar by comparing destination with
source. For part 2, it checkes hashes of source and destination
files or the target files of symbolic links.
This is a supplement to the #11601 fix.
Signed-off-by: Yestin Sun <sunyi0804@gmail.com>
