an [abandoned] attempt to slice out the docker/docker/pkg package
c45a95b9c0
The race is between pools.Put which calls buf.Reset and exec.Cmd doing io.Copy from the buffer; it caused a runtime crash, as described in #16924: ``` docker-daemon cat the-tarball.xz | xz -d -c -q | docker-untar /path/to/... (aufs ) ``` When docker-untar side fails (like try to set xattr on aufs, or a broken tar), invokeUnpack will be responsible to exhaust all input, otherwise `xz` will be write pending for ever. this change add a receive only channel to cmdStream, and will close it to notify it's now safe to close the input stream; in CmdStream the change to use Stdin / Stdout / Stderr keeps the code simple, os/exec.Cmd will spawn goroutines and call io.Copy automatically. the CmdStream is actually called in the same file only, change it lowercase to mark as private. [...] INFO[0000] Docker daemon commit=0a8c2e3 execdriver=native-0.2 graphdriver=aufs version=1.8.2 DEBU[0006] Calling POST /build INFO[0006] POST /v1.20/build?cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=Dockerfile&memory=0&memswap=0&rm=1&t=gentoo-x32&ulimits=null DEBU[0008] [BUILDER] Cache miss DEBU[0009] Couldn't untar /home/lib-docker-v1.8.2-tmp/tmp/docker-build316710953/stage3-x32-20151004.tar.xz to /home/lib-docker-v1.8.2-tmp/aufs/mnt/d909abb87150463939c13e8a349b889a72d9b14f0cfcab42a8711979be285537: Untar re-exec error: exit status 1: output: operation not supported DEBU[0009] CopyFileWithTar(/home/lib-docker-v1.8.2-tmp/tmp/docker-build316710953/stage3-x32-20151004.tar.xz, /home/lib-docker-v1.8.2-tmp/aufs/mnt/d909abb87150463939c13e8a349b889a72d9b14f0cfcab42a8711979be285537/) panic: runtime error: slice bounds out of range goroutine 42 [running]: bufio.(*Reader).fill(0xc208187800) /usr/local/go/src/bufio/bufio.go:86 +0x2db bufio.(*Reader).WriteTo(0xc208187800, 0x7ff39602d150, 0xc2083f11a0, 0x508000, 0x0, 0x0) /usr/local/go/src/bufio/bufio.go:449 +0x27e io.Copy(0x7ff39602d150, 0xc2083f11a0, 0x7ff3960261f8, 0xc208187800, 0x0, 0x0, 0x0) /usr/local/go/src/io/io.go:354 +0xb2 github.com/docker/docker/pkg/archive.func·006() /go/src/github.com/docker/docker/pkg/archive/archive.go:817 +0x71 created by github.com/docker/docker/pkg/archive.CmdStream /go/src/github.com/docker/docker/pkg/archive/archive.go:819 +0x1ec goroutine 1 [chan receive]: main.(*DaemonCli).CmdDaemon(0xc20809da30, 0xc20800a020, 0xd, 0xd, 0x0, 0x0) /go/src/github.com/docker/docker/docker/daemon.go:289 +0x1781 reflect.callMethod(0xc208140090, 0xc20828fce0) /usr/local/go/src/reflect/value.go:605 +0x179 reflect.methodValueCall(0xc20800a020, 0xd, 0xd, 0x1, 0xc208140090, 0x0, 0x0, 0xc208140090, 0x0, 0x45343f, ...) /usr/local/go/src/reflect/asm_amd64.s:29 +0x36 github.com/docker/docker/cli.(*Cli).Run(0xc208129fb0, 0xc20800a010, 0xe, 0xe, 0x0, 0x0) /go/src/github.com/docker/docker/cli/cli.go:89 +0x38e main.main() /go/src/github.com/docker/docker/docker/docker.go:69 +0x428 goroutine 5 [syscall]: os/signal.loop() /usr/local/go/src/os/signal/signal_unix.go:21 +0x1f created by os/signal.init·1 /usr/local/go/src/os/signal/signal_unix.go:27 +0x35 Signed-off-by: Derek Ch <denc716@gmail.com> |
||
|---|---|---|
| ansiescape | ||
| archive | ||
| broadcaster | ||
| chrootarchive | ||
| devicemapper | ||
| directory | ||
| discovery | ||
| fileutils | ||
| graphdb | ||
| homedir | ||
| httputils | ||
| idtools | ||
| integration | ||
| ioutils | ||
| jsonlog | ||
| jsonmessage | ||
| listenbuffer | ||
| longpath | ||
| mflag | ||
| mount | ||
| namesgenerator | ||
| nat | ||
| parsers | ||
| pidfile | ||
| plugins | ||
| pools | ||
| progressreader | ||
| promise | ||
| proxy | ||
| pubsub | ||
| random | ||
| reexec | ||
| signal | ||
| sockets | ||
| stdcopy | ||
| streamformatter | ||
| stringid | ||
| stringutils | ||
| symlink | ||
| sysinfo | ||
| system | ||
| tailfile | ||
| tarsum | ||
| term | ||
| timeoutconn | ||
| timeutils | ||
| tlsconfig | ||
| truncindex | ||
| ulimit | ||
| units | ||
| urlutil | ||
| useragent | ||
| version | ||
| README.md | ||
pkg/ is a collection of utility packages used by the Docker project without being specific to its internals.
Utility packages are kept separate from the docker core codebase to keep it as small and concise as possible. If some utilities grow larger and their APIs stabilize, they may be moved to their own repository under the Docker organization, to facilitate re-use by other projects. However that is not the priority.
The directory pkg is named after the same directory in the camlistore project. Since Brad is a core
Go maintainer, we thought it made sense to copy his methods for organizing Go code :) Thanks Brad!
Because utility packages are small and neatly separated from the rest of the codebase, they are a good place to start for aspiring maintainers and contributors. Get in touch if you want to help maintain them!