Commit graph

43 commits

Author SHA1 Message Date
Nalin Dahyabhai
aded075f80 Rework some of the build machinery
- Run tests in VMs with sufficiently-new gccgo
- Assume that all binary builds build dynamic binaries

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2016-07-28 14:01:54 -04:00
Nalin Dahyabhai
2783450a92 Rename: cow/cowman -> storage/oci-storage
Rename the library module and CLI wrapper.
Rename daemon/graphdriver to drivers.
Catch up vendoring to match modules we've pruned.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2016-07-18 17:35:50 -04:00
Derek McGowan
003449018b Set permission on atomic file write
Perform chmod before rename with the atomic file writer.
Ensure writeErr is set on short write and file is removed on write error.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-06-29 13:09:13 -07:00
Alexander Morozov
a6496e5b41 ioutils: fix race in access closeErr in bytespipe
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2016-06-06 11:29:47 -07:00
allencloud
b4488c647b fix typos
Signed-off-by: allencloud <allen.sun@daocloud.io>
2016-06-02 17:17:22 +08:00
Brian Goff
1d2a9f2ace Fix concurrent map access in bytespipe
When getting and returning a buffer, need to make sure to syncronize
access to the pools map.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2016-05-12 10:04:05 -04:00
Tonis Tiigi
9333729a55 Safer file io for configuration files
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2016-04-21 11:31:15 -07:00
Tonis Tiigi
3b3d686788 Fix closing attach streams on lost tcp connection
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2016-04-06 21:27:47 -07:00
Brian Goff
729c6a44bc Improve performance/reduce allocs of bytespipe
Creates a `fixedBuffer` type that is used to encapsulate functionality
for reading/writing from the underlying byte slices.

Uses lazily-loaded set of sync.Pools for storing buffers that are no
longer needed so they can be re-used.

```
benchmark                     old ns/op     new ns/op     delta
BenchmarkBytesPipeWrite-8     138469        48985         -64.62%
BenchmarkBytesPipeRead-8      130922        56601         -56.77%

benchmark                     old allocs     new allocs     delta
BenchmarkBytesPipeWrite-8     18             8              -55.56%
BenchmarkBytesPipeRead-8      0              0              +0.00%

benchmark                     old bytes     new bytes     delta
BenchmarkBytesPipeWrite-8     66903         1649          -97.54%
BenchmarkBytesPipeRead-8      0             1             +Inf%
```

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2016-04-05 13:06:04 -04:00
Alexander Morozov
dfeb86d734 pkg: cleanup some unused code
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2016-03-31 11:39:59 -07:00
Brian Goff
7982606371 Cleanup WriteFlusher
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2016-02-09 14:02:26 -05:00
Justas Brazauskas
244162287a Fix typos found across repository
Signed-off-by: Justas Brazauskas <brazauskasjustas@gmail.com>
2015-12-13 18:04:12 +02:00
Aaron Lehmann
41193db82e Improved push and pull with upload manager and download manager
This commit adds a transfer manager which deduplicates and schedules
transfers, and also an upload manager and download manager that build on
top of the transfer manager to provide high-level interfaces for uploads
and downloads. The push and pull code is modified to use these building
blocks.

Some benefits of the changes:

- Simplification of push/pull code
- Pushes can upload layers concurrently
- Failed downloads and uploads are retried after backoff delays
- Cancellation is supported, but individual transfers will only be
  cancelled if all pushes or pulls using them are cancelled.
- The distribution code is decoupled from Docker Engine packages and API
  conventions (i.e. streamformatter), which will make it easier to split
  out.

This commit also includes unit tests for the new distribution/xfer
package. The tests cover 87.8% of the statements in the package.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-12-09 19:13:35 -08:00
Aaron Lehmann
039582f49e Cap the amount of buffering done by BytesPipe
Turn BytesPipe's Read and Write functions into blocking, goroutine-safe
functions. Add a CloseWithError function to propagate an error code to
the Read function.

Adjust tests to work with the blocking Read and Write functions.

Remove BufReader, since now its users can use BytesPipe directly.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-11-12 10:11:29 -08:00
Stephen J Day
c02491b8c3 Avoid panic on write after close in http
By adding a (*WriteFlusher).Close, we limit the Write calls to possibly
deallocated http response buffers to the lifetime of an http request.
Typically, this is seen as a very confusing panic, the cause is usually a
situation where an http.ResponseWriter is held after request completion. We
avoid the panic by disallowing further writes to the response writer after the
request is completed.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-11-02 18:14:43 -08:00
Tibor Vass
e5a4257fa7 Abstract builder and implement server-side dockerfile builder
This patch creates interfaces in builder/ for building Docker images.
It is a first step in a series of patches to remove the daemon
dependency on builder and later allow a client-side Dockerfile builder
as well as potential builder plugins.

It is needed because we cannot remove the /build API endpoint, so we
need to keep the server-side Dockerfile builder, but we also want to
reuse the same Dockerfile parser and evaluator for both server-side and
client-side.

builder/dockerfile/ and api/server/builder.go contain implementations
of those interfaces as a refactoring of the current code.

Signed-off-by: Tibor Vass <tibor@docker.com>
2015-10-06 19:10:19 -04:00
Brian Goff
92c1310f61 Merge pull request #16570 from duglin/ReaderFix
Make Close() on simpleReaderCloser actually close the reader
2015-09-29 12:55:03 -04:00
Srini Brahmaroutu
2d0927a08b goroutine fairness is not guaranteed causing the hang with GCCGO
Signed-off-by: Srini Brahmaroutu <srbrahma@us.ibm.com>
2015-09-25 14:40:16 +00:00
Doug Davis
83d0df9cdc Make Close() on simpleReaderCloser actually close the reader
Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-09-24 20:22:59 -07:00
Jess Frazelle
f09705e6aa Merge pull request #16190 from LK4D4/drain_refactor
Refactoring of bufReader
2015-09-24 01:54:40 -07:00
Tonis Tiigi
3c6dcfb6ca Make bytesPipe use linear allocations
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2015-09-23 17:12:54 -07:00
Alexander Morozov
46b458fe3b Refactor bufReader to use BytesPipe
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2015-09-23 16:00:28 -07:00
Alexander Morozov
434c274741 Add BytesPipe datastructure to ioutils
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2015-09-23 16:00:27 -07:00
Hu Keping
0f16fb62e3 Format output of docker info
Format those info which will only be displayed when daemon is
in debug mode.

Signed-off-by: Hu Keping <hukeping@huawei.com>
2015-09-23 14:35:13 +08:00
unclejack
cab12d1a89 pkg/ioutils: lower bufReader reset timeout
Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com>
2015-08-28 23:13:02 +03:00
Veres Lajos
0f78f158aa typofix - https://github.com/vlajos/misspell_fixer
Signed-off-by: Veres Lajos <vlajos@gmail.com>
2015-08-07 23:25:49 +01:00
Lei Jitang
fcb8d8d48e Enable golint part of #14756
pkg/broadcastwriter
pkg/graphdb
pkg/httputils
pkg/ioutils

Signed-off-by: Lei Jitang <leijitang@huawei.com>
2015-08-03 09:45:05 +08:00
David Calavera
62ccc23024 Fix reset timeout for buffer readers.
Use our goroutine-safe random source.

Signed-off-by: David Calavera <david.calavera@gmail.com>
2015-07-28 14:30:18 -07:00
Brian Goff
7d99b19364 Split reader interface from logger interface
Implement new reader interface on jsonfile.
Moves jsonlog decoding from daemon to jsonfile logger.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2015-07-21 20:47:31 -04:00
David Calavera
13a2948dc8 Merge pull request #14605 from brahmaroutu/gccgo_scheduler
Go Scheduler issue with sync.Mutex
2015-07-17 08:16:32 -07:00
root
b0e927fed5 Go Scheduler issue with sync.Mutex using gccgo
Signed-off-by: Srini Brahmaroutu <srbrahma@us.ibm.com>
2015-07-17 01:33:58 +00:00
Tibor Vass
1601bba9a5 remove pkg/transport and use the one from distribution
Signed-off-by: Tibor Vass <tibor@docker.com>
2015-07-16 13:13:46 -04:00
Victor Vieux
27435a1e56 no not print empty keys in docker info
Signed-off-by: Victor Vieux <victorvieux@gmail.com>
2015-06-01 15:05:20 -07:00
vagrant
958e93287e fix to gccgo compilation error
Signed-off-by: Srini Brahmaroutu <srbrahma@us.ibm.com>
2015-05-14 16:28:23 +00:00
Tianon Gravi
a50705d3d6 Move WriteFlusher out of utils into ioutils
Signed-off-by: Andrew "Tianon" Page <admwiggin@gmail.com>
2015-05-08 12:33:33 -06:00
Vincent Demeester
6b59e983e3 Add more ioutils tests.
Closes #11595

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2015-05-04 19:56:10 +02:00
Antonio Murdaca
d7a5d5b94c Refactor utils/utils, fixes #11923
Signed-off-by: Antonio Murdaca <me@runcom.ninja>
2015-04-14 01:37:36 +02:00
unclejack
3b45ed0fee pkg/ioutils: avoid huge Buffer growth in bufreader
Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com>
2015-03-18 19:51:55 +02:00
unclejack
0706016675 pkg/ioutils: add tests for BufReader
Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com>
2015-03-18 19:32:46 +02:00
unclejack
2a633df603 pkg/ioutils: add ReaderErrWrapper to readers
Docker-DCO-1.1-Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com> (github: unclejack)
2014-09-03 11:36:21 +03:00
unclejack
65f57a95bb pkg/ioutils: add NewBufReaderWithDrainbufAndBuffer
Docker-DCO-1.1-Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com> (github: unclejack)
2014-09-03 11:36:21 +03:00
unclejack
028b28e406 pkg/ioutils: add WriteCloserWrapper
Docker-DCO-1.1-Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com> (github: unclejack)
2014-09-03 11:36:21 +03:00
unclejack
27a8494fa3 move some io related utils to pkg/ioutils
Docker-DCO-1.1-Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com> (github: unclejack)
2014-09-03 11:36:21 +03:00