Commit graph

494 commits

Author SHA1 Message Date
Alexander Larsson
cd911b83ce beam: Fix double close of fds in SendUnix
Instead of calling syscall.Close() on the fds in sendUnix() we call
Close() on the *os.File in Send(). Otherwise the fd will be closed, but
the *os.File will continue to live, and when it is finalized the
fd will be closed (which by then may be reused and can be anything).

This also adds a note to Send() the the file is closed.

Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
2014-04-22 15:50:22 -07:00
Solomon Hykes
8104b14a8d beam: more unit tests
This adds testing to SendConn.

Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:22 -07:00
Solomon Hykes
0e093988bf beam/examples/beamsh: add scripts to reproduce various bugs
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:22 -07:00
Solomon Hykes
69e180cae7 beam/examples/beamsh: fix 'print' to be pass-through
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:22 -07:00
Solomon Hykes
ae0b194839 beam/examples/beamsh: rename 'log' to 'logger' to avoid conflict with stdout/stderr
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:22 -07:00
Solomon Hykes
c92abca6c7 beam/examples/beamsh: 'prompt' asks the user for a value and sends it.
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:22 -07:00
Solomon Hykes
5220dbeffd beam/examples/beamsh: commands are messages.
Commands in the pipeline should either implement or pass-through command messages.

This amounts to a proof-of-concept implementation of the "pipeline"
design of Docker plugins.

Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:22 -07:00
Solomon Hykes
2edf4802d8 beam/examples/beamsh: cosmetic fix in Fatalf
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:22 -07:00
Solomon Hykes
c27c56ca37 beam: replace SendPair() with the simpler SendConn()
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:21 -07:00
Solomon Hykes
e92f305c1c beam/examples/beamsh: miniserver.ds demo
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:21 -07:00
Solomon Hykes
9c5fdb249f beam: improve the API with Sender/Receiver interfaces and utilities: Copy/SendPipe/SendPair
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:21 -07:00
Alexander Larsson
071e5e5a65 beam: Make extracted Fds CloseOnExec
Grab forklock to make sure no forks accidentally inherit the new fds
before they are made CLOEXEC There is a slight race condition between
ReadMsgUnix returns and when we grap the lock, so this is not
perfect. Unfortunately There is no way to pass MSG_CMSG_CLOEXEC to
recvmsg() nor any way to implement non-blocking i/o in go, so this is
hard to fix.

Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
2014-04-22 15:50:21 -07:00
Solomon Hykes
f534839c8b beam/examples/beamsh: add a few example dockerscripts
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:21 -07:00
Solomon Hykes
7958f295e8 beam/examples/beamsh: simplify code by using sendWPipe utility
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:21 -07:00
Solomon Hykes
f7971cfc1e beam/examples/beamsh: move code around for readability
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:21 -07:00
Solomon Hykes
11c2feae5b beam/examples/beamsh: 'exec' adds 'fromcmd' field to its output
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:21 -07:00
Solomon Hykes
b27e7baa02 beam/examples/beamsh: simple 'log' command tees streams to a local directory
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:21 -07:00
Solomon Hykes
96bb0cf4a4 beam/examples/beamsh: 'render' and 'beamsh -x'
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:20 -07:00
Solomon Hykes
2198d2423c beam/examples/beamsh: run commands in an implicit context of pre-loaded 'plugins'
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:20 -07:00
Solomon Hykes
e99e1288cc beam/examples/beamsh: convenience commands 'devnull', 'echo' and 'stdio'
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:20 -07:00
Solomon Hykes
88b74798a4 beam/examples/beamsh: 'emit' supports key=value syntax to compose arbitrary objects
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:20 -07:00
Solomon Hykes
32a674a4fe dockerscript: '=' is not a special character
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:20 -07:00
Solomon Hykes
529e741f08 beam/examples/beamsh: support for background commands with '&' terminator
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:20 -07:00
Solomon Hykes
086724e1a3 beam/data: Message.Parse creates a message from shell-style 'key=value' arguments
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:20 -07:00
Solomon Hykes
99bbfb108d dockerscript: support '#' line comments
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:20 -07:00
Solomon Hykes
1a3c8051c5 beam/examples/beamsh: scripts can be passed as filenames
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:20 -07:00
Solomon Hykes
6344b09af3 beam/examples/beamsh: remote communication over beam (experimental).
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:20 -07:00
Solomon Hykes
d4fb68ee0f beam/examples/beamsh: 'in' chdirs to a directory. 'pass' does simple passthrough
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:19 -07:00
Solomon Hykes
d955d77dc8 beam/examples/beamsh: 'beamsend' command serializes all messages and sends them over a network connection
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:19 -07:00
Solomon Hykes
f49734592c beam/examples/beamsh: 'exec' command correctly closes stdout and stderr when the process exists
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:19 -07:00
Solomon Hykes
7e73a97a41 beam/examples/beamsh: don't print discarded messages in Devnull
This avoids false alarms when process exits without printing. Devnull
doesn't require synchronization.

Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:19 -07:00
Solomon Hykes
0f36b222f3 beam/examples/beamsh: more useful debugging
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:19 -07:00
Solomon Hykes
ca0aacdb6b beam/data: expose EncodeString for convenience access to the underlying netstring primitive
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:19 -07:00
Solomon Hykes
251353380e beam/examples/beamsh: utility function 'fileToConn'
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:19 -07:00
Solomon Hykes
23dd017763 beam/examples/beamsh: 'connect' command
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:19 -07:00
Solomon Hykes
7c4e5e60e8 beam/examples/beamsh: 'exec' and 'listen' commands
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:19 -07:00
Solomon Hykes
0dfc004450 beam/examples/beamsh: prettier devnull
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:18 -07:00
Solomon Hykes
bfb13651fa beam/data: prettier Message.Pretty()
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:18 -07:00
Solomon Hykes
f636d3fcec beam/examples/beamsh: prettier 'trace' command
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:18 -07:00
Solomon Hykes
4841bc9226 beam/data: convenience Message.Pretty() function
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:18 -07:00
Solomon Hykes
f295ccc72a beam/examples/beamsh: basic implementation of the pipeline design, with static handlers for now.
In the pipeline design, several beam commands can be run concurrently,
with their respective inputs and outputs connected in such a way that
beam messages flow from the first to last. This is similar to the way
a unix shell executes commands in a pipeline: instead of STDIN and
STDOUT, each beam command has a "BEAMIN" and "BEAMOUT".

Since beam allows for richer communication than plain byte streams, beam
pipelines can express more powerful computation, while retaining the
fundamental elegance and ease of use of unix-style composition.

Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:18 -07:00
Solomon Hykes
9f4bcf010a beam/examples/beamsh: better debugging messages
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:18 -07:00
Solomon Hykes
5efb90b0cd beam/examples/beamsh: simple 'exec' command
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:18 -07:00
Solomon Hykes
8b8e477ede beam/examples/beamsh: catch introspection calls from jobs for proper nesting
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:18 -07:00
Solomon Hykes
5df3aaeda2 beam: fix FD leaks in SendPipe
This fixes file descriptor leaks in the SendPipe function.

Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:18 -07:00
Solomon Hykes
3b5c1c4203 beam: more hooks for interactive debugging
This inserts low-level hooks for interactive step-by-step debugging.
Hooks are triggered by setting the *TEST* environment variable.

This is particularly useful for tracking down file descriptor leaks,
double-closing, or other issues which are difficult to debug with
the usual toolbox.

Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:17 -07:00
Solomon Hykes
a14c83eca3 beam/examples/beamsh: first try at nested execution
The "wiring" is broken because engine does not keep a reference for handling introspection calls.

Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:17 -07:00
Solomon Hykes
438caf6e06 beam/examples/beamsh: fix a bug in the log command
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:17 -07:00
Solomon Hykes
cd44538901 beam/data: convenience Message.Get
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:17 -07:00
Solomon Hykes
b78465fc2a beam/examples/beamsh: use beam/data
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
2014-04-22 15:50:17 -07:00