Commit graph

1941 commits

Author SHA1 Message Date
Chris Evich
4df075abd2 Improve control-host CPU performance
When running from a central host, where multiple other playbooks may
also be executing, CPU time quickly becomes the scaleability bottleneck.

* Reduce the vars compression level at the cost of network utilization.
This assumes the number of vars being transfered back/forth remains
reasonably low, where there wouldn't be much advantage from higher
compression anyway.

Another enhancement ``ControlPersist`` (for ssh) is apt to fall
back to opening new connections (slow) for every request under some
conditions.  This happens if the socket filename is too large (108
characters, including path) - a kernel limitation.

Unfortunately, in cloud environments, auto-assigned VM hostnames tend to
be rather large to avoid clashes.  Worse, in a CI environment, the default
home-directory path also tends to be lengthy for the same reason.

* Address this by sticking persistent-connection, background socket files
in '/tmp/cri-o' (avoid %d).  Also remove the username (%r) designation,
since this will almost always be the same user anyway.  The tradeoff
here is clashes between jobs against the same host (unlikely) and
weakened security on the control host (less important for CI jobs).

Signed-off-by: Chris Evich <cevich@redhat.com>
2017-11-03 15:53:16 -04:00
Chris Evich
51fed53139 Consolidate plays/tasks/tags
Simplify use of play-level tagging on three separate plays (with one
task).  Instead, make them all the same play, and apply the tags at
the task level instead.

Signed-off-by: Chris Evich <cevich@redhat.com>
2017-11-03 15:53:16 -04:00
Chris Evich
8964657140 Avoid relying on subject environment-vars
There are so many ways/places they can change values on the host from
one moment to the next.  Yet as written, the value of ansible_env.GOPATH
is really only fixed at "fact gathering" time.  In other words, the
environment variable can change (even during a play), but won't be
noticed until possibly much later.  This can cause very strange things
to happen which aren't easy to debug.

Fix this by using established facts (variables), and continuously
establishing them as environment variables.  This way, if/when a task
fails, the value of the environment will be present w/in the failure
message instead of obscrured by the shell.

Signed-off-by: Chris Evich <cevich@redhat.com>
2017-11-03 15:53:16 -04:00
Chris Evich
8e37304a96 Support testing against remote subjects.
It's a severe anti-pattern for a playbook to assume execution always
on a specific host.  The normal/expected pattern is to execute from a
"control host", against an inventory of (possibly-remote) subjects.

This doesn't preclude the inventory from only ever containing
'localhost', it simply means the plays and tasks should not assume
the inventory contents.

This concept is one of the central design-pillars of Ansible's,
and tantamount to it's usefulness and flexibility.  However, in
practice (and by ``integration/readme.md``), plays
specify ``- hosts: all`` but assume inventory_hostname == 'localhost'
(always).

Fix both the playbooks and ``readme.md`` to remove this anti-pattern,
while also allowing the control-host to be the subject-host as needed.
This is accomplished by ensuring low-level Ansible dependencies are
always installed, and writing tasks for steps previously performed
externally (in the CI/automation machinery).

Also update ``readme.md`` to recommend execution occurs through
the ``venv-ansible-playbook.sh`` wrapper to ensure consistent, stable,
version-locked execution dependencies on the control-host.

Remove ``remote_user: root`` from main, since this is better left
to the inventory and command-line.

Signed-off-by: Chris Evich <cevich@redhat.com>
2017-11-03 15:53:16 -04:00
Chris Evich
fe82f1b8aa Update all packages before installing new ones
Every now and again, a host will be in an initial state that prevents
installing new packages due to existing packages having some script or
obsoletes problem.  Avoid this by first updating all packages, then
installing new ones.

Signed-off-by: Chris Evich <cevich@redhat.com>
2017-11-03 15:48:39 -04:00
Chris Evich
ab949957fa Add ability to enable/disable SELinux during tests
Add a pair of variables to control whether or not SELinux
is enabled during particular tests.  In all cases, make
sure it's re-enabled afterwards.

Signed-off-by: Chris Evich <cevich@redhat.com>
2017-11-03 15:48:39 -04:00
Chris Evich
91f3c13dd1 Revert Revert "Idempotent Swapping"
This puts back the better qualified Idempotent Swapping, but
adds two variables which control whether or not swapping is
enabled during testing.  This addresses a short-term issue of
occasionally failing integration tests under some scenarios,
but not others.

The integration OOM-test isn't properly failing because
the cgroup memory control doesn't account for swap usage
(by design) in ``limit_in_bytes``.

Fixing this for the long-term requires repairing the
test to also set ``memory.memsw.limit_in_bytes=0``
(in addition to memory.limit_in_bytes=5m).

N/B: Normally these things are passed down from k8s,
which is why the same fix isn't currently needed
for the e2e tests - hence the new variable is ``True`` by
default.

Signed-off-by: Chris Evich <cevich@redhat.com>
2017-11-03 15:48:39 -04:00
Chris Evich
70b83fd641 Fix e2e test dependency on building k8s
These two should not be tightly-coupled.  e.g. for RPM testing, the
packages will be built/installed, then e2e tests will run.  Having k8s
secondarily built/installed will seriously screw with rpm-building and
testing.

Signed-off-by: Chris Evich <cevich@redhat.com>
2017-11-03 15:42:16 -04:00
Chris Evich
369aca6dcf Fix /test ami fail on duplicate k8s checkout
Signed-off-by: Chris Evich <cevich@redhat.com>
2017-11-03 15:42:16 -04:00
Antonio Murdaca
19d90e7c23
Merge pull request #1114 from mrunalp/env_fix_1.8
Add HOSTNAME env var to container
2017-11-03 16:49:46 +01:00
Mrunal Patel
70201fdf96 travis: Take out make lint for go tip
It is failing and our source can't be compatible with stable and tip
at the same time.

Signed-off-by: Mrunal Patel <mrunalp@gmail.com>
2017-11-02 20:25:34 -07:00
Mrunal Patel
c44c712a42 test: Add a test for HOSTNAME env
Signed-off-by: Mrunal Patel <mrunalp@gmail.com>
2017-11-02 20:25:07 -07:00
Mrunal Patel
fa1ad4f54e Add HOSTNAME env var to container
Signed-off-by: Mrunal Patel <mrunalp@gmail.com>
2017-11-02 10:24:56 -07:00
Mrunal Patel
6a43d07bae
Merge pull request #947 from runcom/release-1.8
bump to 1.8
2017-11-02 09:53:56 -07:00
Antonio Murdaca
4dce8e12a0
build k8s and cri-tools in place
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2017-11-02 16:12:47 +01:00
Mrunal Patel
4e2c6911ad
container: Don't add rprivate to all mounts
This fixes the mount propagation tests

Signed-off-by: Mrunal Patel <mpatel@redhat.com>
2017-11-02 16:07:52 +01:00
Mrunal Patel
815bb7652b
sandbox: Use first class sysctls instead of annotations
Signed-off-by: Mrunal Patel <mpatel@redhat.com>
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2017-11-02 16:07:52 +01:00
Antonio Murdaca
c25530ac0b
server: implement update container resources
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2017-11-02 16:07:52 +01:00
Antonio Murdaca
7d7024999b
sandbox, ctrs: fixup seccomp for 1.8
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2017-11-02 16:07:52 +01:00
Antonio Murdaca
c70198617f
container_create: set mount propagation
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2017-11-02 16:07:51 +01:00
Antonio Murdaca
e41ba62b19
container_create: honor no_new_privs
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2017-11-02 16:07:51 +01:00
Antonio Murdaca
91d9b4fc29
cmd: crio: correctly stop the streaming server
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2017-11-02 16:07:51 +01:00
Antonio Murdaca
d6e819133d
*: initial update to kube 1.8
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2017-11-02 16:07:51 +01:00
Daniel J Walsh
2453222695
Merge pull request #1108 from runcom/md2man-ubunut
README.md: add go-md2man to build dep in Ubuntu
2017-11-02 10:20:10 -04:00
Antonio Murdaca
ed9f4c094a
README.md: add go-md2man to build dep in Ubuntu
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2017-11-02 12:27:34 +01:00
Mrunal Patel
87192d0c8c
Merge pull request #1106 from runcom/add-dot-github
*: add .github
2017-11-01 14:23:16 -07:00
Antonio Murdaca
0478365d95
*: add .github
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2017-11-01 20:18:00 +01:00
Daniel J Walsh
fe69289566
Merge pull request #1080 from baude/kpod_create.pr
Kpod create.pr
2017-11-01 13:44:28 -04:00
Mrunal Patel
62b157c324
Merge pull request #1103 from spiffxp/assignees-to-approvers
Rename OWNERS assignees: to approvers:
2017-11-01 10:13:32 -07:00
baude
b85fe5ab90 Fix conmon and runc paths for kpod tests
Tests for kpod create and run were failing because the conmon
binary was being hardcoded.  We added a  --conmon global optioni
for kpod so we could pass in the conmon path from the helpers
file during tests

Signed-off-by: baude <bbaude@redhat.com>
2017-11-01 10:53:52 -05:00
Aaron Crickenberger
46742e1216 Rename OWNERS assignees: to approvers:
They are effectively the same, assignees is deprecated

Signed-off-by: Aaron Crickenberger <spiffxp@gmail.com>
2017-11-01 08:48:44 -07:00
Mrunal Patel
c23e8fc78f
Merge pull request #1101 from mrunalp/network_settings
test: Add networking settings for critest
2017-11-01 07:51:40 -07:00
Daniel J Walsh
37fff3cff3
Merge pull request #1104 from rhatdan/master
Remove VERSION file
2017-11-01 09:18:30 -04:00
Daniel J Walsh
699fc11edf Remove VERSION file
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2017-11-01 08:06:40 -04:00
Daniel J Walsh
dce6f3c2ef
Merge pull request #1100 from rhatdan/args
Report error when arguments given to crio command
2017-11-01 08:03:18 -04:00
Daniel J Walsh
063b25cef5 Report error when arguments given to crio command
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2017-10-31 16:15:54 -04:00
Mrunal Patel
87e7280f5a test: Add networking settings for critest
Signed-off-by: Mrunal Patel <mrunalp@gmail.com>
2017-10-31 13:14:50 -07:00
baude
c6cc205b78 Reviewer comments and suggestions incorporated.
Signed-off-by: baude <bbaude@redhat.com>
2017-10-31 14:10:47 -05:00
Mrunal Patel
4c06116c18
Merge pull request #654 from nalind/storage-update
Do not merge: Bump containers/storage and containers/image
2017-10-31 10:31:26 -07:00
baude
7f7ccc375f kpod create and run
Add the ability to run create a container with kpod.  Also, be able to run
(create and start) a container.  If the user asks for -it, be able to
attach a terminal to the container.

Signed-off-by: baude <bbaude@redhat.com>
2017-10-31 09:55:35 -05:00
baude
484a26d540 cmd/kpod/parse.go: Provided by Urvashi Mohnani
Signed-off-by: baude <bbaude@redhat.com>
2017-10-31 09:55:35 -05:00
Daniel J Walsh
409a228a73 Add kpod run and kpod create CLI front ends
Add kpod-run/kpod-create man page and command completions

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>

This patch implements the ability to create and run containers
using kpod

Signed-off-by: Matthew Heon <mheon@redhat.com>
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2017-10-31 09:55:35 -05:00
Nalin Dahyabhai
2e5e92730a Switch to ImageServer.UntagImage in RemoveImage handler
Add an UntagImage() method to pkg/storage/ImageServer, which will check
if the passed-in NameOrID is a name.  If so, it merely removes that name
from the image, removing the image only if it was the last name that the
image had.  If the NameOrID is an image ID, the image is removed, as
RemoveImage() does.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2017-10-31 10:30:18 -04:00
Nalin Dahyabhai
3f2bc09231 Return image references in ImageStatus()
The image's canonical reference is a name with a digest of the image's
manifest, so compute and return that value as the image's reference in
ImageStatus() and in ContainerStatus().

We don't auto-store a name based on the image digest when we pull one by
tag, but then CRI doesn't need us to do that.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2017-10-31 10:29:06 -04:00
Nalin Dahyabhai
beef44840e Update kpod push test
Since we've got stable image IDs now, pushing an image from local
storage to local storage ends up attaching the both names to a single
image, so we need to update the test's expectations.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2017-10-31 10:29:05 -04:00
Nalin Dahyabhai
9dab0eee24 Remove dead code
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2017-10-31 10:29:05 -04:00
Nalin Dahyabhai
88deb3934f Update PushImage() for newer containers/image
The updated containers/image and containers/storage don't require as
many workarounds to be able to push images.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2017-10-31 10:29:05 -04:00
Nalin Dahyabhai
1346755565 Bump containers/image and containers/storage
Update to proposed changes in containers/image, and bump
containers/storage to 04ad0b827097209ca65e59b5fd768511f3b1ae91, which is
currently the tip of the master branch.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2017-10-31 10:29:05 -04:00
Daniel J Walsh
c5e73ba65f
Merge pull request #1085 from jasonbrooks/patch-1
add golang-github-cpuguy83-go-md2man dep
2017-10-31 10:27:19 -04:00
Daniel J Walsh
051cafbd62
Merge pull request #1095 from TomSweeneyRedHat/dev/tsweeney/docfix/12
Change runc location of runc in tutorial.md
2017-10-31 10:26:37 -04:00