Commit graph

62 commits

Author SHA1 Message Date
Joseph Schorr
cd1b4042ae Move SSH keys and other config hard-coded config out of the build worker template
In addition to removing our public keys from git, this will also allow additional customization

Fixes https://jira.coreos.com/browse/QUAY-1035
2018-09-05 17:36:01 -04:00
Antoine Legrand
cdb3722c17 Use $QUAYPATH and $QUAYDIR in conf and init files 2017-07-05 16:23:54 +02:00
Charlton Austin
e30cd931d1 feat(buildtrigger): allow use to specify dockerfile
users can only specify the folder and the
  dockerfile must be names "Dockerfile" this allows
  users to specify the file and it can be called
  "Dockerfile" or <some name>.Dockerfile
2017-03-06 21:20:17 -05:00
Joseph Schorr
b407f88a26 Remove unnecessary CloudWatch metrics
They are spamming the API and costing us a lot of money
2017-02-01 13:08:21 -05:00
Evan Cordell
dd5f7cbe6c Fix the ephemeral build metrics 2016-12-13 18:28:04 -05:00
Brad Ison
2c59bd9ee5 Set builder hostnames to build UUID 2016-11-15 12:35:48 -08:00
Charlton Austin
83e8d62bea Merge pull request #2085 from charltonaustin/move_ephemeral_binary
Moving the binary location.
2016-11-08 11:42:31 -05:00
Brad Ison
a8c0376c06 Set imagePullPolicy to IfNotPresent for k8s builder 2016-11-07 17:20:40 -05:00
Joseph Schorr
ef41e57aad Add executor-specific setup time support
This will allow us to make the setup time TTL for k8s-based builds much lower (on the order of a minute), which means faster timeouts and fallbacks (which is a better user experience).
2016-11-07 15:45:15 -05:00
Charlton Austin
7a2dca9c53 Moving the binary location. 2016-11-04 15:53:43 -04:00
Charlton Austin
bba51787b5 Adding in a new location for the default popen executor. 2016-11-04 14:22:26 -04:00
Brad Ison
779f0f1b54 Add emptyDir volume to builder pods to mask secrets
This adds a empty volume on a tmpfs to builder pods and mounts it over
the directory Kubernetes uses for secrets, which should prevent pods
from having access to the default service account.
2016-10-05 14:27:07 -04:00
Brad Ison
087dca3482 Only set memory request on Kubernetes builds
This removes the aboslute limits on Kubernetes builds for now (KVM
will still limit resources) and only sets the memory request as a hint
to the scheduler.
2016-10-04 20:42:51 -04:00
Brad Ison
febf3751c0 Merge pull request #1937 from coreos-inc/k8s-resource-limits
Fix kubernetes resource limits
2016-10-04 14:11:46 -04:00
Brad Ison
cee7c4be96 Fix kubernetes resource limits 2016-10-04 11:56:06 -04:00
Brad Ison
d8aa22103e Add a dash to generated k8s job names 2016-10-01 14:02:28 -04:00
josephschorr
0c2b4ed9c1 Merge pull request #1897 from coreos-inc/hash-executor-whitelist
Add hash-based staged rollout to build executors
2016-09-30 17:52:19 +02:00
Joseph Schorr
51a519f653 Add hash-based staged rollout to build executors
Fixes #1882
2016-09-29 22:48:42 +02:00
Evan Cordell
832ee89923 Add duration metric collector decorator (#1885)
Track time-to-start for builders
Track time-to-build for builders
Track ec2 builder fallbacks
Track build time
2016-09-29 15:44:06 -04:00
Brad Ison
593c3eb9c7 Set dnsPolicy to Default on k8s build jobs
This prevents the builder pods from having resolv.conf pointed at the
kube-dns service, which they won't have access to.
2016-09-29 11:22:11 -04:00
Brad Ison
631ad0422d Default to 4GB memory for k8s builders 2016-09-29 11:20:49 -04:00
Brad Ison
2a1cf2bfd1 Always pull latest image in k8s builds 2016-09-08 15:00:12 -04:00
Joseph Schorr
cd2d0341a7 Fix k8s builder to use the declared volume size
Fixes #1773
2016-08-29 15:16:28 -04:00
Joseph Schorr
5e1a117ff3 Delete the job first to prevent Kubernetes from starting another pod 2016-08-16 16:33:43 -04:00
Joseph Schorr
2c1880b944 Bug fixes, refactoring and "new" tests for the build manager
- Fixes various bugs introduced in the most recent build system commit
- Refactors state management in the build manager to be cleaner and more contained
- Adds back in the mock-based tests, fixed to not use threads and adjusted for the refactoring
- Adds some more simplified unit tests around non-etch related flows
2016-07-18 13:46:48 -04:00
Joseph Schorr
74b87fa813 Build manager cleanup and more logging 2016-07-14 14:33:14 -04:00
Joseph Schorr
d8b72e8503 Switch to using a defined branch and not always pulling the VM image 2016-07-08 17:53:25 -04:00
Joseph Schorr
3d4af78f01 Fix label to never allow a space (which breaks Kubernetes) 2016-07-08 17:09:06 -04:00
Joseph Schorr
811413fe9c Add multiple executor and whitelist support to build manager 2016-07-08 15:50:51 -04:00
Joseph Schorr
7471d0e35f Small code cleanup before whitelist addition 2016-07-08 15:50:51 -04:00
Colin Hom
1e3351f3f4 local-docker.sh now accepts env vars 2016-07-08 15:50:51 -04:00
Colin Hom
bc13333f20 Kubernetes build worker 2016-07-08 15:50:51 -04:00
Joseph Schorr
713ba3abaf Further updates to the Prometheus client code 2016-07-01 14:16:51 -04:00
Joseph Schorr
1173192739 Move channel back, as it is referenced by generate_cloud_config 2016-06-22 17:25:06 -04:00
Joseph Schorr
61695eb439 Allow the build node AMI to be overridden in config 2016-06-22 15:13:54 -04:00
Joseph Schorr
7292524d69 Add a cloud watch metric when we fail to start a build via EC2
Fixes #1555
2016-06-17 16:19:57 -04:00
Joseph Schorr
f9469a84b3 Make the size of the build node HDD configurable
Fixes #1520
2016-06-06 11:35:10 -04:00
Joseph Schorr
5262535945 Boto error_code is a string, not the HTTP status code 2015-12-23 15:12:01 -05:00
Joseph Schorr
946e5fabc0 Add timeout and failure if an EC2 instance could not be found when tagging
Fixes #994
2015-12-09 14:28:19 -05:00
Jimmy Zelinskie
46b2f10d7f check for VPC subnet ID before using builder VPC
This means you can use legacy networking machines by simply changing the
instance type and removing the specified 'EC2_VPC_SUBNET_ID' from the
executor config.
2015-10-22 14:50:54 -04:00
Joseph Schorr
0f37e66cc8 Better error handling for the build manager
Fixes #604
2015-10-13 11:40:07 -04:00
Joseph Schorr
04cc471585 Increase the HD size on the build nodes
Fixes #228
2015-07-14 15:20:17 +03:00
Jake Moshenko
6eead7c860 Add logentries reporting to the ephemeral builders. 2015-03-27 15:28:08 -04:00
Joseph Schorr
2eaec092f0 Handle the case where we cannot write the tags on the build nodes 2015-02-25 13:47:36 -05:00
Joseph Schorr
98b4f62ef7 Switch to using a squashed image for the build workers 2015-02-10 15:43:01 -05:00
Jake Moshenko
5b8d65991e Update the space on the builder nodes because its cheap. 2015-02-04 11:58:58 -05:00
Joseph Schorr
361fb33574 - Add a small build script
- Take in the build worker branch name from config
- Add additional logging (to be removed after we figure out the problem)
2015-02-03 12:48:41 -05:00
Jake Moshenko
2215ec6669 Associate a public IP with the network interfaces on our VPC instances. 2015-02-02 15:28:40 -05:00
Jake Moshenko
db8493f254 update the executor template to use VPC instances. 2015-02-02 14:55:34 -05:00
Jake Moshenko
f2471a86f6 Fix the python requirements. Add the ability to map in etcd client certs and ca. 2015-01-22 10:53:23 -05:00