Joseph Schorr
bc670611ef
Increase the timeout on the atomic lock
...
Some nodes were still performing the action twice when falling outside of the 30s window
2016-08-23 12:50:38 -04:00
Joseph Schorr
3112388004
Fix multiple reporting of incomplete
2016-08-17 16:01:28 -04:00
Joseph Schorr
0b50928900
Fix build start check for the ephemeral case
2016-08-16 17:18:57 -04:00
Joseph Schorr
433b157531
Add extra check to ensure a build cannot be started without on_ready called
2016-08-16 16:38:48 -04:00
Joseph Schorr
5e1a117ff3
Delete the job first to prevent Kubernetes from starting another pod
2016-08-16 16:33:43 -04:00
Joseph Schorr
742e153133
Fix watch of the jobs key in the build manager
2016-08-16 15:43:09 -04:00
Joseph Schorr
313d65a6a4
Make sure the etcd watch coroutines get called
2016-08-16 13:02:27 -04:00
josephschorr
cddba20ffe
Merge pull request #1731 from coreos-inc/k8s-cleanup
...
Cleanup old executions that never start
2016-08-15 17:00:13 -04:00
Joseph Schorr
d78361b041
Cleanup old executions that never start
...
Fixes #1727
2016-08-15 16:54:02 -04:00
Brad Ison
d37f32b9c7
Add bison's SSH key to builders
2016-08-15 15:53:26 -04:00
Joseph Schorr
acdfc9369d
Allow the version of CoreOS to be specified when building QEMU image
2016-08-05 16:46:11 -04:00
Joseph Schorr
c29f9ccc7f
Fix TTL on heartbeat in etcd
...
Until now, once the heartbeat has expired, we would issue a TTL that is negative, which causes etcd to either raise an exception or simply ignore the expiration (depending on the version of etcd). This change ensures that once the key is expired, it is removed immediately via a set of a TTL of 0. Also adds tests for this case and the normal expiration case.
2016-08-03 11:15:03 -04:00
Joseph Schorr
428a7cb435
Fix decreased setup timeout on ephemeral build manager
2016-07-22 13:35:38 -04:00
Joseph Schorr
392242d20b
Another fix for the record keeping in buildman
...
Adds some more mocked tests as well
2016-07-22 12:01:30 -04:00
Joseph Schorr
68baa51d55
Fix cross-manager handling of realm components
2016-07-21 15:47:25 -04:00
Joseph Schorr
4420b1bac9
Add temporary back-compat shims for the build manager
2016-07-20 13:41:01 -04:00
Joseph Schorr
2c1880b944
Bug fixes, refactoring and "new" tests for the build manager
...
- Fixes various bugs introduced in the most recent build system commit
- Refactors state management in the build manager to be cleaner and more contained
- Adds back in the mock-based tests, fixed to not use threads and adjusted for the refactoring
- Adds some more simplified unit tests around non-etch related flows
2016-07-18 13:46:48 -04:00
Joseph Schorr
74b87fa813
Build manager cleanup and more logging
2016-07-14 14:33:14 -04:00
Joseph Schorr
d8b72e8503
Switch to using a defined branch and not always pulling the VM image
2016-07-08 17:53:25 -04:00
Joseph Schorr
3d4af78f01
Fix label to never allow a space (which breaks Kubernetes)
2016-07-08 17:09:06 -04:00
Joseph Schorr
811413fe9c
Add multiple executor and whitelist support to build manager
2016-07-08 15:50:51 -04:00
Joseph Schorr
7471d0e35f
Small code cleanup before whitelist addition
2016-07-08 15:50:51 -04:00
Colin Hom
1e3351f3f4
local-docker.sh now accepts env vars
2016-07-08 15:50:51 -04:00
Colin Hom
bc13333f20
Kubernetes build worker
2016-07-08 15:50:51 -04:00
Joseph Schorr
713ba3abaf
Further updates to the Prometheus client code
2016-07-01 14:16:51 -04:00
Matt Jibson
3d9acf2fff
Use prometheus as a metric backend
...
This entails writing a metric aggregation program since each worker has its
own memory, and thus own metrics because of python gunicorn. The python
client is a simple wrapper that makes web requests to it.
2016-07-01 14:16:50 -04:00
Joseph Schorr
1173192739
Move channel back, as it is referenced by generate_cloud_config
2016-06-22 17:25:06 -04:00
Joseph Schorr
61695eb439
Allow the build node AMI to be overridden in config
2016-06-22 15:13:54 -04:00
josephschorr
20a6fdc73f
Merge pull request #1557 from jzelinskie/buildargs
...
buildman: mark missing buildargs as failure
2016-06-20 14:40:17 -04:00
Jimmy Zelinskie
871c1634ed
buildman: mark missing buildargs as failure
2016-06-17 18:33:54 -04:00
Joseph Schorr
7292524d69
Add a cloud watch metric when we fail to start a build via EC2
...
Fixes #1555
2016-06-17 16:19:57 -04:00
Jimmy Zelinskie
5298452fa7
builder cloudconfig: shutdown server after 3 hours ( #1554 )
2016-06-17 16:03:40 -04:00
Joseph Schorr
f9469a84b3
Make the size of the build node HDD configurable
...
Fixes #1520
2016-06-06 11:35:10 -04:00
Jimmy Zelinskie
7d356c451b
buildman: fix misspell
2016-06-03 15:42:14 -04:00
Jimmy Zelinskie
44b56ae2cf
queue: explicitly declare ordering requirement
...
This change defaults the ordering requirement of queue items to be off
and only enables it for the build manager. This should make the queries
for getting queueitems significantly faster for every other use case.
2016-05-27 14:44:30 -04:00
Jimmy Zelinskie
79aa78906a
buildman: refresh and add Evan's key to builders
2016-05-24 14:05:39 -04:00
Joseph Schorr
5262535945
Boto error_code is a string, not the HTTP status code
2015-12-23 15:12:01 -05:00
Jimmy Zelinskie
601b99a083
buildman: add git checkout failure
2015-12-16 14:49:37 -05:00
Joseph Schorr
773e73861f
Change error into info in build manager
...
Fixes #1046
2015-12-09 14:30:14 -05:00
josephschorr
c06e5cc9c7
Merge pull request #1002 from coreos-inc/buildertagexc
...
Add timeout and failure if an EC2 instance could not be found when ta…
2015-12-09 14:28:31 -05:00
Joseph Schorr
946e5fabc0
Add timeout and failure if an EC2 instance could not be found when tagging
...
Fixes #994
2015-12-09 14:28:19 -05:00
Joseph Schorr
edd9a03af5
Catch additional key not found exception
...
Fixes #806
2015-12-01 12:29:58 -05:00
Joseph Schorr
fbc4927544
Change to only exception logging internal errors on builds
...
Fixes #993
2015-11-30 14:30:55 -05:00
Jake Moshenko
c4b637521c
Remove Matt Jibson's public key
2015-11-23 18:18:42 -05:00
Matt Jibson
2325328bbd
Update mjibson ssh key
2015-11-06 15:34:52 -05:00
Jimmy Zelinskie
e973289397
Revert "Revert "Merge pull request #682 from jzelinskie/revertrevert""
...
This reverts commit 278bc736e3
.
2015-10-23 15:26:33 -04:00
Jimmy Zelinskie
278bc736e3
Revert "Merge pull request #682 from jzelinskie/revertrevert"
...
This reverts commit 627ad25c9c
, reversing
changes made to 31c392fecc
.
2015-10-22 16:02:07 -04:00
Jimmy Zelinskie
46b2f10d7f
check for VPC subnet ID before using builder VPC
...
This means you can use legacy networking machines by simply changing the
instance type and removing the specified 'EC2_VPC_SUBNET_ID' from the
executor config.
2015-10-22 14:50:54 -04:00
Jimmy Zelinskie
39cfe77d42
Revert "Merge pull request #557 from coreos-inc/revert-migration"
...
This reverts commit c4f938898a
, reversing
changes made to 7ad2522dbe
.
2015-10-21 15:29:57 -04:00
Joseph Schorr
0f37e66cc8
Better error handling for the build manager
...
Fixes #604
2015-10-13 11:40:07 -04:00