Commit graph

62 commits

Author SHA1 Message Date
Jake Moshenko
5935e93eb8 Linter fixes. 2016-11-18 15:56:08 -05:00
Charlton Austin
ddcea4969f Adding in more worker errors. 2016-11-03 15:03:17 -04:00
Charlton Austin
1cde22e76c Making some refactors to make it easier to cancel the build at any time. 2016-10-24 15:59:33 -04:00
Jimmy Zelinskie
9b67fff78f buildman: spawn_notification w/ attrdict 2016-09-30 18:54:09 -04:00
Joseph Schorr
f50bb8a1ce Add missing call to set_phase when a build doesn't start
This change fixes the build manager ephemeral executor to tell the overall build server to call set_phase when a build never starts. Before this change, we'd properly adjust the queue item, but not the repo build row or the logs, which is why users just saw "Preparing Build Node", with no indicating the node failed to start.

Fixes #1904
2016-09-30 14:54:49 +02:00
Joseph Schorr
02b8afe127 Add labeling of built manifests with their build IDs
Also sends the digests to the notification

Fixes #593
2016-09-29 10:58:45 +02:00
Joseph Schorr
818ea38dac Add repo-specific reporting of repository builds 2016-09-09 15:36:54 -04:00
Joseph Schorr
2c1880b944 Bug fixes, refactoring and "new" tests for the build manager
- Fixes various bugs introduced in the most recent build system commit
- Refactors state management in the build manager to be cleaner and more contained
- Adds back in the mock-based tests, fixed to not use threads and adjusted for the refactoring
- Adds some more simplified unit tests around non-etch related flows
2016-07-18 13:46:48 -04:00
Joseph Schorr
811413fe9c Add multiple executor and whitelist support to build manager 2016-07-08 15:50:51 -04:00
Jimmy Zelinskie
601b99a083 buildman: add git checkout failure 2015-12-16 14:49:37 -05:00
Jimmy Zelinskie
e973289397 Revert "Revert "Merge pull request #682 from jzelinskie/revertrevert""
This reverts commit 278bc736e3.
2015-10-23 15:26:33 -04:00
Jimmy Zelinskie
278bc736e3 Revert "Merge pull request #682 from jzelinskie/revertrevert"
This reverts commit 627ad25c9c, reversing
changes made to 31c392fecc.
2015-10-22 16:02:07 -04:00
Jimmy Zelinskie
39cfe77d42 Revert "Merge pull request #557 from coreos-inc/revert-migration"
This reverts commit c4f938898a, reversing
changes made to 7ad2522dbe.
2015-10-21 15:29:57 -04:00
Joseph Schorr
2d3092b826 Make build system resistant to Redis being broken
Fixes #549
2015-09-30 15:15:10 -04:00
Silas Sewell
9000169b53 Revert "Merge pull request #491 from jakedt/migratebackp2"
This reverts commit 7ad2522dbe, reversing
changes made to a0b191ffa1.
2015-09-28 16:09:22 -04:00
Jake Moshenko
8baacd2741 Migrate old data to new locations, read only new. 2015-09-17 15:47:13 -04:00
Jimmy Zelinskie
239f76d39f Merge pull request #368 from coreos-inc/buildarchive
Allow builds to be started with an external archive URL
2015-08-17 17:09:14 -04:00
Joseph Schorr
f092c00621 Allow builds to be started with an external archive URL
Fixes #114
2015-08-17 17:01:49 -04:00
Matt Jibson
cfb6e884f2 Refactor metric collection
This change adds a generic queue onto which metrics can be pushed. A
separate module removes metrics from the queue and adds them to Cloudwatch.
Since these are now separate ideas, we can easily change the consumer from
Cloudwatch to anything else.

This change maintains near feature parity (the only change is there is now
just one queue instead of two - not a big deal).
2015-08-12 12:15:52 -04:00
Jake Moshenko
18100be481 Refactor the util directory to use subpackages. 2015-08-03 16:04:19 -04:00
Jake Moshenko
3efaa255e8 Accidental refactor, split out legacy.py into separate sumodules and update all call sites. 2015-07-17 11:56:15 -04:00
Joseph Schorr
9b974f6b80 Add more build information to the events and have better messaging
Fixes #79
2015-06-16 23:16:36 -04:00
Joseph Schorr
598fc6ec46 Add the error code to the worker error logged to redis 2015-05-18 15:01:48 -04:00
Jimmy Zelinskie
9f31bdd571 buildman: add new io.quay.builder.gitfailure error 2015-05-11 15:25:22 -04:00
Jimmy Zelinskie
15fdae6688 buildman: show base error for buildpack failures
Whereas before these were reserved only for S3 errors, users need these
specifics to debug custom-git configurations.
2015-05-11 14:18:48 -04:00
Joseph Schorr
31260d50f5 Rename the new images method to a slightly better name 2015-04-24 16:37:37 -04:00
Joseph Schorr
e70343d849 Faster cache lookup by removing a join with the ImagePlacementTable, removing the extra loop to add the locations and filtering the images looked up by the base image 2015-04-24 16:22:19 -04:00
Jimmy Zelinskie
db79ad2dde unused import 2015-02-25 15:26:36 -05:00
Joseph Schorr
b7901d2adb Add trigger metadata (which includes the SHA) and the built image_id to the event data 2015-02-24 15:13:51 -05:00
Jimmy Zelinskie
9ab3554226 buildreporter: does not execute in a coroutine! 2015-02-18 17:11:45 -05:00
Jimmy Zelinskie
0d38e0b00b metrics: use config['name'] to get metric conf 2015-02-18 16:05:36 -05:00
Jimmy Zelinskie
f53dea46b7 buildman: address PR #11 comments 2015-02-18 14:13:36 -05:00
Jimmy Zelinskie
1a71925125 buildreporter: remove unused logging 2015-02-17 17:02:37 -05:00
Jimmy Zelinskie
25fc999d50 buildreporter: handle app=None 2015-02-17 15:30:09 -05:00
Jimmy Zelinskie
ca0d2b1721 buildreporter: getattr method 2015-02-17 12:21:22 -05:00
Jimmy Zelinskie
0a00453024 buildreporter: rm pylint comments 2015-02-17 12:20:46 -05:00
Jimmy Zelinskie
0e7418ffce buildman: add BuildMetrics and BuildReporter 2015-02-17 10:56:09 -05:00
Joseph Schorr
f84d1bad45 Handle internal errors in a better fashion: If a build would be marked as internal error, only do so if there are retries remaining. Otherwise, we mark it as failed (since it won't be rebuilt anyway) 2015-02-12 16:19:44 -05:00
Joseph Schorr
e1a15464a1 Fix typo, add some logging and fix command comparison 2015-02-11 16:02:36 -05:00
Joseph Schorr
893ae46dec Add an ImageTree class and change to searching *all applicable* branches when looking for the best cache tag. 2015-02-10 21:46:58 -05:00
Joseph Schorr
9f1ec9d47d Fix loading of partial caching under a tag 2015-02-09 16:29:15 -05:00
Joseph Schorr
b0e315c332 Fix issues in cache comment comparison 2015-02-09 15:48:36 -05:00
Joseph Schorr
9b0e43514b Fix typos 2015-02-09 14:53:18 -05:00
Joseph Schorr
6cb1212da6 Add logging 2015-02-09 13:54:14 -05:00
Joseph Schorr
4310f47dee Some code cleanup in the cached tag determination code 2015-02-09 12:16:43 -05:00
Joseph Schorr
0065ac8503 Add back in the cache checking code and remove the old 0.1 build pack code 2015-02-09 12:13:40 -05:00
Joseph Schorr
48949627e0 Merge master in delta 2015-02-09 12:07:43 -05:00
Joseph Schorr
9ffb53cd47 Add support for v2 of the build worker, which performs the Dockerfile parsing on its own. Note that this version is backwards compatible with v1-beta of the build worker, so it should be pushed first. Also note that this version is temporary until such time as we get the caching branches merged. 2015-02-03 21:05:18 -05:00
Joseph Schorr
a1938593a9 Better handling of retries on build errors 2015-02-03 16:29:47 -05:00
Joseph Schorr
d709e0f64a Fix the new notifications code to work 2015-02-03 13:08:38 -05:00