Commit graph

87 commits

Author SHA1 Message Date
Joseph Schorr
ec42303750 image_and_tag must be before we use it 2014-10-03 13:00:41 -04:00
Joseph Schorr
9c88ca16b5 Add the docker version to the build logs 2014-09-23 14:45:22 -04:00
Jake Moshenko
b55e79e5d3 Merge remote-tracking branch 'origin/master' into waltermitty 2014-09-12 11:45:07 -04:00
Jake Moshenko
5388633f9a Merge remote-tracking branch 'origin/master' into pullfail 2014-09-12 10:36:38 -04:00
Jake Moshenko
2455c17f96 Merge remote-tracking branch 'origin/master' into waltermitty
Conflicts:
	app.py
	data/userfiles.py
2014-09-11 11:18:28 -04:00
Jake Moshenko
29d40db5ea Add a new RadosGW storage engine. Allow engines to distinguish not only between those that can support direct uploads and downloads, but those that support doing it through the browser. Rename resumeable->resumable. 2014-09-09 15:54:03 -04:00
Jake Moshenko
451e034ca1 Archived logs commit 1. Squash me. 2014-09-08 16:43:17 -04:00
Joseph Schorr
07aab4274c Fix parameters for logging the extra data needed 2014-08-28 19:19:20 -04:00
Joseph Schorr
5744f0f888 Make the dockerfilebuild error checking less harsh 2014-08-28 16:07:56 -04:00
Joseph Schorr
463a3c55c3 Make worker error messages more descriptive 2014-08-27 19:02:53 -04:00
Joseph Schorr
510bbe7889 Add more check conditions for unhealthy workers and make the messaging better. 2014-08-26 12:41:43 -04:00
Joseph Schorr
daa43c3bb9 Add better messaging around pulling of base images when they fail due to invalid or missing credentials 2014-08-18 20:34:39 -04:00
Joseph Schorr
736af3165b Add a default message if the build pack MIME processor fails 2014-08-15 18:23:43 -04:00
Joseph Schorr
8681dd9cb9 Add a new exposed 'unpacking' phase to the build and make sure that the unzip/untar/etc always occurs under a try-except 2014-08-15 17:58:11 -04:00
Joseph Schorr
1b7379df29 Fix workers to not always be marked as unhealthy 2014-08-08 15:24:19 -04:00
Jake Moshenko
0372013f70 Merge remote-tracking branch 'origin/redalert'
Conflicts:
	app.py
2014-08-04 16:56:34 -04:00
Joseph Schorr
7e935f5a8c Make build workers report that they are unhealthy when we get an LXC error or a Docker connection issue 2014-07-30 17:54:58 -04:00
Joseph Schorr
752efb9e0f Fix the spawn_notification to work in all cases and clean up some of the remaining code 2014-07-18 16:34:52 -04:00
Joseph Schorr
591cd020b8 Merge branch 'master' into redalert 2014-07-18 15:58:56 -04:00
Joseph Schorr
af31bde997 Add support for the remaining events to the frontend and the backend 2014-07-18 15:58:18 -04:00
Jake Moshenko
cceb09d4f6 Remove some unused dependencies and update the rest. 2014-07-17 12:08:07 -04:00
Joseph Schorr
8b3659fefa Dockerfile build worker should not report inner JobException's twice 2014-07-11 12:05:52 -04:00
Joseph Schorr
9d1ae8ba87 FROM line check needs to be on the tuple result, not the join 2014-06-16 14:01:17 -04:00
Joseph Schorr
f795868b5b Handle the case where there is no FROM command in the Dockerfile 2014-06-13 16:56:48 -04:00
Jake Moshenko
0b6552d6cc Fix the metrics so they are usable for scaling the workers down and up. Switch all datetimes which touch the database from now to utcnow. Fix the worker Dockerfile. 2014-05-23 14:16:26 -04:00
Jake Moshenko
d14798de1d Add a queue capacity reporter plugin to the queue. Move the queue definitions to app. Add a cloudwatch reporter to the dockerfile build queue. 2014-05-21 19:50:37 -04:00
Jake Moshenko
b8466169ac Integrate sentry with the build worker. 2014-05-19 13:50:45 -04:00
Jake Moshenko
212a4650f4 Rework the config to use runit logging. 2014-05-18 17:19:14 -04:00
Jake Moshenko
cc47e77156 Upgrade to the 0.11.1 tutum version of docker. Package it as a Dockerfile using Docker in Docker. Add a status server option to the workers to utilize the new termination signal and status features of gantry. 2014-05-16 18:31:24 -04:00
Jake Moshenko
bcb993a914 Set up the build logs to use our fake build logs on test and local. 2014-05-09 18:45:11 -04:00
Jake Moshenko
8a3af93b8c Improve the builder response to being terminated or dying. 2014-05-06 18:46:19 -04:00
Jake Moshenko
450928674b Use a new caching algorithm which can limit the size for the build nodes. Stop treating public images as special. Add a new phase to the builder for pulling. 2014-04-30 18:48:36 -04:00
jakedt
2bc3d24543 Update the build worker to remove all tags from expired images. 2014-04-18 18:36:11 -04:00
jakedt
0a9ee6c49f Bust the dockerfile build cache across repository lines. 2014-04-16 15:45:41 -04:00
jakedt
0827e0fbac Merge remote-tracking branch 'origin/master' into ncc1701
Conflicts:
	endpoints/web.py
	static/directives/signup-form.html
	static/js/app.js
	static/js/controllers.js
	static/partials/landing.html
	static/partials/view-repo.html
	test/data/test.db
2014-04-14 19:37:22 -04:00
jakedt
724fec1b74 Test third party repo images for public-ness in the builder. Always clean up private images that we dont know about before build. Pull the base image to refresh before every build. 2014-04-14 18:54:39 -04:00
jakedt
40f82a9d16 Work harder to reset the state of the docker env on the build worker. 2014-04-14 15:59:57 -04:00
jakedt
de18236358 Allow for caching of previous docker builds for 24 hours. 2014-04-14 15:21:05 -04:00
jakedt
61a6db236f Finish the implementation of local userfiles. Strip charsets from mimetypes in the build worker. Add canonical name ordering to the build queue. Port all queues to the canonical naming version. 2014-04-11 18:34:47 -04:00
jakedt
576fbe4f0d Switch over to phusion baseimage. Prevent everything from daemonizing and start it with runit under phusion. Make workers trap and handle sigint and sigterm. Extend the reservation to 1hr for dockerfilebuild. Update nginx to remove the dependency on libgd. Merge the requirements and requirements enterprise files. 2014-04-11 13:32:45 -04:00
jakedt
8076ad0a20 Add x-gzip as a mimetype for tarballs. 2014-04-07 10:58:53 -04:00
jakedt
37ad8a414a Revert a change to the way zip files are extracted. 2014-04-04 16:06:41 -04:00
jakedt
e87ffa20cf First attempt at making config loadable through string config overrides in an env variable. 2014-04-03 17:31:46 -04:00
jakedt
b95d3ec329 Add a watchdog timer to the build worker to kill a build step that takes more than 20 minutes. 2014-04-02 19:32:41 -04:00
Joseph Schorr
9a79d1562a Change to store the pull robot on the repository build and only add the credentials to the queue item. This prevents the credentials from being exposed to the end user. Also fixes the restart build option 2014-04-01 21:49:06 -04:00
jakedt
ca1970a2f4 Use the real registry endpoint in the login command. 2014-04-01 19:00:11 -04:00
jakedt
3525e383df Merge remote-tracking branch 'origin/master' into pullinprivate
Conflicts:
	test/data/test.db
2014-04-01 18:28:48 -04:00
jakedt
7c44932c87 Use safer tar extraction. Handle error messages in the build process more intelligently. 2014-04-01 13:46:41 -04:00
jakedt
d67a1cddc2 Merge remote-tracking branch 'origin/master' into pullinprivate
Conflicts:
	workers/dockerfilebuild.py
2014-03-31 18:10:34 -04:00
jakedt
e7c20e1052 Add tarball support to the builder and pull github code as a tarball. 2014-03-31 15:40:24 -04:00