Commit graph

72 commits

Author SHA1 Message Date
jakedt
2bc3d24543 Update the build worker to remove all tags from expired images. 2014-04-18 18:36:11 -04:00
jakedt
0d8725e778 Update the instructions for starting and running the workers. 2014-04-17 16:18:53 -04:00
jakedt
0a9ee6c49f Bust the dockerfile build cache across repository lines. 2014-04-16 15:45:41 -04:00
jakedt
0827e0fbac Merge remote-tracking branch 'origin/master' into ncc1701
Conflicts:
	endpoints/web.py
	static/directives/signup-form.html
	static/js/app.js
	static/js/controllers.js
	static/partials/landing.html
	static/partials/view-repo.html
	test/data/test.db
2014-04-14 19:37:22 -04:00
jakedt
724fec1b74 Test third party repo images for public-ness in the builder. Always clean up private images that we dont know about before build. Pull the base image to refresh before every build. 2014-04-14 18:54:39 -04:00
jakedt
40f82a9d16 Work harder to reset the state of the docker env on the build worker. 2014-04-14 15:59:57 -04:00
jakedt
de18236358 Allow for caching of previous docker builds for 24 hours. 2014-04-14 15:21:05 -04:00
jakedt
61a6db236f Finish the implementation of local userfiles. Strip charsets from mimetypes in the build worker. Add canonical name ordering to the build queue. Port all queues to the canonical naming version. 2014-04-11 18:34:47 -04:00
jakedt
576fbe4f0d Switch over to phusion baseimage. Prevent everything from daemonizing and start it with runit under phusion. Make workers trap and handle sigint and sigterm. Extend the reservation to 1hr for dockerfilebuild. Update nginx to remove the dependency on libgd. Merge the requirements and requirements enterprise files. 2014-04-11 13:32:45 -04:00
jakedt
8fac0474b5 Get staging to run under docker on an EC2 host. 2014-04-10 18:30:09 -04:00
jakedt
8076ad0a20 Add x-gzip as a mimetype for tarballs. 2014-04-07 10:58:53 -04:00
jakedt
37ad8a414a Revert a change to the way zip files are extracted. 2014-04-04 16:06:41 -04:00
jakedt
e87ffa20cf First attempt at making config loadable through string config overrides in an env variable. 2014-04-03 17:31:46 -04:00
jakedt
b95d3ec329 Add a watchdog timer to the build worker to kill a build step that takes more than 20 minutes. 2014-04-02 19:32:41 -04:00
Joseph Schorr
9a79d1562a Change to store the pull robot on the repository build and only add the credentials to the queue item. This prevents the credentials from being exposed to the end user. Also fixes the restart build option 2014-04-01 21:49:06 -04:00
jakedt
ca1970a2f4 Use the real registry endpoint in the login command. 2014-04-01 19:00:11 -04:00
jakedt
3525e383df Merge remote-tracking branch 'origin/master' into pullinprivate
Conflicts:
	test/data/test.db
2014-04-01 18:28:48 -04:00
jakedt
7c44932c87 Use safer tar extraction. Handle error messages in the build process more intelligently. 2014-04-01 13:46:41 -04:00
jakedt
d67a1cddc2 Merge remote-tracking branch 'origin/master' into pullinprivate
Conflicts:
	workers/dockerfilebuild.py
2014-03-31 18:10:34 -04:00
jakedt
e7c20e1052 Add tarball support to the builder and pull github code as a tarball. 2014-03-31 15:40:24 -04:00
jakedt
7c14190d2a Freeze the version of docker-py to 0.3.0 and create a fancy streaming json decoder to work around the lack of newlines in push statuses. 2014-03-28 17:53:33 -04:00
Joseph Schorr
2006917e03 Add support for pull credentials on builds and build triggers 2014-03-27 18:33:13 -04:00
Jake Moshenko
40e81f478f If JSON decoding of a stream response fails, just ignore it since it's probably a blank line. 2014-03-27 22:15:59 +00:00
Jake Moshenko
2767c7e1f7 Change the docker version to one that starts using the lxc driver. 2014-03-27 21:11:51 +00:00
Jake Moshenko
d6f1ea36d4 Upgrade the worker docker to the patched version of 0.9. 2014-03-27 20:39:29 +00:00
jakedt
536a91cbb8 Handle messages with the word message in them. 2014-03-07 12:27:30 -05:00
jakedt
4566d76e45 Add a mime type for x-zip-compressed. Report unknown mime types as errors. 2014-03-06 23:40:21 -05:00
jakedt
9a733a7320 Sometimes status messages have an extra level of wrapping for some reason. 2014-03-05 17:44:22 -05:00
jakedt
270a62b8c1 Fix the build worker to use the latest docker-py. 2014-03-05 10:32:24 -05:00
jakedt
c13f7cd9df Add the resource_key back to the repository build. 2014-02-25 18:22:02 -05:00
jakedt
13dea98499 Prepare the build worker to support multiple tags and subdirectories. Change the build database config to accept a job config object instead of breaking out the parameters into independent blocks. 2014-02-24 16:11:23 -05:00
yackob03
91da26e317 Improve the instructions for starting the worker on the build node. 2014-02-13 12:46:51 -05:00
yackob03
1e8ec634e2 Some small fixes to the dockerfile build worker. 2014-02-13 11:44:39 -05:00
yackob03
b920a0cb1f Really try to emulate the logs format with the test logs. Switch to a simplified flat logs format. 2014-02-12 18:58:40 -05:00
yackob03
83fb806e51 Apparently gdebi doesn't support multiple packages in one command? lame. 2014-02-12 10:07:10 -05:00
yackob03
d8df243a0a Add the binary dependencies for the build worker and some instructions on how to get it running on a host. 2014-02-11 19:55:22 -05:00
yackob03
6fd343741b Change to the new paging format with the commands available at the top. 2014-02-10 19:12:43 -05:00
yackob03
52d2be7953 Identify build commands separately from their output. 2014-02-10 15:03:55 -05:00
yackob03
fab699530c Check for errors before delegating to the cleanup function. 2014-02-05 19:59:26 -05:00
yackob03
67e0736fc6 Separate out the dockerfile build stuff to a separate class. Add a dependency on docker-py master. 2014-02-05 11:07:54 -05:00
yackob03
5270066d6d Switch to the redis backed build logs and status. 2014-02-03 19:08:37 -05:00
yackob03
aa58b840a5 Fix the dockerfile build login to work on http and https registries. 2014-01-24 16:25:25 -05:00
yackob03
72559fb948 First stab at the new builder. 2014-01-24 14:40:36 -05:00
yackob03
8981f576fc Fixes to the build server. 2013-12-31 16:22:58 -05:00
yackob03
bf60862d8e Switch to the new Digital Ocean Docker image. 2013-12-23 15:07:10 -05:00
yackob03
e787d8b2d8 Make the worker post json. Add a retry timeout after an incompletel queue item is processed. Submit webhook jobs to the queue on a successful push. 2013-11-16 15:05:26 -05:00
yackob03
c1ea6263e1 Flesh out the webworkers a bit. 2013-11-15 16:45:02 -05:00
yackob03
14263de7f8 Extract some boilerplate from the worker and create a base class. Port the diffs worker over to the base. 2013-11-15 15:50:20 -05:00
yackob03
0f3d942b5e Re-add the missing method for getting repository builds. Add more logging to try to see why some builds are not getting removed from the queue. 2013-11-13 17:29:26 -05:00
yackob03
d7592fd133 Update the diffs worker to not fail if the repository is removed before diffs are computed. 2013-11-13 14:41:20 -05:00