Jimmy Zelinskie
09ce33e0dc
fix case where query broke on empty list
2015-11-13 12:35:18 -05:00
Joseph Schorr
927a0b639c
Add check for empty locations list
2015-11-13 12:23:02 -05:00
Joseph Schorr
030c69d7d2
Further merge fixes
2015-11-12 22:00:28 -05:00
Joseph Schorr
7816b0c657
Merge master into vulnerability-tool
2015-11-12 21:52:47 -05:00
Joseph Schorr
25b8b7590f
Fix all the things!
2015-11-12 20:55:41 -05:00
Jimmy Zelinskie
37ce84f6af
tiny fixes to securityworker
2015-11-12 17:18:04 -05:00
Jimmy Zelinskie
f6a34c5d06
refactor securityworker
...
Fixes #772 .
2015-11-12 16:03:10 -05:00
Jake Moshenko
ab340e20ea
Merge remote-tracking branch 'upstream/master' into python-registry-v2
2015-11-11 16:41:40 -05:00
Joseph Schorr
ca7d736db2
Only send vulnerability events if the minimum priority is gte to that specified
...
Fixes #770
2015-11-10 16:05:55 -05:00
Jimmy Zelinskie
8e2868737b
rename secscan_endpoint and move db close to API
2015-11-10 15:22:31 -05:00
Jimmy Zelinskie
da31714fb5
specify securityworker skip message
2015-11-10 15:22:30 -05:00
Jimmy Zelinskie
52962b3732
close db connections when calling out to clair
2015-11-10 15:22:30 -05:00
Jimmy Zelinskie
d651ea4b48
initial security notification worker
2015-11-10 15:22:30 -05:00
Quentin Machu
16c364a90c
Rename secscan_endpoint where required, fix index and indentation
2015-11-09 15:18:42 -05:00
Joseph Schorr
2d2662f53f
Fix deleting repos and images under MySQL
...
MySQL doesn't handle constraints at the end of transactions, so deleting images currently fails. This removes the constraint and just leaves parent_id as an int
2015-11-09 14:42:05 -05:00
Quentin Machu
7dbe15e339
Remove checksum from Clair's worker and adjust line length
2015-11-09 14:31:24 -05:00
Joseph Schorr
b408cfd2cc
Ready for demo
2015-11-09 12:51:05 -05:00
Joseph Schorr
7fa4fe08e7
Fix worker
2015-11-09 12:50:39 -05:00
Joseph Schorr
407eaae137
WIP: Towards sec demo
2015-11-09 12:50:39 -05:00
Quentin Machu
37118423a5
Add support for Quay's vulnerability tool
2015-11-09 12:49:19 -05:00
Jake Moshenko
c2fcf8bead
Merge remote-tracking branch 'upstream/phase4-11-07-2015' into python-registry-v2
2015-11-06 18:18:29 -05:00
Quentin Machu
af4511455f
Remove .distinct() from these queries
2015-11-06 15:22:18 -05:00
Quentin Machu
3677947521
Add support for Quay's vulnerability tool
2015-11-06 15:22:18 -05:00
Quentin Machu
1b41200e49
Fix PostgresSQL compatibility and parent omittance securityworker
2015-11-06 15:22:18 -05:00
Quentin Machu
f59e35cc81
Add support for Quay's vulnerability tool
2015-11-06 15:22:18 -05:00
Jake Moshenko
9da64f3aba
Stop writing to deprecated columns for image data.
2015-10-24 14:45:15 -04:00
Jake Moshenko
e7a6176594
Merge remote-tracking branch 'upstream/v2-phase4' into python-registry-v2
2015-10-22 16:59:28 -04:00
Jake Moshenko
ce94931540
Stop writing to deprecated columns for image data.
2015-10-22 12:14:39 -04:00
josephschorr
8e7b20a0d7
Merge pull request #675 from coreos-inc/distinctgc
...
Reduce GC work time and make sure to use distinct query
2015-10-21 12:01:26 -04:00
Silas Sewell
fd96f7c1e3
Merge pull request #667 from coreos-inc/error-georeplication-local-storage
...
workers.storagereplication: error on LocalStorage
2015-10-20 20:29:24 -04:00
Silas Sewell
03f5fe6143
workers.storagereplication: error on LocalStorage
...
Ensure we don't start when LocalStorage is in the config.
Fixes #502
2015-10-20 19:04:31 -04:00
Joseph Schorr
4e5c8a9281
Reduce GC work time and make sure to use distinct query
2015-10-20 18:13:29 -04:00
Joseph Schorr
5941f3937c
Enable async GC for all
...
Fixes #569
2015-10-19 14:22:41 -04:00
Jimmy Zelinskie
7c82e0b5b3
move UseThenDisconnect into queueworker
...
This makes the tests pass while maintaining the same behavior.
2015-09-21 13:34:12 -04:00
Joseph Schorr
96d5bbb155
Fix exceptions raised by the diffs worker
...
Fixes #465
2015-09-10 14:12:16 -04:00
Joseph Schorr
3ee4147117
Switch the build logs archiver to a more performant query
...
Fixes #459
2015-09-09 13:59:45 -04:00
Joseph Schorr
724b1607d7
Add automatic storage replication
...
Adds a worker to automatically replicate data between storages and update the database accordingly
2015-09-01 14:53:32 -04:00
Matt Jibson
7407bca728
Correct fix for notification get repo
...
The fix in #366 was wrong. Not sure how I tested it and it worked.
2015-08-17 17:54:33 -04:00
Matt Jibson
132bc4491b
Fix notification worker's use of get repo notification
2015-08-14 15:42:31 -04:00
Joseph Schorr
c3d7ef2ec4
Only start workers once setup is complete on the registry
...
Fixes #326
2015-08-07 13:44:14 -04:00
Joseph Schorr
14f511bb5a
Make sure to set a default for Raven client
...
Fixes #327
2015-08-07 13:03:38 -04:00
Joseph Schorr
572d6ba53c
Fix broken tests
2015-07-29 14:21:29 -04:00
Joseph Schorr
ac0cca2d90
Switch to a unified worker system
...
- Handles logging
- Handles reporting to Sentry
- Removes old code around serving a web endpoint (unused now)
2015-07-28 17:26:12 -04:00
Joseph Schorr
70de107268
Make GC of repositories fully async for whitelisted namespaces
...
This change adds a worker to conduct GC on repositories with garbage every 10s.
Fixes #144
2015-07-28 15:30:04 -04:00
Jake Moshenko
3efaa255e8
Accidental refactor, split out legacy.py into separate sumodules and update all call sites.
2015-07-17 11:56:15 -04:00
Jake Moshenko
acbcc2e206
Start of a v2 API.
2015-07-17 11:50:41 -04:00
Joseph Schorr
6eaf1dbb3f
Make the repositoryactioncount worker disconnect from the DB between runs
2015-04-22 17:11:08 -04:00
Joseph Schorr
657ba576a8
Make sure to import app so that the DB proxy gets properly initialized
2015-04-13 14:25:09 -04:00
Joseph Schorr
3f1e8f3c27
Add a RepositoryActionCount table so we can use it (instead of LogEntry) when scoring repo search results
2015-04-13 13:31:07 -04:00
Joseph Schorr
3872d29de9
Add a transaction around the extend_processing call
2015-01-29 18:40:41 -05:00
Jake Moshenko
11562a74de
Remove the old builder infrastructure.
2015-01-29 11:03:23 -05:00
Joseph Schorr
dbac8c7e3d
Fix build code:
...
- Fix issue with the queue_item in extend processing
- Add the new compiled docker binary with the lxc volume fix
2014-12-04 17:49:39 +01:00
Joseph Schorr
b8e9f2d1fa
Disable the lxc stability check. LXC is so broken that this was causing the build fleet to thrash when it encountered real issues with LXC/user namespacing.
2014-11-25 04:18:50 -05:00
Joseph Schorr
b2a0e58756
Use the new kwargs_from_env so that we can test with boot2docker and fix the issue with the .history call.
2014-11-24 20:28:48 -05:00
Jimmy Zelinskie
716d7a737b
Strip whitespace from ALL the things.
2014-11-24 16:07:38 -05:00
Jake Moshenko
f4681f2c18
Merge branch 'master' into nomenclature
...
Conflicts:
test/data/test.db
2014-11-17 17:59:59 -05:00
Joseph Schorr
c06f57a6e7
Make sure builders close the db handle when no work comes in and make the metrics transaction smaller in scope
2014-10-24 11:40:02 -04:00
Jake Moshenko
1461310ab8
Merge remote-tracking branch 'origin/master' into nomenclature
...
Conflicts:
endpoints/common.py
endpoints/notificationhelper.py
test/data/test.db
workers/dockerfilebuild.py
2014-10-23 13:25:37 -04:00
Jake Moshenko
1ccd6a9c5d
Change the max_instances for the workers to only allow one parallel job execution.
2014-10-22 18:09:00 -04:00
Joseph Schorr
07f3bd6f8c
Add a synthetic .git directory containing the commit sha so that 'git rev-parse HEAD' works from inside builds
2014-10-10 17:20:07 -04:00
Jake Moshenko
ed8bcff39e
Merge remote-tracking branch 'origin/master' into nomenclature
...
Conflicts:
test/data/test.db
workers/dockerfilebuild.py
2014-10-06 10:29:39 -04:00
Joseph Schorr
ec42303750
image_and_tag must be before we use it
2014-10-03 13:00:41 -04:00
Jake Moshenko
e8b3d1cc4a
Phase 4 of the namespace to user migration: actually remove the column from the db and remove the dependence on serialized namespaces in the workers and queues
2014-10-01 14:23:46 -04:00
Joseph Schorr
9c88ca16b5
Add the docker version to the build logs
2014-09-23 14:45:22 -04:00
Joseph Schorr
f23038c6ee
Update the worker code to better handle exceptions, fix the utcdate issue and make sure we send the proper retry. Also updates notification workers to send JobExceptions rather than returning true or false
2014-09-22 12:52:57 -04:00
Jake Moshenko
15a2f0d56f
Allow most builds to be archived without writing a file to the disk.
2014-09-16 00:03:04 -04:00
Jake Moshenko
efb66f7c1e
Select the random row function based on DB driver.
2014-09-15 15:58:56 -04:00
Jake Moshenko
c01de4a916
Set redis logs entries to expire rather than to immediately delete them to make the logs archiver idempotent.
2014-09-12 13:13:14 -04:00
Jake Moshenko
353da13b16
Set up the service monitoring for the log migration script. Add a database migration to add the logs_archived column.
2014-09-12 11:57:47 -04:00
Jake Moshenko
b55e79e5d3
Merge remote-tracking branch 'origin/master' into waltermitty
2014-09-12 11:45:07 -04:00
Jake Moshenko
5388633f9a
Merge remote-tracking branch 'origin/master' into pullfail
2014-09-12 10:36:38 -04:00
Jake Moshenko
8b3a3178b0
Finish the build logs archiver, add handlers for cloud and local that handle gzip encoded archived content.
2014-09-11 15:33:10 -04:00
Jake Moshenko
2455c17f96
Merge remote-tracking branch 'origin/master' into waltermitty
...
Conflicts:
app.py
data/userfiles.py
2014-09-11 11:18:28 -04:00
Jake Moshenko
29d40db5ea
Add a new RadosGW storage engine. Allow engines to distinguish not only between those that can support direct uploads and downloads, but those that support doing it through the browser. Rename resumeable->resumable.
2014-09-09 15:54:03 -04:00
Jake Moshenko
451e034ca1
Archived logs commit 1. Squash me.
2014-09-08 16:43:17 -04:00
Joseph Schorr
07aab4274c
Fix parameters for logging the extra data needed
2014-08-28 19:19:20 -04:00
Joseph Schorr
5744f0f888
Make the dockerfilebuild error checking less harsh
2014-08-28 16:07:56 -04:00
Joseph Schorr
463a3c55c3
Make worker error messages more descriptive
2014-08-27 19:02:53 -04:00
Joseph Schorr
510bbe7889
Add more check conditions for unhealthy workers and make the messaging better.
2014-08-26 12:41:43 -04:00
Joseph Schorr
67905c277e
Remove webhook worker
2014-08-25 19:13:40 -04:00
Joseph Schorr
daa43c3bb9
Add better messaging around pulling of base images when they fail due to invalid or missing credentials
2014-08-18 20:34:39 -04:00
Joseph Schorr
736af3165b
Add a default message if the build pack MIME processor fails
2014-08-15 18:23:43 -04:00
Joseph Schorr
8681dd9cb9
Add a new exposed 'unpacking' phase to the build and make sure that the unzip/untar/etc always occurs under a try-except
2014-08-15 17:58:11 -04:00
Joseph Schorr
728af56384
Make the watchdog in the build worker also requeue the current item if the worker has gone bad
2014-08-13 19:04:51 -04:00
Joseph Schorr
b9e9064af2
Only retry on unhealthy exceptions, not JobException's.
2014-08-10 18:28:20 -04:00
Joseph Schorr
1b7379df29
Fix workers to not always be marked as unhealthy
2014-08-08 15:24:19 -04:00
Jake Moshenko
0372013f70
Merge remote-tracking branch 'origin/redalert'
...
Conflicts:
app.py
2014-08-04 16:56:34 -04:00
Jake Moshenko
0aa6e92b02
Finish porting the workers over to apscheduler 3.0
2014-08-01 18:38:02 -04:00
Jake Moshenko
6b38ddb9b6
Remove the gpled loremipsum module.
2014-07-31 16:46:02 -04:00
Joseph Schorr
49801bc2c4
- Add web hook queue code back in. We'll remove it and turn it off after this CL goes to prod
...
- Make notification lookup always be by repo and its UUID, rather than the internal DB ID
- Add the init script for the notification worker
2014-07-31 13:30:54 -04:00
Joseph Schorr
bab3a0949c
Make sure completion marking is also under the lock
2014-07-30 18:45:40 -04:00
Joseph Schorr
4aec422e24
Add a lock around accessing the current queue item and make sure to report it as incomplete whenever the worker becomes unhealthy
2014-07-30 18:30:54 -04:00
Joseph Schorr
7e935f5a8c
Make build workers report that they are unhealthy when we get an LXC error or a Docker connection issue
2014-07-30 17:54:58 -04:00
Joseph Schorr
752efb9e0f
Fix the spawn_notification to work in all cases and clean up some of the remaining code
2014-07-18 16:34:52 -04:00
Joseph Schorr
591cd020b8
Merge branch 'master' into redalert
2014-07-18 15:58:56 -04:00
Joseph Schorr
af31bde997
Add support for the remaining events to the frontend and the backend
2014-07-18 15:58:18 -04:00
Jake Moshenko
74d1c4e6b0
Update the worker status endpoint to be ELB friendly.
2014-07-18 15:04:20 -04:00
Joseph Schorr
8d7493cb86
Convert over to notifications system. Note this is incomplete
2014-07-17 22:51:58 -04:00
Jake Moshenko
cceb09d4f6
Remove some unused dependencies and update the rest.
2014-07-17 12:08:07 -04:00
Joseph Schorr
8b3659fefa
Dockerfile build worker should not report inner JobException's twice
2014-07-11 12:05:52 -04:00
Joseph Schorr
9d1ae8ba87
FROM line check needs to be on the tuple result, not the join
2014-06-16 14:01:17 -04:00
Joseph Schorr
f795868b5b
Handle the case where there is no FROM command in the Dockerfile
2014-06-13 16:56:48 -04:00
Jake Moshenko
e8355f301e
Remove our deploy key from the workers/Readme which gets included in the Docker image.
2014-05-27 15:19:23 -04:00
Jake Moshenko
0b6552d6cc
Fix the metrics so they are usable for scaling the workers down and up. Switch all datetimes which touch the database from now to utcnow. Fix the worker Dockerfile.
2014-05-23 14:16:26 -04:00
Jake Moshenko
d14798de1d
Add a queue capacity reporter plugin to the queue. Move the queue definitions to app. Add a cloudwatch reporter to the dockerfile build queue.
2014-05-21 19:50:37 -04:00
Jake Moshenko
b8466169ac
Integrate sentry with the build worker.
2014-05-19 13:50:45 -04:00
Jake Moshenko
212a4650f4
Rework the config to use runit logging.
2014-05-18 17:19:14 -04:00
Jake Moshenko
cc47e77156
Upgrade to the 0.11.1 tutum version of docker. Package it as a Dockerfile using Docker in Docker. Add a status server option to the workers to utilize the new termination signal and status features of gantry.
2014-05-16 18:31:24 -04:00
Jake Moshenko
c92ce54a37
Reduce a step in the worker bootstrap.
2014-05-13 17:44:45 -04:00
Jake Moshenko
bcb993a914
Set up the build logs to use our fake build logs on test and local.
2014-05-09 18:45:11 -04:00
Jake Moshenko
8a3af93b8c
Improve the builder response to being terminated or dying.
2014-05-06 18:46:19 -04:00
Jake Moshenko
55f18a2ecf
Add the missing uid translation range to the root user.
2014-05-01 17:54:59 -04:00
Jake Moshenko
ec282999bf
Use the docker version which works with 14.04 lxc.
2014-05-01 17:24:58 -04:00
Jake Moshenko
32583a5675
First steps toward running the builder on trusty.
2014-05-01 15:39:33 -04:00
Jake Moshenko
b888c05bc4
Change the version of our docker binary because the public registry is blocking the tutum agent name.
2014-05-01 11:44:59 -04:00
Jake Moshenko
450928674b
Use a new caching algorithm which can limit the size for the build nodes. Stop treating public images as special. Add a new phase to the builder for pulling.
2014-04-30 18:48:36 -04:00
jakedt
58dbb540a1
Run a worker task immediately when it starts.
2014-04-22 13:55:54 -04:00
jakedt
2bc3d24543
Update the build worker to remove all tags from expired images.
2014-04-18 18:36:11 -04:00
jakedt
0d8725e778
Update the instructions for starting and running the workers.
2014-04-17 16:18:53 -04:00
jakedt
0a9ee6c49f
Bust the dockerfile build cache across repository lines.
2014-04-16 15:45:41 -04:00
jakedt
0827e0fbac
Merge remote-tracking branch 'origin/master' into ncc1701
...
Conflicts:
endpoints/web.py
static/directives/signup-form.html
static/js/app.js
static/js/controllers.js
static/partials/landing.html
static/partials/view-repo.html
test/data/test.db
2014-04-14 19:37:22 -04:00
jakedt
724fec1b74
Test third party repo images for public-ness in the builder. Always clean up private images that we dont know about before build. Pull the base image to refresh before every build.
2014-04-14 18:54:39 -04:00
jakedt
40f82a9d16
Work harder to reset the state of the docker env on the build worker.
2014-04-14 15:59:57 -04:00
jakedt
de18236358
Allow for caching of previous docker builds for 24 hours.
2014-04-14 15:21:05 -04:00
jakedt
61a6db236f
Finish the implementation of local userfiles. Strip charsets from mimetypes in the build worker. Add canonical name ordering to the build queue. Port all queues to the canonical naming version.
2014-04-11 18:34:47 -04:00
jakedt
576fbe4f0d
Switch over to phusion baseimage. Prevent everything from daemonizing and start it with runit under phusion. Make workers trap and handle sigint and sigterm. Extend the reservation to 1hr for dockerfilebuild. Update nginx to remove the dependency on libgd. Merge the requirements and requirements enterprise files.
2014-04-11 13:32:45 -04:00
jakedt
8fac0474b5
Get staging to run under docker on an EC2 host.
2014-04-10 18:30:09 -04:00
jakedt
8076ad0a20
Add x-gzip as a mimetype for tarballs.
2014-04-07 10:58:53 -04:00
jakedt
37ad8a414a
Revert a change to the way zip files are extracted.
2014-04-04 16:06:41 -04:00
jakedt
e87ffa20cf
First attempt at making config loadable through string config overrides in an env variable.
2014-04-03 17:31:46 -04:00
jakedt
b95d3ec329
Add a watchdog timer to the build worker to kill a build step that takes more than 20 minutes.
2014-04-02 19:32:41 -04:00
Joseph Schorr
9a79d1562a
Change to store the pull robot on the repository build and only add the credentials to the queue item. This prevents the credentials from being exposed to the end user. Also fixes the restart build option
2014-04-01 21:49:06 -04:00
jakedt
ca1970a2f4
Use the real registry endpoint in the login command.
2014-04-01 19:00:11 -04:00
jakedt
3525e383df
Merge remote-tracking branch 'origin/master' into pullinprivate
...
Conflicts:
test/data/test.db
2014-04-01 18:28:48 -04:00
jakedt
7c44932c87
Use safer tar extraction. Handle error messages in the build process more intelligently.
2014-04-01 13:46:41 -04:00
jakedt
d67a1cddc2
Merge remote-tracking branch 'origin/master' into pullinprivate
...
Conflicts:
workers/dockerfilebuild.py
2014-03-31 18:10:34 -04:00
jakedt
e7c20e1052
Add tarball support to the builder and pull github code as a tarball.
2014-03-31 15:40:24 -04:00
jakedt
7c14190d2a
Freeze the version of docker-py to 0.3.0 and create a fancy streaming json decoder to work around the lack of newlines in push statuses.
2014-03-28 17:53:33 -04:00
Joseph Schorr
2006917e03
Add support for pull credentials on builds and build triggers
2014-03-27 18:33:13 -04:00
Jake Moshenko
40e81f478f
If JSON decoding of a stream response fails, just ignore it since it's probably a blank line.
2014-03-27 22:15:59 +00:00
Jake Moshenko
2767c7e1f7
Change the docker version to one that starts using the lxc driver.
2014-03-27 21:11:51 +00:00
Jake Moshenko
d6f1ea36d4
Upgrade the worker docker to the patched version of 0.9.
2014-03-27 20:39:29 +00:00
jakedt
536a91cbb8
Handle messages with the word message in them.
2014-03-07 12:27:30 -05:00
jakedt
4566d76e45
Add a mime type for x-zip-compressed. Report unknown mime types as errors.
2014-03-06 23:40:21 -05:00
jakedt
9a733a7320
Sometimes status messages have an extra level of wrapping for some reason.
2014-03-05 17:44:22 -05:00
jakedt
270a62b8c1
Fix the build worker to use the latest docker-py.
2014-03-05 10:32:24 -05:00
jakedt
c13f7cd9df
Add the resource_key back to the repository build.
2014-02-25 18:22:02 -05:00
jakedt
13dea98499
Prepare the build worker to support multiple tags and subdirectories. Change the build database config to accept a job config object instead of breaking out the parameters into independent blocks.
2014-02-24 16:11:23 -05:00
yackob03
91da26e317
Improve the instructions for starting the worker on the build node.
2014-02-13 12:46:51 -05:00
yackob03
1e8ec634e2
Some small fixes to the dockerfile build worker.
2014-02-13 11:44:39 -05:00
yackob03
b920a0cb1f
Really try to emulate the logs format with the test logs. Switch to a simplified flat logs format.
2014-02-12 18:58:40 -05:00
yackob03
83fb806e51
Apparently gdebi doesn't support multiple packages in one command? lame.
2014-02-12 10:07:10 -05:00
yackob03
d8df243a0a
Add the binary dependencies for the build worker and some instructions on how to get it running on a host.
2014-02-11 19:55:22 -05:00
yackob03
6fd343741b
Change to the new paging format with the commands available at the top.
2014-02-10 19:12:43 -05:00
yackob03
52d2be7953
Identify build commands separately from their output.
2014-02-10 15:03:55 -05:00
yackob03
fab699530c
Check for errors before delegating to the cleanup function.
2014-02-05 19:59:26 -05:00
yackob03
67e0736fc6
Separate out the dockerfile build stuff to a separate class. Add a dependency on docker-py master.
2014-02-05 11:07:54 -05:00
yackob03
5270066d6d
Switch to the redis backed build logs and status.
2014-02-03 19:08:37 -05:00
yackob03
aa58b840a5
Fix the dockerfile build login to work on http and https registries.
2014-01-24 16:25:25 -05:00
yackob03
72559fb948
First stab at the new builder.
2014-01-24 14:40:36 -05:00
yackob03
8981f576fc
Fixes to the build server.
2013-12-31 16:22:58 -05:00
yackob03
bf60862d8e
Switch to the new Digital Ocean Docker image.
2013-12-23 15:07:10 -05:00
yackob03
e787d8b2d8
Make the worker post json. Add a retry timeout after an incompletel queue item is processed. Submit webhook jobs to the queue on a successful push.
2013-11-16 15:05:26 -05:00
yackob03
c1ea6263e1
Flesh out the webworkers a bit.
2013-11-15 16:45:02 -05:00
yackob03
14263de7f8
Extract some boilerplate from the worker and create a base class. Port the diffs worker over to the base.
2013-11-15 15:50:20 -05:00
yackob03
0f3d942b5e
Re-add the missing method for getting repository builds. Add more logging to try to see why some builds are not getting removed from the queue.
2013-11-13 17:29:26 -05:00
yackob03
d7592fd133
Update the diffs worker to not fail if the repository is removed before diffs are computed.
2013-11-13 14:41:20 -05:00
yackob03
2cd98fc58e
Make the app config more powerful in terms of injecting fake dependencies. Refactor the tests to use metaclasses and to actually all run.
2013-11-06 23:21:12 -05:00
yackob03
f8f4c55f89
Close the db connections on the workers occasionally.
2013-10-30 13:22:00 -04:00
yackob03
1bbcd91ae0
Make sure the daemon starts up in the right directory to access the certs file.
2013-10-29 19:59:29 -04:00
yackob03
7adf37e6b5
Move the parmiko import to work around a bug with python daemonization and paramiko.
2013-10-29 18:56:57 -04:00
yackob03
5e81f999d1
Make the build server and build worker slightly more robust to errors.
2013-10-29 18:15:12 -04:00
yackob03
6f105326aa
Return unfinished items to the queue when they can be retried.
2013-10-29 15:42:19 -04:00
yackob03
798cf78172
Re-add cleaning up the DO node.
2013-10-29 15:39:44 -04:00
yackob03
c91b40f356
Switch to using environment variables to pass information to the build node, this closes down a security loophole with the admin endpoint of the server.
2013-10-29 13:42:26 -04:00
yackob03
1c7398e920
Allow build node logs to flow to docker. Clean up the pull key before executing user code. Actually clean up the DO node when we are done. Run the status server threaded just in case.
2013-10-28 14:49:23 -04:00
yackob03
d7f51fb764
Bug fixes and associated changes involved with spinning up build nodes and sending jobs to them.
2013-10-27 19:06:20 -04:00
yackob03
78d2d6cad0
Add a test endpoint that allows us to manually create builds. Add access tokens to the build data.
2013-10-25 18:17:43 -04:00
yackob03
684ce83058
Add a test api to test repositories which are currenlty building.
2013-10-25 15:47:34 -04:00
yackob03
ffc33e454c
Fixes and refinements to the dockerfile build system.
2013-10-25 15:13:11 -04:00
yackob03
9b9a29c310
Flesh out the create API and wire everything up together. Next up, testing.
2013-10-25 01:14:38 -04:00
yackob03
32ccc599fb
Revert some fabric changes to the build worker.
2013-10-24 16:39:25 -04:00
yackob03
847e91b696
Add the first part of the build worker that runs on quay.
2013-10-24 16:37:03 -04:00
yackob03
0c979cd79f
Fix the diffsworker to run as a daemon.
2013-10-20 14:52:56 -04:00
yackob03
abb6efda40
Process as many jobs as the queue will give us in a row, sleep for longer in between checks.
2013-10-20 03:26:06 -04:00
yackob03
607bf3ecc4
Daemonize the worker.
2013-10-18 17:52:55 -04:00
yackob03
2fcd8df42b
Get the worker and the registry playing nice together.
2013-10-18 17:27:09 -04:00
yackob03
4514f5a969
Fix some typos and bugs in the worker.
2013-10-18 15:28:16 -04:00
yackob03
93b856bdb3
First few changes for the image diffs feature.
2013-10-17 18:25:19 -04:00