Jimmy Zelinskie
8b9f9478a4
pylint formatting
2016-10-28 17:12:46 -04:00
Jimmy Zelinskie
a30b358709
add staggered worker startup
...
Fixes #787
2016-10-28 17:12:39 -04:00
Jimmy Zelinskie
2bd1e76267
workers.queuecleanup: s/week/day cleanup frequency
2016-10-20 13:47:07 -04:00
Jimmy Zelinskie
20ef43d5fb
workers.queuecleanup: remove direct peewee usage
2016-10-20 13:46:00 -04:00
Joseph Schorr
30af8aef1a
Add a worker for reporting global stats to Prometheus
...
Fixes #1789
2016-09-12 16:19:19 -04:00
josephschorr
5c64646629
Merge pull request #1778 from coreos-inc/redlock
...
Fix locking via RedLock
2016-08-29 16:12:01 -04:00
Joseph Schorr
aa7c87d765
Fix locking via RedLock
...
Fixes #1777
2016-08-29 16:06:26 -04:00
Joseph Schorr
08a3b70b56
Extend processing before processing security notifications
...
Makes sure queue items don't expire during processing
Fixes #1776
2016-08-29 13:08:38 -04:00
Jake Moshenko
a113f548db
Accidentally forgot a line in the gc worker.
2016-08-02 10:44:53 -04:00
Jake Moshenko
05e2773fa7
Get rid of remaining slow query for garbage collection.
2016-08-01 18:22:38 -04:00
Joseph Schorr
b8d2570725
Don't raise an error on duplicate placements
...
This can happen if two pushes are racing on the same storage.
2016-07-19 16:44:05 -04:00
Joseph Schorr
5cd793331e
Fix storage replication for CAS and add tests
2016-07-12 13:46:06 -04:00
Joseph Schorr
3b994431eb
Auto expire the build status and logs in redis
2016-06-20 13:53:13 -04:00
Jake Moshenko
a1cf12e460
Add a sitemap.txt for popular public repos
...
and reference it from the robots.txt
2016-06-17 14:34:20 -04:00
Joseph Schorr
8887f09ba8
Use the instance service key for registry JWT signing
2016-06-07 11:58:10 -04:00
Joseph Schorr
dd0dd39bf0
Fix the queue cleanup worker to delete the items that have expired, not unexpired
2016-06-03 22:14:14 -04:00
Joseph Schorr
5746b42c69
Add a cleanup worker for the queue item table
...
Fixes #784
2016-06-02 15:00:44 -04:00
josephschorr
ec492bb683
Merge pull request #1323 from coreos-inc/secworkerreturn
...
Move security notification work into its own method to allow for retu…
2016-06-02 13:59:25 -04:00
Jake Moshenko
9221a515de
Use the registry API for security scanning
...
when the storage engine doesn't support direct download url
2016-05-04 18:04:06 -04:00
Joseph Schorr
73fa593d02
Various small fixes in prep for QE release
2016-05-04 15:20:27 -04:00
Jimmy Zelinskie
f842545b3e
rename config values to remove "Quay" ( #1431 )
2016-05-03 13:11:21 -04:00
Evan Cordell
489752a0b7
Only refresh current instance service key
2016-04-29 14:10:33 -04:00
Evan Cordell
a6f6a114c2
service key worker to refresh automatic keys
2016-04-29 14:10:33 -04:00
Jimmy Zelinskie
128b0cd38c
logrotateworker: archive every 24 hours
2016-04-18 13:02:30 -04:00
Jimmy Zelinskie
ef65822410
logrotateworker: perf optimizations
...
This removes our needless transaction, only calculates the cutoff date
once, removes the logs generator, and uses a tested optimal
MIN_LOGS_PER_ROTATION.
2016-04-15 16:51:17 -04:00
Jimmy Zelinskie
3d190b786f
userfiles: make handler optional
2016-04-15 13:56:07 -04:00
Jimmy Zelinskie
c7c52e6c74
logrotateworker: save to storage via userfiles
2016-04-14 13:29:29 -04:00
Joseph Schorr
d62ec22fc9
Move security notification work into its own method to allow for return values
...
Fixes #1302
Fixes #1304
2016-03-31 14:08:33 -04:00
Joseph Schorr
dc8f9713f8
Change logs worker to use a global lock in the inner loop and move storage out of the transaction
2016-03-24 14:09:48 -04:00
Joseph Schorr
aa5587c93c
Fixes and added tests for the security notification worker
...
Fixes #1301
- Ensures that the worker uses pagination properly
- Ensures that the worker handles failure as expected
- Moves marking the notification as read to after the worker processes it
- Increases the number of layers requested to 100
2016-03-18 20:28:06 -04:00
Quentin Machu
5b7d6b0638
Merge pull request #1275 from Quentin-M/min_id_once
...
Compute min_id only once during securityworker's lifetime
2016-03-04 14:02:47 -05:00
Quentin Machu
54153c9b80
Compute min_id only once during securityworker's lifetime
2016-03-04 14:02:28 -05:00
Jimmy Zelinskie
b5d904f373
Merge pull request #1218 from jzelinskie/logrotate5ever
...
vastly simplify log rotation
2016-03-04 13:48:21 -05:00
Quentin Machu
888f976e8d
Use a feature flag to toggle security notifications
2016-03-01 15:54:18 -05:00
Joseph Schorr
f498e92d58
Implement against new Clair paginated notification system
2016-02-25 15:58:42 -05:00
Joseph Schorr
c0374d71c9
Refactor the security worker and API calls and add a bunch of tests
2016-02-25 12:29:41 -05:00
Quentin Machu
e5da33578c
Adapt security worker for Clair v1.0 (except notifications)
2016-02-19 17:44:14 -05:00
Quentin Machu
f62a05f6d7
various securityworker fixes
2016-02-09 21:25:07 -05:00
Quentin Machu
1d2b31a581
Mark layers that Clair can't extract as failed
2016-02-09 18:24:35 -05:00
Jimmy Zelinskie
ee705fe7a9
vastly simplify log rotation
2016-02-09 18:20:14 -05:00
Quentin Machu
13c10ba7b1
Double the securityworker indexing interval
2016-02-09 14:49:10 -05:00
Joseph Schorr
ab166c4448
Delete the image diff feature
...
Fixes #1077
2015-12-23 13:08:01 -05:00
Jimmy Zelinskie
f439ad7804
Merge pull request #618 from jzelinskie/logsworker
...
add a log rotation worker
2015-12-16 17:25:50 -05:00
Jimmy Zelinskie
e1f955a3f6
add a log rotation worker
...
Fixes #609 .
2015-12-16 17:22:28 -05:00
Joseph Schorr
c888a8b3be
Make GC timeout configurable
2015-12-16 15:45:02 -05:00
Jake Moshenko
2f626f2691
Unify the database connection lifecycle across all workers
2015-12-04 15:51:53 -05:00
Joseph Schorr
544fa40a5f
Add a base class for a global worker that locks via Redis
2015-11-24 16:18:45 -05:00
Silas Sewell
1162814734
securityworker: mark children we can't analyze
...
This allows us to differentiate between images that are queued and those we
can't analyze in constant time.
2015-11-19 11:22:15 -05:00
Quentin Machu
88e85cded0
Fix security worker (again?)
2015-11-18 19:45:09 -05:00
Quentin Machu
7e9faa6c54
Add missing import
2015-11-18 17:39:27 -05:00
