Commit graph

329 commits

Author SHA1 Message Date
josephschorr
78652de3ee Merge pull request #2766 from coreos-inc/joseph.schorr/QUAY-634/buildlogsarchiver-data-interface
Change buildlogsarchiver to use a data model interface
2017-07-19 16:40:05 -04:00
josephschorr
9bd4cee029 Merge pull request #2765 from coreos-inc/joseph.schorr/QUAY-629/globalprom-data-interface
Switch globalpromstats worker to use a data interface
2017-07-19 16:39:36 -04:00
Joseph Schorr
89fad7568d Fix temp fix PR for notifications 2017-07-16 12:09:30 +03:00
Joseph Schorr
026a0d28df Temporary fix for empty event config JSON 2017-07-15 14:01:27 +03:00
Joseph Schorr
bf3e941d7f Fix notification system to use the new tuple correctly 2017-07-13 22:43:26 +03:00
josephschorr
fdb21aa5dc Merge pull request #2777 from coreos-inc/joseph.schorr/QUAY-618/notificationworker-data-interface
Change notificationworker to use data interface
2017-07-13 00:23:15 +03:00
josephschorr
2206c81a95 Merge pull request #2776 from coreos-inc/joseph.schorr/QUAY-652/servicekeyworker-data-interface
Change service key worker to use a data interface
2017-07-13 00:22:49 +03:00
Joseph Schorr
4ed73d247b yapf 2017-07-12 18:11:51 +03:00
Joseph Schorr
b6f1782642 Change notificationworker to use a data interface 2017-07-12 17:40:45 +03:00
Joseph Schorr
8ec198228c Change notificationworker test to pytest 2017-07-12 17:35:09 +03:00
Joseph Schorr
50c2f1fde8 Move notification worker test under its own package 2017-07-12 17:35:09 +03:00
Joseph Schorr
fbfd78532c Move notification worker to its own package 2017-07-12 17:35:09 +03:00
Joseph Schorr
932db23a5c Change servicekeyworker to use a data interface 2017-07-12 16:37:32 +03:00
Joseph Schorr
0afc222214 Add basic unit test for the servicekeyworker 2017-07-12 16:19:30 +03:00
Joseph Schorr
3b496e2759 Move serverkeyworker into its own package 2017-07-12 15:57:02 +03:00
Joseph Schorr
e2cf2d6f2b Move teamsyncworker into its own package 2017-07-12 15:53:01 +03:00
josephschorr
dc6c6b30fc Merge pull request #2768 from coreos-inc/joseph.schorr/QUAY-653/blobuploadcleanupworker-data-interface
Change blobuploadcleanupworker to use a data interface
2017-07-12 00:32:09 +03:00
josephschorr
96d1fd128d Merge pull request #2757 from coreos-inc/joseph.schorr/QUAY-606/logarchive-georep
Add support for QE customers to enable log rotation
2017-07-12 00:30:04 +03:00
Joseph Schorr
8ded8f573d yapf 2017-07-11 16:58:30 +03:00
Joseph Schorr
bdab367285 Change blobuploadcleanupworker to use a data interface 2017-07-11 16:58:09 +03:00
Joseph Schorr
b2053829f9 Add a basic test for blob upload cleanup 2017-07-11 16:35:10 +03:00
Joseph Schorr
b87415129f Move blobuploadcleanupworker into its own package 2017-07-11 15:38:10 +03:00
Joseph Schorr
c7f1944bd5 yapf 2017-07-11 15:33:48 +03:00
Joseph Schorr
8ba71f7a45 Change buildlogsarchiver to use a data interface 2017-07-11 15:33:28 +03:00
Joseph Schorr
b7a2a4390b Add a basic build logs archiver test 2017-07-11 15:12:34 +03:00
Joseph Schorr
22f088d90a Move buildlogsarchiver worker to its own package 2017-07-11 14:42:18 +03:00
Joseph Schorr
8e179cb865 Switch globalpromstats worker to use a data interface 2017-07-11 14:01:07 +03:00
Joseph Schorr
0629a13da2 Add very basic test for stats worker 2017-07-11 13:57:20 +03:00
Joseph Schorr
13922fd194 Remove unused imports 2017-07-11 13:52:35 +03:00
Joseph Schorr
265520d071 Move globalpromstats worker into its own package 2017-07-11 13:52:15 +03:00
EvB
6e2fad2b9c refactor(queueworker): remove unused function and import
Remove `_close_db_handle` method from `QueueWorker` class. Nowhere
calls this method, so it is safe to remove. This function was the
only place using the `db` imported from `data.model`, so we can
remove that import as well.

Testing: need to look into it
2017-07-10 10:49:39 -04:00
Joseph Schorr
fa21e42ffb Add default location for action log archiving
Prevents the logs from being written to the preferred storage, which means they will jump around
2017-07-10 12:37:44 +03:00
josephschorr
a96555511b Merge pull request #2743 from coreos-inc/joseph.schorr/QUAY-663/gcworker-interface
Change GC worker to use new data interface style
2017-06-29 20:54:04 +03:00
Joseph Schorr
27ed3bedcc yapf 2017-06-29 09:43:04 +03:00
Joseph Schorr
138881dab8 yapf format 2017-06-29 09:40:39 +03:00
Joseph Schorr
76c9339453 Rename GC worker package to gc 2017-06-29 09:37:32 +03:00
Joseph Schorr
420a5e5a3a Change GC worker to use data interface 2017-06-28 15:13:11 +03:00
Joseph Schorr
38f1752a2d Move gcworker into its own package 2017-06-28 15:04:10 +03:00
Joseph Schorr
1ddb09ac11 Change security worker to use data interface 2017-06-28 14:50:52 +03:00
Joseph Schorr
ec81148d73 Add super basic security worker test 2017-06-28 14:03:57 +03:00
Joseph Schorr
7b72cf8b27 Small fix for georeplication and add better logs
Previously, if we attempted to georeplicate storage from the existing location and, somehow, that existing location did not exist, we'd still mark the new location as invalid. This is a major problem for storage engines that are not consistent. Now, we first try a back off strategy to find the image in the existing storage and, as well, if the replication fails in any way, we log it.
2017-06-23 17:07:05 -04:00
Antoine Legrand
f0dd2e348b Merge pull request #2551 from coreos-inc/structured-logs
Add log formatter class
2017-06-07 08:22:18 -07:00
Antoine Legrand
3c99928a27 Add log JSON formatter 2017-06-07 00:02:52 +02:00
Kenny Lee Sin Cheong
ad1a0e0840 logger.exception dumps a stack trace by default 2017-06-02 17:21:40 -04:00
Kenny Lee Sin Cheong
3302a96f88 Log the APIRequestFailure at ERROR level 2017-06-02 14:49:50 -04:00
Kenny Lee Sin Cheong
b5f8e7e24d Returning from the method instead of calling sleep
Simply returning from the method will give DEFAULT_INDEXING_INTERVAL seconds
before the next scan operation.
2017-06-02 12:28:17 -04:00
Kenny Lee Sin Cheong
203c0b76e0 Raise an APIRequestFailure exception when security scanner is unavailable
Put worker to sleep for the duration of the default indexing interval
when an APIRequestFailure occurs, when the API request fails due to a
connection error, timeout, or other ambiguous errors, from
analyze_layer or get_layer_data .
2017-05-24 11:04:44 -04:00
Charlton Austin
4dbd1e2eca fix(notification_worker): added in correct exception catching
before we were not catching the correct exception

[TESTING -> locally using docker]

Issue: https://www.pivotaltracker.com/story/show/144646649

- [ ] It works!
- [ ] Comments provide sufficient explanations for the next contributor
- [ ] Tests cover changes and corner cases
- [ ] Follows Quay syntax patterns and format
2017-05-22 11:32:28 -04:00
Charlton Austin
993f2a174c feat(full-stack): disable notifications after 3 failures
This stops notifications from firing over and over again if they are repeatedly failing.

[TESTING -> locally with docker compose, DATABASE MIGRATION -> there is a single migration]

Issue: https://www.pivotaltracker.com/story/show/b144646649n

- [ ] It works!
- [ ] Comments provide sufficient explanations for the next contributor
- [ ] Tests cover changes and corner cases
- [ ] Follows Quay syntax patterns and format
2017-05-19 16:58:46 -04:00
Charlton Austin
b40ad361db style(workers): add in line
there should be two lines between functions and other code

Issue: https://www.pivotaltracker.com/story/show/b144646649n

- [ ] It works!
- [ ] Comments provide sufficient explanations for the next contributor
- [ ] Tests cover changes and corner cases
- [ ] Follows Quay syntax patterns and format
2017-05-19 16:58:22 -04:00