Joseph Schorr
91e9fe8050
Optimize the logs archiving worker to not issue extremely long queries
...
Fixes https://jira.coreos.com/browse/QUAY-1371
2019-02-26 14:37:42 -05:00
Joseph Schorr
cdb49dbfd3
Add LogEntry3 table without the extra indexes and switch to writing to it
2019-01-03 13:50:43 -05:00
Joseph Schorr
204eb74c4f
Put aggregated log query and log exports behind feature flags
2019-01-02 16:04:15 -05:00
Joseph Schorr
58d7dd07b9
Revert the changes to drop LogEntry2 table definition
2019-01-02 13:29:35 -05:00
Joseph Schorr
6a94eba1a2
Remove all references to LogEntry2 from the code
...
The migration to actually drop the table will go in after this code has been pushed to production
2018-12-11 15:54:16 -05:00
Joseph Schorr
a4fff886a6
Handle the case where the log metadata JSON cannot be parsed
...
This can happen if the JSON overflowed the text field in the table, for example
Fixes https://sentry.io/coreos/backend-production/issues/670957592/
2018-09-07 13:34:22 -04:00
Sam Chow
84f604739f
Add log rotation threshold configuration
2018-06-29 17:16:44 -04:00
Joseph Schorr
a007332d4c
Temporarily change to storing logs in a new LogEntry2 table
...
This will prevent us from running out of auto-incrementing ID values until such time as we can upgrade to peewee 3 and change the field type to a BigInt
Fixes https://jira.coreos.com/browse/QUAY-943
2018-05-18 20:15:16 -04:00
Joseph Schorr
8a4d583f65
Disable default debug logs for workers
...
https://coreosdev.atlassian.net/browse/QUAY-771
2017-08-24 14:25:51 -04:00
Joseph Schorr
fa21e42ffb
Add default location for action log archiving
...
Prevents the logs from being written to the preferred storage, which means they will jump around
2017-07-10 12:37:44 +03:00
Antoine Legrand
3c99928a27
Add log JSON formatter
2017-06-07 00:02:52 +02:00
Joseph Schorr
aa7c87d765
Fix locking via RedLock
...
Fixes #1777
2016-08-29 16:06:26 -04:00
Jimmy Zelinskie
128b0cd38c
logrotateworker: archive every 24 hours
2016-04-18 13:02:30 -04:00
Jimmy Zelinskie
ef65822410
logrotateworker: perf optimizations
...
This removes our needless transaction, only calculates the cutoff date
once, removes the logs generator, and uses a tested optimal
MIN_LOGS_PER_ROTATION.
2016-04-15 16:51:17 -04:00
Jimmy Zelinskie
3d190b786f
userfiles: make handler optional
2016-04-15 13:56:07 -04:00
Jimmy Zelinskie
c7c52e6c74
logrotateworker: save to storage via userfiles
2016-04-14 13:29:29 -04:00
Joseph Schorr
dc8f9713f8
Change logs worker to use a global lock in the inner loop and move storage out of the transaction
2016-03-24 14:09:48 -04:00
Jimmy Zelinskie
ee705fe7a9
vastly simplify log rotation
2016-02-09 18:20:14 -05:00
Jimmy Zelinskie
e1f955a3f6
add a log rotation worker
...
Fixes #609 .
2015-12-16 17:22:28 -05:00
