Joseph Schorr
2d34b89203
Decrease redis timeouts to within the WAMP message window
...
This ensures that if the builder sends a heartbeat, but redis is down, we don't time out the build while waiting to connect or receive. Since Redis data is ephemeral anyway, this should give us more stability in builds if/when redis is down
2018-03-02 16:53:37 -05:00
Joseph Schorr
bbdf9e074c
Add metrics for tracking when instance key renewal succeeds and fails, as well as when instance key *lookup* fails
2018-02-02 11:14:42 -05:00
Joseph Schorr
4ad3682b9c
Make health check failures report their reasons
...
Note that we add a new block with expanded service info, to avoid breaking compatibility with existing callers of the health endpoint
2017-07-19 16:17:02 +03:00
Antoine Legrand
ec847ce613
Switch from expire to delete redis log_entries
2017-03-17 15:35:47 +01:00
Joseph Schorr
1b8820f2e7
Change the append build log method to execute the two calls via one pipelined connection
...
Should reduce the amount of packets used by the build manager
Reference: https://github.com/andymccurdy/redis-py#pipelines
2016-11-18 11:47:16 -05:00
Joseph Schorr
3b994431eb
Auto expire the build status and logs in redis
2016-06-20 13:53:13 -04:00
Joseph Schorr
986d20bcad
Switch to generic RedisError
...
Fixes #1558
2016-06-20 11:20:17 -04:00
Joseph Schorr
04df2410ec
Add better errors if Redis is down
...
Fixes #1497
2016-05-31 15:24:36 -04:00
Matt Jibson
afa119d82e
Set timeout for redis commands
...
fixes #779
2015-11-06 18:48:47 -05:00
Joseph Schorr
c518874ded
I hate Redis!
...
- Remove redis check from our health endpoint in prod entirely
- Have the redis check have a maximum timeout of 1 second
2015-10-22 14:24:42 -04:00
Joseph Schorr
6e0ca735a5
Add a better redis health check that reads and writes
...
This will hopefully catch issues earlier with Redis
2015-09-30 15:23:19 -04:00
Jimmy Zelinskie
716d7a737b
Strip whitespace from ALL the things.
2014-11-24 16:07:38 -05:00
Joseph Schorr
2c5cc7990f
Allow for additional REDIS config such as password and port
2014-10-01 14:16:42 -04:00
Jake Moshenko
74e35f917e
Switch socket_timeout to socket_connect_timeout for redis.
2014-09-23 14:59:00 -04:00
Jake Moshenko
a6225ad34a
Set an aggressive timeout on the redis connections to allow the unit tests to run when communication with redis is disabled.
2014-09-22 18:42:22 -04:00
Jake Moshenko
c01de4a916
Set redis logs entries to expire rather than to immediately delete them to make the logs archiver idempotent.
2014-09-12 13:13:14 -04:00
Jake Moshenko
b55e79e5d3
Merge remote-tracking branch 'origin/master' into waltermitty
2014-09-12 11:45:07 -04:00
Jake Moshenko
8b3a3178b0
Finish the build logs archiver, add handlers for cloud and local that handle gzip encoded archived content.
2014-09-11 15:33:10 -04:00
Joseph Schorr
daa43c3bb9
Add better messaging around pulling of base images when they fail due to invalid or missing credentials
2014-08-18 20:34:39 -04:00
Jake Moshenko
d1f4fbdacc
Split out the redis hostname for user events and build logs as a string config. Modularize the user events and fix all callers.
2014-05-30 14:25:29 -04:00
Jake Moshenko
8b5c781f84
Change the health check to ping the db and the redis build logs.
2014-05-13 18:53:42 -04:00
Jake Moshenko
bcb993a914
Set up the build logs to use our fake build logs on test and local.
2014-05-09 18:45:11 -04:00
Joseph Schorr
795de4235d
Change "cannot connect to redid" to be a raised exception
2014-03-28 14:20:06 -04:00
Joseph Schorr
16d3ddd8cc
Nicely handle the case where we cannot connect to Redis
2014-03-25 13:29:06 -04:00
yackob03
b920a0cb1f
Really try to emulate the logs format with the test logs. Switch to a simplified flat logs format.
2014-02-12 18:58:40 -05:00
yackob03
6fd343741b
Change to the new paging format with the commands available at the top.
2014-02-10 19:12:43 -05:00
yackob03
5270066d6d
Switch to the redis backed build logs and status.
2014-02-03 19:08:37 -05:00
