Redis::TimeoutError
incorvia opened this issue · 5 comments
I have Redis::TimeoutError's coming from:
sidekiq-throttled (0.17.0) lib/sidekiq/throttled/fetch.rb in block in brpop at line 81
It seems like throttled sets a 2 second timeout for a response on a blocking (brpop) operation. I'm trying to understand if these are leading to a loss of work or not. Is sidekiq throttled just checking and it finds an empty queue so this times out? Is it possible the job is pulled off the sidekiq queue, lost in transit, and the job is lost on timeout? Or is this more that sidekiq is just checking and finding an empty queue but blocking until timeout and nothing really is a problem here..
2 seconds timeout is the default in basic fetch of sidekiq. brpop just blocks the connection for a given timeout until any of the given queues will have a job to pop. In other words, the only way job might be lost is if it was fetched (popped from the queue) and the worker died of OOM, for example.
Sidekiq 7 seems to set timeouts at 1 sec, upon installing throttled i was getting sidekiq client timeouts, i just changed it 5 and no more errors, this is something we may want to look at.
I'm working on massive fetch class refactoring along with some cleanups for the 1.0.0 release that should fix this (831f8b5#diff-aa76f769de4b8b43696ee5d2a1aff0721708eb8561960bbc2005d54ef1231ba4)
Should be fixed as of 1.0.0.alpha