Consumer thread interrupt
wojozer opened this issue · 4 comments
Hello amazing sneaker,
Hope everything is OK with you!
I ran into a problem where the process takes a long time when the consumer processes the messages. When the number of messages is too high, the consumer thread probability will break and retry without any exception. I have modified the timeout time without effect.
Strangely, I modified the number of workers, which defaults to 4, and I modified it to 2 and it no longer happens. What is the reason for this? I just can't understand it.
The CPU is 4-core.
That suggests that your CPU resources are insufficient for the consumer workload and four workers * N threads is too much for them but two workers * N threads is OK. The effective prefetch value is another factor that can potentially be relevant.
Thanks @michaelklishin ,
How can I be sure that this is the cause? Because I don't see any exception logs occurring, and the consumer is retrying.
Also I see that the CPU monitor is always in a healthy state. threads
is set to 1.
WORKER_OPTIONS = {
ack: true,
threads: 1,
prefetch: 1,
heartbeat: 10,
routing_key: routing_key,
max_retries: 6,
arguments: {
'x-dead-letter-exchange' => "#{queue_name}.dlx",
'x-dead-letter-routing-key' => routing_key,
},
}.freeze
Sneakers.configure amqp: rabbitmq_url,
handler: SneakersHandlers::ExponentialBackoffHandler,
metrics: Sneakers::Metrics::NewrelicMetrics.new,
workers: 4,
......
Best Regards!
Resource exhaustion may result in timeouts without any exceptions, or any exceptions that provide enough clues.