JMLX42/cocorico

Webhooks are called one after the other

YoannAubineau opened this issue · 3 comments

YoannAubineau commented

Currently, webhooks are called synchronously. This is because we process webhook messages from RabbitMQ one by one through a single channel (a RabbitMQ channel is synchronous by nature).

Potential solutions are:

  • consume multiple messages at the same time and acknowledge them when associated webhook is successfully called (this might imply switching to the "promise" paradigme);
  • communicate with RabbitMQ through multiple channels in parallel ;
  • start multiple instances of the webhook worker (pm2 might be helpful here).
JMLX42 commented

Thanks for reporting this.
Not a big issue for now, right?

You should check this out: https://nodejs.org/api/cluster.html

YoannAubineau commented

No, it's not a big deal for now. But it might become a problem if multiple votes are ran at the same time and one of those vote's remote server is unreachable. Each failed attempt to call an unreachable server will delay all subsequent messages from all votes by 30 seconds (or whatever value the TCP TTL is set to).

This makes me think that an other potential solution could be to shard webhook messages by vote: dedicate a queue and a worker for each active vote. I am not a big fan of this approach though as creating queues and spawning workers dynamically seems like a recipe for problems.

JMLX42 commented

One queue per vote means we have to sync votes and their respective queue. Maybe we should... add a queue for that? Not a good idea indeed.

I'd rather have a cluster with a dozen forks.