Batches should be published in paralel when using multiple pipeline workers
Closed this issue · 0 comments
honzakral commented
Given that this plugin has concurrency :single
(meaning a single instance will be shared by workers) the behavior of Batcher.enqueue
causes the sending to be serialized across workers causing a bottleneck.
I believe the desired behavior is to have all workers sending data independently of each other whenever they have a batch ready.