High throughput endpoints may fail to set messages as dispatched
Closed this issue · 3 comments
Problem
Currently, the Cleanup queue that is used for tracking items, is a reliable non-concurrent queue. When SetAsDispatched is called, we use EnqueueAsync which obtains the operational lock for enqueuing. This means, that only a single message can be set as dispatched at the same time, limiting a throughput of the endpoint to 1 when executing SetAsDispatched. It's worth to notice that several operations take place during the transaction for setting as dispatch, limiting the throughput and extending lock duration for even longer period of time.
We were able to replicate it using a new test provided by #60 .
Solution
A potential solution could be based on the new ReliableConcurrentQueue. It's a non-locking queue with no TryPop method so it'd require an additional shelf, to put near-now messages that will be collected in a moment. For this purpose, we could reuse already existing queue.
/cc @danielmarbach Please take a look.
👍 ReliableConcurrentQueue was introduced farely recently and by its definition "is designed to deliver high throughput and low latency".