Write queue not able to keep up with tasks
Opened this issue · 0 comments
kldavis4 commented
I switched from the MemoryStore to this sqlite implementation because I was running out of memory with a large number of queued tasks in batch mode. I am finding that the _writeQueue length is growing continuously (only dropping a little on occasion). This eventually leads to an out of memory error. Any tips on how to remedy this? I can throttle things on the job generation side, but the motivating reason for adding the work queue was to avoid that.