messages drop under load
Closed this issue · 2 comments
Hi,
when a lot of log messages are created the CostBoundConcurrentQueue starts dropping messages on the floor. You can find this logic at at this line
I'd like to.
- Completely remove the CostBoundConcurrentQueue, the locking logic that occurs causes some slow downs inside our app
- There is no reason why logs should be dropped on the floor.
- I'd also like to create a
int?MinFlushInterval to tell the queue to start flushing early if the value is not null and greater than 0
@latkin I'd like to hear your thoughts on these 3 items, before really making something.
It is by design that the buffered appender enforces a maximum size. This is to prevent high log load from OOM-ing your app. The max size can be adjusted with the parameter MaxQueueSizeBytes. Is there a problem with setting that to, say, 4GB? The default appears to be 1MB.
If you have optimizations to the locking scheme, or new parameters/features then I'd be open to bringing those in.
Okay when you frame it that way it actually makes 100% sense that it would drop the logs I am going to figure out how to improve the performance around the lock I believe we may also end up adding the other feature min flush interval