Better documentation/defaults for high throughput scenarios
Opened this issue · 1 comments
robbavey commented
When using this plugin in certain scenarios, such as HTTP based logstash->logstash communication, or when sending to endpoints with large amounts of latency, using defaults can have a significantly detrimental impact on performance.
We should improve the documentation to include recommendations (http_compression=>true, format=>json_batch, increased batch sizes) to enable improved performance for these scenarios.
Other potential improvements include setting format=>json_batch as default in a future major version, or logging a warning that single http events are being sent via this output when users specify a large batch size in their logstash configuration.
Catwoolfii commented
@robbavey, hi!
How is the size of json_batch determined?