logstash logstash-output-http send entire event - without copying under a new root field?
richard-mauri opened this issue · 1 comments
Can we send the entire event to http output?
I'm using logstash 7.17.6 with logstash-output-http 5.50
We want to send the complete event that is formed by the filter to an http endpoint.
The logstash pipeline outputs the event to stdout/rubydebug and elasticsearch and then to http.
There are many ways to configure the logstash-output-http, but all the documentation and examples are refering to specific fields within thbe event.
We have applications that often send large payloads.
Imagine large exception stack traces and other fields in the event.
At the moment, Ihave a ruby filter script iterating over all the k/v in the event and then calling event.set("[@metadata][mytopleveleventfield]", ....) This forms a single field that I can pass to the http output.
The iterating over event k,v and forming a new event field is wasting cpu and memory.
I wonder if the iterative event.set to form the new tree is cuausing additional memory copies?
Just like stdout/rubydebyg or elasticsearch output; I want the entire event w/o all the copying.
We are seeing lots of gc and I think this copying approach is contributing to that.
Can we send the entire event to http output?
- it does work !