Do something with the server response
Opened this issue · 2 comments
It would be incredibly useful for me if I could do something with the server's reply. The tool to which I am posting events responds with a tracking GUID that I would like to catalog somehow. Best case is that there's an event.set in the output-http plugin that adds this server response to the current event. I can subsequently send the enriched event somewhere else with another output plugin later in the configuration - like sending it to Elasticsearch.
This would allow us to tell the consumer of our data that they did receive the event and provide them with the GUID they generated.
Hey Mike, yes, your use case makes total sense.
What kind of throughput are you processing?
There may be a way to solve this using an HTTP filter instead, although we don't have such a plugin officially supported at the moment. You can look out in the community though, like this one. The idea would be that the filter performs the request that posts to this API, and you instruct the filter to grab this GUID from the response and enhance your event with it.
One issue with this approach is that currently, filters are pretty much one event at a time, regardless of whether they implement multi_filter. This should get better soon, though. Logstash 6.3's --experimental-java-execution=true should help in multi_filter getting more than one event at a time. I think the Java execution engine has issues on 6.2.
Our throughput would be low, on the order of 1->10 events per minute.
On the surface, the http filter looks good, but I need the full functionality of the HTTP output plugin - auth, certs, etc. Perhaps, there could be a configurable destination to send the plain-formatted text from the response: like response_file_path (path), response_udp_ip (string), response_udp_port(int), response_tcp_ip (string), response_tcp_port (int), response_include_message (Boolean), etc.
From there, I can set up another pipeline to monitor that file or listen on the specified port on loopback as a new input.