logstash-plugins/logstash-output-kafka

Support to write to Kafka Headers

Opened this issue · 6 comments

jguay commented

Current functionality :
Kafka output plugin can only write messages with various fields per event (including @metadata)

Requested feature :
Kafka v0.11.0.0 adds support for custom headers.
Request is for the ability to write to the kafka messages headers

Versions :
Logstash 6.7.0
logstash-output-kafka 7.3.2
Kafka cluster 2.x

Similar issue raised for input plugin here

arijeetmkh commented

+1 These headers are typically used for transport routing, filtering etc, which can be very effective in putting up some logic to achieve these goals without having to parse the message body.

Original Apache Kafka Headers Implementation Discussion/Motivation: https://cwiki.apache.org/confluence/display/KAFKA/KIP-82+-+Add+Record+Headers

  • 👍19
agrawalvinay commented

Do we have an option in Kafka input plugin to consider the headers field as well along with the message now ?

John-Athan commented

This functionality is still not implemed, right?

amitaiblevy commented

Any progress here? This functionality is very important to us.

whipfeng commented

There's still no support for this, right？

weichenongithub commented

hope this feature can be implemented soon