[Feature request] Kafka to Kafka

[worldtiki]

The current implementation only supports tracing data from an http input.
Adding the ability to read data from Kafka would cover more market space.

[azhmaylik]

The current implementation only supports tracing data from an http input.
Adding the ability to read data from Kafka would cover more market space.

What do you mean by reading from Kafka ? Do we want to report kafka client metrics?

[worldtiki]

Hi @azhmaylik.
Zipkin can use Kafka as a message bus. Currently Pitchfork only accepts spans if they are sent via http. This feature is to read messages from a topic in zipkin format and pushing them to the haystack topic.

[codefromthecrypt]

incidentally, that is something folks had as a prior use case. we had this, but it died due to lack of maintenance https://github.com/openzipkin-attic/zipkin-sparkstreaming that said, recently folks have keen interest in this continuing, minus spark bias, as people often use zipkin server as a sort of rosetta stone and want to to some staged things https://cwiki.apache.org/confluence/display/ZIPKIN/2018-09-20+Filtering%2C+Aggregations+and+Proxies+at+Salesforce For example, this was something similar in concept on that topic (pun intended), though unlike haystack, we are unlikely to split streams in the main server.. right not we are considering kafka-kafka or something else (proxy), but not more than simple transforms prior openzipkin/zipkin#2060 hope the context helps!

[eleduardo]

I have some of this code in the context of haystack. It does zipkin v1 to haystack protobuf. I'll add a PR for this feature and have some of you folks review.