morganstanley/modern-cpp-kafka

How to join consumer group so Zookeeper can restart dead consumer

adammpolak opened this issue · 3 comments

The examples of a consumer providing a heartbeat to Zookeeper online has been in Java.

Is there an example of how to create a C++ consumer app that can be monitored and healed by zookeeper?

It seems people took a great effort to remove the dependency towards zookeeper.
https://www.confluent.io/blog/removing-zookeeper-dependency-in-kafka/
https://blogs.apache.org/kafka/entry/what-s-new-in-apache5

Thus I don't think the underlying C library, i.e. librdkafka (https://github.com/edenhill/librdkafka) would provide any support for zookeeper related stuff.

@kenneth-jia

Thank you.

How does a consumer group get healed in this new approach?

I don't quite understand your question, -- what does "get healed" mean? The failed consumer would not be restarted (unless you add additional logic in your application to manage the process state), while the partitions (which were assigned to the failed consumers) would be rebalanced to the other consumers (within the same consumer.group).