Released latest version 1.0.8 of the Receiver
dibbhatt opened this issue · 0 comments
dibbhatt commented
Released latest version 1.0.8 of Receiver based Kafka Consumer for Spark Streaming.
Receiver is compatible with Kafka versions 0.8.x, 0.9.x and 0.10.x and All Spark Versions
Available at Spark Packages : https://spark-packages.org/package/dibbhatt/kafka-spark-consumer
Salient Features :
- End to End No Data Loss without Write Ahead Log
- ZK Based offset management for both consumed and processed offset
- No dependency on WAL and Checkpoint for recovery on Driver failure
- In-built PID Controller for Rate Limiting and Backpressure management
- Custom Message Interceptor
Please refer to https://github.com/dibbhatt/kafka-spark-consumer/blob/master/README.md for more details