dibbhatt/kafka-spark-consumer
High Performance Kafka Connector for Spark Streaming.Supports Multi Topic Fetch, Kafka Security. Reliable offset management in Zookeeper. No Data-loss. No dependency on HDFS and WAL. In-built PID rate controller. Support Message Handler . Offset Lag checker.
JavaApache-2.0
Issues
- 1
- 2
Can a higher version of kafka be supported
#71 opened by jasonhuxinlei - 1
example doesn't build
#68 opened by IRobL - 2
Trying to fetch Multi topic In Local , But It is showing warning like this
#69 opened by parthasarathy93 - 0
Manipulation of offsetRanges in each batch
#66 opened by Bonnie16 - 11
How to use the application to handle multi topic
#40 opened by zdxMike - 28
- 22
- 1
Does this support spark structured streaming
#62 opened by puneetloya - 5
It works well in local model,but when I submit it in cluster model,the fixed rate is too small
#61 opened by yangcong3643 - 9
- 8
The Spark Streaming can not read kafka message
#60 opened by wowojyc - 20
May I have a Scala sample of messageHandler to filter out some playload which includes some strings?
#59 opened by yufuxin - 8
Why appear this exception information?
#49 opened - 3
How to use in kerberized context ?
#51 opened by StudioEtrange - 16
Consumer commit not proceeding the offset anymore
#33 opened by krantig - 12
- 11
Not working with Spark 2.2.0
#50 opened by swrholdj - 18
How to recover the failed receiver on a partition which has exception of " Offsets out of range with no configured reset policy for partitions:"
#57 opened by yufuxin - 5
- 13
After long time running, the processing time of "ProcessedOffsetManager.persists(partitonOffset_stream, props)" incresing.
#56 opened by yufuxin - 7
Kafka Headers Support
#55 opened by agolomoodysaada - 6
Exception: Could not compute split, block not found
#53 opened by MLNW - 1
error: object ProcessedOffsetManager is not a member of package consumer.kafka
#45 opened by Srinivasan-pp - 2
- 7
- 43
- 5
ReceiverLauncher.launch can not get data
#43 opened by zlkkuu - 2
- 2
Consumer how to use Kafka utilities like ConsumerOffsetChecker and get to show on Kafka Tool
#34 opened by krantig - 9
PartitionManager not writing Offset to Zookeeper if a record fetched from Kafka only contain one line.
#41 opened by ChorPangChan - 2
Contributing Receiver based Low Level Kafka Consumer from Spark-Packages to Apache Spark Project
#27 opened by dibbhatt - 0
Released latest version 1.0.8 of the Receiver
#31 opened by dibbhatt - 3
mvn repository you provided is not working
#37 opened by zacker330 - 2
Python api?
#36 opened by manugarri - 2
work fine few days, then can't commit offerset, no error log, and no info log about PartitionManager
#35 opened by cyinll - 10
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /brokers/topics/load/partitions
#16 opened by justinjoseph89 - 3
Some jobs remain in processing state forever
#26 opened by sorabh89 - 5
- 7
exception on simple example
#29 opened by fabiofumarola - 2
Static zookeeper hosts configuration
#28 opened by muuki88 - 6
How to decode byte messages from Kafka
#24 opened by fzschornack - 5
Running application on 3 node cluster, 3 instances of the same application are shown with different user
#22 opened by sorabh89 - 21
Unable to receive data
#25 opened by sorabh89 - 7
- 22
getting GC : outofmemory exception
#21 opened by sorabh89 - 10
- 2
Kafka apache spark Streaming recive 0
#19 opened by sidahmedbenkhaoua - 4
Not able to get streaming data
#18 opened by sorabh89 - 6
MEMORY_ONLY, MEMORY_ONLY_* storage levels
#17 opened by jedisct1