/Kafka-Twitter-Streaming

Kafka Producer and Consumer for streaming real time tweets and storing it in ElasticSearch database. Also includes a module with sample code for using Consumer and Producer API.

Primary LanguageJava

Kafka Twitter Streaming App

  • Kafka Producer to stream the live tweets and feed it into the Kafka cluster.
  • Kafka Consumer to consume the data from Kafka and store it into the ElasticSearch database.

Instructions

Start Zookeeper

zookeeper-server-start config\zookeeper.properties

Start Kafka Server

kafka-server-start config\server.properties

Start ElasticSearch Server

elasticsearch

Create Kafka Topic

kafka-topics --zookeeper 127.0.0.1:2181 --topic twitterApp --create --partitions 6 --replication-factor 1