/twitterStreamingSparkKafkaDemo

a demo project to Analyze most popular twitter hashtags using Java 8 Spring-Boot Spark Streaming Kafka & Docker Demo.

Primary LanguageJavaMIT LicenseMIT

Twitter Streaming Analyze using Java 8, Spark Streaming, Kafka,

A demo project using Spark Streaming to analyze popular hashtags from twitter. The data comes from the Twitter Streaming API source and is fed to Kafka. The consumer com.twitter.producer.service receives data from Kafka and then processes it in a stream using Spark Streaming.

Requirements

Quickstart guide

  1. Change Twitter configuration in \producer\src\main\resources\application.yml with your API Key, client Id and Secret Id.

  2. Run the kafka image using docker-compose(keep in mind that the kafka image need to pull zookeper too):

~> docker-compose -f producer/src/main/docker/kafka-docker-compose.yml up -d   
  1. Check if ZooKeeper and Kafka is running (from command prompt)
~> docker ps 
  1. Run poducer and consumer app with:
~> mvn spring-boot:run

References