/datastreaming_team1

Target Ready - Spark/Scala - Team 1 - Kafka Data Streaming

Primary LanguageScala

datastreaming_team1

Target Ready - Spark/Scala - Team 1 - Kafka Data Streaming

Data Streaming - Kafka

Use case:

Produce a kafka topic,post messages and consume data.

Steps:

  1. Create a dataset of 1 million records and store it in your local machine.
  2. Read the dataset and store data in any database (Postgress/Mongo).
  3. Create a Kafka topic.
  4. Read the dataset using spark and post these 1 million rows as messages into the Kafka topic.
  5. Read the messages from the topic and store it any database or text format.