shreyamedehal/datastreaming

Target Ready - Spark/Scala - Team 1 - Kafka Data Streaming

datastreaming_team1

Target Ready - Spark/Scala - Team 1 - Kafka Data Streaming

Data Streaming - Kafka

Use case:

Produce a kafka topic,post messages and consume data.

Steps:

1. Create a dataset of 1 million records and store it in your local machine.
2. Read the dataset and store data in any database (Postgress/Mongo).
3. Create a Kafka topic.
4. Read the dataset using spark and post these 1 million rows as messages into the Kafka topic.
5. Read the messages from the topic and store it any database or text format.