/kafka-flink-to-cassandra-data-pipeline

Primary LanguageScala

kafka-flink-to-cassandra-data-pipeline

Requirements

Docker or colima, sbt.

Steps to run

  1. Run docker or colima

colima start --cpu 4 --memory 8

  1. Run kafka, zookeeper, kafka-ui, cassandra via docker-compose

docker-compose up -d

  1. Wait some time when all be prepared

  2. Connect to cassandra

docker exec -it kafka-flink-cassandra-cassandra-1 cqlsh -u cassandra -p cassandra

  1. Create keyspace and table

CREATE KEYSPACE IF NOT EXISTS example WITH replication = {'class': 'SimpleStrategy', 'replication_factor': '1'};

CREATE TABLE IF NOT EXISTS example.wordcount (word text,count bigint,PRIMARY KEY(word));

  1. Install deps and run project

sbt update && sbt run

UI

Kafka-ui

Flink