streaming-data-pipeline

Streaming pipeline repo for data engineering training program

See producers and consumers set up README in their respective directories

local environment setup

Make sure you have sbt installed. Make sure you have docker installed.

Run ./sbin/buildAndRunLocal.sh to create Docker containers for running and testing this setup on your local machine