Hruodperht/lambda-architecture-demo

Developing a Lambda Architecture pipeline using Apache Kafka, Spark Structured Streaming, Redshift, S3, Python

Developing Lambda Architecture

This is a repository for the code found in my series of blog posts on implementing the Lambda Architecture::

• Introduction to Lambda Architecture
• Implementing Data Ingestion using Apache Kafka, Tweepy
• Implementing Batch Layer using Kafka, S3, Redshift
• Implementing Speed Layer using Spark Structured Streaming
• Implementing Serving Layer using Redshift