/TRAC-Price-Streaming-Pipeline

Primary LanguageJupyter Notebook

Stock Market Kafka Real Time Data Engineering Project

Introduction

In this project, I executed an End-To-End Data Engineering Project on Real-Time Cryptocurrency Market Data using Kafka.

I used different technologies such as Python, Amazon Web Services (AWS), Apache Kafka, Glue, Athena, and SQL.

Architecture

Technology Used

  • Programming Language - Python
  • Amazon Web Service (AWS)
  1. S3 (Simple Storage Service)
  2. Athena
  3. Glue Crawler
  4. Glue Catalog
  5. EC2
  • Apache Kafka

Dataset Used

Coinbase API - https://api.exchange.coinbase.com/products/TRAC-USD/ticker