/realtime-analytics-fastapi-kafka

2. Real-time Analytics:

  • GitHub Repo Name: realtime-analytics-fastapi-kafka
  • MVP Goal: Simulate data streams, push them to Kafka, and verify they are being received.
  • Key Components:
    • Kafka:
      • A simple Kafka setup (using Docker).
      • A topic for the data stream.
    • FastAPI:
      • A single endpoint to receive simulated data and publish to Kafka.
    • Simple data Generator:
      • A script that creates some random data to be sent to the API
  • Steps:
    1. Dockerize Kafka: Set up Kafka and Zookeeper using docker-compose.
    2. Develop FastAPI Application: Create the API endpoint and install the python kafka library.
    3. Create producer Script: Implement script to send random data to the API.
    4. Verify Kafka Data: Check if data has been sent to Kafka using a client or another producer.
    5. Documentation: Add README with setup, how to use producer and verify Kafka topic has received data.