/pipeliner

Large Scale Twitter Tweets Data Pipeline with Sentiment Analysis that run in your Local

Primary LanguagePython

Pipeliner

My simple project to learn data engineering

Architecture

Pipeliner Architecture

How to run it in local

  1. Run the Script manage.sh - wrapper for docker-compose works as a managing tool.

    • Build project infrastructure
    ./manage.sh up
    • Stop project infrastructure
    ./manage.sh stop
    • Delete project infrastructure
    ./manage.sh down