basic_airflow_project

This project explains about how to create a basic data pipeline using Airflow as an orchestrator for all the data tasks

Pre-Requisite

Plese keep the airflow installation ready with docker and then perform this tasks

Create virtual environment for python 3.8

  conda create -n <environment-name> python=3.8 -y

Enter the virtual environment

    activate <environment-name>

install all dependencies of the this project by

  pip install -r requirements.txt