Datapipeline using Airflow
The Datapipeline is bulit to extract the weather data from API and push the data to Azure SQL Database
Based on Airflow official Image [apache/airflow:latest-python3.8] Install Docker
Install Docker Compose
Installation
setp 1: Clone the repo git clone build the docker compose docker-compose up -d docker contianer up docker-compose up
Usage
Open the local port Airflow: localhost:8080 After the pipeline executed down the docker compose docker-compose down