/airflow-covid-data

Sample Airflow DAGs to load data from the CovidTracking API to Snowflake via an AWS S3 intermediary.

Primary LanguagePythonApache License 2.0Apache-2.0

Airflow DAGs to Aggregate COVID Data

This repo contains DAGs to get daily state-related COVID data from the covidtracking.com API, push that data as a series of CSV files to Amazon S3, and then load those CSVs downstream into Snowflake tables using the S3 to Snowflake Transfer Operator.

These DAGs are mostly used for Airflow workshops and training sessions. This repo is also set to deploy to Astronomer via a CI process defined in Github Actions- you may need to change either the registry address or service account key if builds are failing.

Tools Used