This repo contains demo DAGs covered in Astronomer's Intro-to-Airflow webinar.
The easiest way to run these example DAGs is to use the Astronomer CLI to get an Airflow instance up and running locally:
- Install the Astronomer CLI
- Clone this repo somewhere locally and navigate to it in your terminal
- Initialize an Astronomer project by running
astro dev init - Start Airflow locally by running
astro dev start - Navigate to localhost:8080 in your browser and you should see the tutorial DAGs there
This repo contains 6 example DAGs to help you get started with Airflow:
example-dag-basic.py: contains 3 basic Python tasks defined with the TaskFlow API to show DAG syntax.example-dag-advanced.py: contains a DAG with different operators that demonstrates complex branching and task groups.dependencies_example.py: contains a DAG with BashOperators to show implementing complex task dependencies.etl_example_standard_dag.py: contains a basic ETL workflow implementing using standard PythonOperators and demonstrating passing data between tasks.etl_example_taskflow_dag.py: contains a basic ETL workflow implemented using TaskFlow API Python decorators.data_quality_example_dag.py: contains Common SQL provider tasks which complete data quality checks on a Snowflake database.
The exercises.md file in the root of this directory contains exercises to help you get more comfortable with Airflow and build on the examples in this repo.
For more instruction on getting started with Airflow and working with these examples, we recommend the following resources:
- Intro to Airflow webinar walks through this repo in detail and answers audience questions.
- Astronomer Webinars contains all of our webinar and LIVE events on various Airflow topics.
- Astronomer Learn contains concept docs and tutorials for basic through advanced use cases.