This repository contains the DAG code used in the Regression with Airflow + MLflow use case example.
The DAGs in this repository use the following packages:
This section explains how to run this repository with Airflow. Note that you will need to copy the contents of the .env_example
file to a newly created .env
file. No external connections are necessary to run this repository locally, but you can add your own credentials in the file if you wish to connect to your tools.
Run this Airflow project without installing anything locally.
- Fork this repository.
- Create a new GitHub codespaces project on your fork. Make sure it uses at least 4 cores!
- After creating the codespaces project the Astro CLI will automatically start up all necessary Airflow components and the local MinIO and MLflow instances. This can take a few minutes.
- Once the Airflow project has started, access the Airflow UI by clicking on the Ports tab and opening the forward URL for port 8080. The MLflow instance is accessible at port 5000, the MinIO instance at port 9000.
Download the Astro CLI to run Airflow locally in Docker. astro
is the only package you will need to install locally.
- Run
git clone https://github.com/astronomer/use-case-mlflow.git
on your computer to create a local clone of this repository. - Install the Astro CLI by following the steps in the Astro CLI documentation. Docker Desktop/Docker Engine is a prerequisite, but you don't need in-depth Docker knowledge to run Airflow with the Astro CLI.
- Run
astro dev start
in your cloned repository. - After your Astro project has started. View the Airflow UI at
localhost:8080
, the MLflow UI atlocalhost:5000
and the MinIO UI atlocalhost:9000
.