This setup is used and tested in airflow 1.10.10. It's a good idea to use latest version of Airflow 2.3.x+. Need help setting up infrastructure and training your team? Contact me bhavanicodes@gmail.com
The setup files are copied directly from airflow's repo and modified to fit the requirements.
One major change is instead of building the docker image from source, we use pip to install airflow
cd scripts/docker
docker build -t airflow .
Push the image to your favorite registry and get the URL
- Find $IMAGE in the repository
- Find $TAG in the repository
- Change it with the docker image URL and tag respectively
cd scripts/
./kube/deploy.sh -d persistent_mode
The other supported modes is git_mode