reginold/airflow-eks-prod

airflow-materials-aws from udemy

The tech stack: Airflow, Helm chart, AWS(CI/CD, EFS, RDS, S3,EKS)

• Configuring the EKS cluster following best practices

• Deploying automatically changes with GitOps

• Using Helm to configure and set up Airflow on Kubernetes

• Configuring the official Helm chart of Airflow to use the Kubernetes Executor and many different features

• Deploying DAGs in Airflow with Git-Sync and AWS EFS

• Deploying DAGs/Airflow through CI/CD pipelines with AWS CodePipeline

• Testing your DAGs automatically

• Securing your credentials and sensitive data in a Secret Backend

• Enabling remote logging with AWS S3

• Creating 3 different environments dev/staging and prod

• Making the production environment scalable and highly available