airflow
There are 3311 repositories under airflow topic.
apache/airflow
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
argoproj/argo-workflows
Workflow Engine for Kubernetes
apache/dolphinscheduler
Apache DolphinScheduler is the modern data orchestration platform. Agile to create high performance workflow with low-code
orchest/orchest
Build data pipelines, the easy way 🛠️
jghoman/awesome-apache-airflow
Curated list of resources about Apache Airflow
puckel/docker-airflow
Docker Apache Airflow
WeBankFinTech/DataSphereStudio
DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.
elyra-ai/elyra
Elyra extends JupyterLab with an AI centric approach.
san089/Udacity-Data-Engineering-Projects
Few projects related to Data Engineering including Data Modeling, Infrastructure setup on cloud, Data Warehousing and Data Lake development.
teamclairvoyant/airflow-maintenance-dags
A series of DAGs/Workflows to help maintain the operation of Airflow
OBenner/data-engineering-interview-questions
More than 2000+ Data engineer interview questions.
san089/goodreads_etl_pipeline
An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.
astronomer/dag-factory
Construct Apache Airflow DAGs Declaratively via YAML configuration files
damklis/DataEngineeringProject
Example end to end data engineering project.
abhishek-ch/around-dataengineering
A Data Engineering & Machine Learning Knowledge Hub
astronomer/astronomer-cosmos
Run your dbt Core or dbt Fusion projects as Apache Airflow DAGs and Task Groups with a few lines of code
alanchn31/Data-Engineering-Projects
Personal Data Engineering Projects
iusztinpaul/energy-forecasting
🌀 𝗧𝗵𝗲 𝗙𝘂𝗹𝗹 𝗦𝘁𝗮𝗰𝗸 𝟳-𝗦𝘁𝗲𝗽𝘀 𝗠𝗟𝗢𝗽𝘀 𝗙𝗿𝗮𝗺𝗲𝘄𝗼𝗿𝗸 | 𝗟𝗲𝗮𝗿𝗻 𝗠𝗟𝗘 & 𝗠𝗟𝗢𝗽𝘀 for free by designing, building and deploying an end-to-end ML batch system ~ 𝘴𝘰𝘶𝘳𝘤𝘦 𝘤𝘰𝘥𝘦 + 2.5 𝘩𝘰𝘶𝘳𝘴 𝘰𝘧 𝘳𝘦𝘢𝘥𝘪𝘯𝘨 & 𝘷𝘪𝘥𝘦𝘰 𝘮𝘢𝘵𝘦𝘳𝘪𝘢𝘭𝘴
ankurchavda/streamify
A data engineering project with Kafka, Spark Streaming, dbt, Docker, Airflow, Terraform, GCP and much more!
raystack/optimus
Optimus is an easy-to-use, reliable, and performant workflow orchestrator for data transformation, data modeling, pipelines, and data quality management.
airflow-helm/charts
The User-Community Airflow Helm Chart is the standard way to deploy Apache Airflow on Kubernetes with Helm. Originally created in 2017, it has since helped thousands of companies create production-ready deployments of Airflow on Kubernetes.
josephmachado/beginner_de_project
Beginner data engineering project - batch edition
rjurney/Agile_Data_Code_2
Code for Agile Data Science 2.0, O'Reilly 2017, Second Edition
fieldryand/goflow
Simple but powerful DAG scheduler and dashboard
andreax79/airflow-code-editor
A plugin for Apache Airflow that allows you to edit DAGs in browser
apache/airflow-client-python
Apache Airflow - OpenApi Client for Python
gocardless/airflow-dbt
Apache Airflow integration for dbt
cubefs/compass
Compass is a task diagnosis platform for bigdata
meta-pytorch/torchx
TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and support for E2E production ML pipelines when you're ready.
hgrif/airflow-tutorial
Airflow basics tutorial
villasv/aws-airflow-stack
Turbine: the bare metals that gets you Airflow
astronomer/astro-sdk
Astro SDK allows rapid and clean development of {Extract, Load, Transform} workflows using Python and SQL, powered by Apache Airflow.
dataplane-app/dataplane
Dataplane is an Airflow inspired unified data platform with additional data mesh and RPA capability to automate, schedule and design data pipelines and workflows. Dataplane is written in Golang with a React front end.
huseinzol05/Gather-Deployment
Gathers Python deployment, infrastructure and practices.
feluelle/airflow-diagrams
Auto-generated Diagrams from Airflow DAGs. 🔮 🪄
airflow-plugins/Example-Airflow-DAGs
Example DAGs using hooks and operators from Airflow Plugins