airflow
There are 2164 repositories under airflow topic.
apache/airflow
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
argoproj/argo-workflows
Workflow Engine for Kubernetes
apache/dolphinscheduler
Apache DolphinScheduler is the modern data orchestration platform. Agile to create high performance workflow with low-code
PipelineAI/pipeline
PipelineAI
orchest/orchest
Build data pipelines, the easy way 🛠️
puckel/docker-airflow
Docker Apache Airflow
jghoman/awesome-apache-airflow
Curated list of resources about Apache Airflow
WeBankFinTech/DataSphereStudio
DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.
elyra-ai/elyra
Elyra extends JupyterLab with an AI centric approach.
teamclairvoyant/airflow-maintenance-dags
A series of DAGs/Workflows to help maintain the operation of Airflow
san089/Udacity-Data-Engineering-Projects
Few projects related to Data Engineering including Data Modeling, Infrastructure setup on cloud, Data Warehousing and Data Lake development.
san089/goodreads_etl_pipeline
An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.
ajbosco/dag-factory
Dynamically generate Apache Airflow DAGs from YAML configuration files
abhishek-ch/around-dataengineering
A Data Engineering & Machine Learning Knowledge Hub
damklis/DataEngineeringProject
Example end to end data engineering project.
OBenner/data-engineering-interview-questions
More than 2000+ Data engineer interview questions.
iusztinpaul/energy-forecasting
🌀 𝗧𝗵𝗲 𝗙𝘂𝗹𝗹 𝗦𝘁𝗮𝗰𝗸 𝟳-𝗦𝘁𝗲𝗽𝘀 𝗠𝗟𝗢𝗽𝘀 𝗙𝗿𝗮𝗺𝗲𝘄𝗼𝗿𝗸 | 𝗟𝗲𝗮𝗿𝗻 𝗠𝗟𝗘 & 𝗠𝗟𝗢𝗽𝘀 for free by designing, building and deploying an end-to-end ML batch system ~ 𝘴𝘰𝘶𝘳𝘤𝘦 𝘤𝘰𝘥𝘦 + 2.5 𝘩𝘰𝘶𝘳𝘴 𝘰𝘧 𝘳𝘦𝘢𝘥𝘪𝘯𝘨 & 𝘷𝘪𝘥𝘦𝘰 𝘮𝘢𝘵𝘦𝘳𝘪𝘢𝘭𝘴
alanchn31/Data-Engineering-Projects
Personal Data Engineering Projects
raystack/optimus
Optimus is an easy-to-use, reliable, and performant workflow orchestrator for data transformation, data modeling, pipelines, and data quality management.
airflow-helm/charts
The User-Community Airflow Helm Chart is the standard way to deploy Apache Airflow on Kubernetes with Helm. Originally created in 2017, it has since helped thousands of companies create production-ready deployments of Airflow on Kubernetes.
ankurchavda/streamify
A data engineering project with Kafka, Spark Streaming, dbt, Docker, Airflow, Terraform, GCP and much more!
astronomer/astronomer-cosmos
Run your dbt Core projects as Apache Airflow DAGs and Task Groups with a few lines of code
rjurney/Agile_Data_Code_2
Code for Agile Data Science 2.0, O'Reilly 2017, Second Edition
josephmachado/beginner_de_project
Beginner data engineering project - batch edition
hgrif/airflow-tutorial
Airflow basics tutorial
gocardless/airflow-dbt
Apache Airflow integration for dbt
villasv/aws-airflow-stack
Turbine: the bare metals that gets you Airflow
andreax79/airflow-code-editor
A plugin for Apache Airflow that allows you to edit DAGs in browser
huseinzol05/Gather-Deployment
Gathers Python deployment, infrastructure and practices.
airflow-plugins/Example-Airflow-DAGs
Example DAGs using hooks and operators from Airflow Plugins
astronomer/astro-sdk
Astro SDK allows rapid and clean development of {Extract, Load, Transform} workflows using Python and SQL, powered by Apache Airflow.
teamclairvoyant/airflow-rest-api-plugin
A plugin for Apache Airflow that exposes rest end points for the Command Line Interfaces
apache/airflow-client-python
Apache Airflow - OpenApi Client for Python
cubefs/compass
Compass is a task diagnosis platform for bigdata
pytorch/torchx
TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and support for E2E production ML pipelines when you're ready.
feluelle/airflow-diagrams
Auto-generated Diagrams from Airflow DAGs. 🔮 🪄