elt
There are 336 repositories under elt topic.
apache/airflow
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
airbytehq/airbyte
The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
apache/doris
Apache Doris is an easy-to-use, high performance and unified analytics database.
dbt-labs/dbt-core
dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.
apache/seatunnel
SeaTunnel is a next-generation super high-performance, distributed, massive data integration tool.
mage-ai/mage-ai
🧙 Build, run, and manage data pipelines for integrating and transforming data.
cloudquery/cloudquery
The developer first cloud governance platform
apache/flink-cdc
Flink CDC is a streaming data integration tool
rudderlabs/rudder-server
Privacy and Security focused Segment-alternative, in Golang and React
dlt-hub/dlt
data load tool (dlt) is an open source Python library that makes data loading easy 🛠️
quarylabs/quary
Open-source BI for engineers
TobikoData/sqlmesh
Efficient data transformation and modeling framework that is backwards compatible with dbt.
meltano/meltano
Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.
ucbepic/docetl
A system for agentic LLM-powered data processing and ETL
dataform-co/dataform
Dataform is a framework for managing SQL based data operations in BigQuery
kuwala-io/kuwala
Kuwala is the no-code data platform for BI analysts and engineers enabling you to build powerful analytics workflows. We are set out to bring state-of-the-art data engineering tools you love, such as Airbyte, dbt, or Great Expectations together in one intuitive interface built with React Flow. In addition we provide third-party data into data science models and products with a focus on geospatial data. Currently, the following data connectors are available worldwide: a) High-resolution demographics data b) Point of Interests from Open Street Map c) Google Popular Times
raystack/optimus
Optimus is an easy-to-use, reliable, and performant workflow orchestrator for data transformation, data modeling, pipelines, and data quality management.
artie-labs/transfer
Database replication platform that leverages change data capture. Stream production data from databases to your data warehouse (Snowflake, BigQuery, Redshift, Databricks) in real-time.
datazip-inc/olake
Fastest open-source tool for replicating Databases to Apache Iceberg or Data Lakehouse. ⚡ Efficient, quick and scalable data ingestion for real-time analytics. Supporting Postgres, MongoDB and MySQL
slingdata-io/sling-cli
Sling is a CLI tool that extracts data from a source storage/database and loads it in a target storage/database.
Datavault-UK/automate-dv
A free to use dbt package for creating and loading Data Vault 2.0 compliant Data Warehouses (powered by dbt, an open source data engineering tool, registered trademark of dbt Labs)
gouline/dbt-metabase
dbt + Metabase integration
vmware/versatile-data-kit
One framework to develop, deploy and operate data workflows with Python and SQL.
osalvador/ReplicaDB
ReplicaDB is open source tool for database replication, designed for efficiently transferring bulk data between relational and non-relational databases
astronomer/astro-sdk
Astro SDK allows rapid and clean development of {Extract, Load, Transform} workflows using Python and SQL, powered by Apache Airflow.
aws-samples/aws-etl-orchestrator
A serverless architecture for orchestrating ETL jobs in arbitrarily-complex workflows using AWS Step Functions and AWS Lambda.
DataRecce/recce
The data-validation toolkit for enhanced dbt (data build tool) PR review
cuebook/cuelake
Use SQL to build ELT pipelines on a data lakehouse.
datacoves/dbt-coves
CLI tool for dbt users to simplify creation of staging models (yml and sql) files
airbytehq/PyAirbyte
PyAirbyte brings the power of Airbyte to every Python developer.
umitkaanusta/reddit-detective
Play detective on Reddit: Discover political disinformation campaigns, secret influencers and more
unytics/airbyte_serverless
Airbyte made simple (no UI, no database, no cluster)
173TECH/sayn
Data processing and modelling framework for automating tasks (incl. Python & SQL transformations).
faros-ai/airbyte-connectors
Airbyte connectors (sources & destinations) + Airbyte CDK for JavaScript/TypeScript
transferia/transferia
Open Source Cloud Native Ingestion engine
yokawasa/databricks-notebooks
Collection of Sample Databricks Spark Notebooks ( mostly for Azure Databricks )