elt-pipeline
There are 56 repositories under elt-pipeline topic.
dataforgelabs/dataforge-core
DataForge helps data teams write functional transformation pipelines by leveraging software engineering principles
Fozan-Talat/divvy-bikeshare-de-project
An end-to-end data pipeline which extracts divvy bikeshare data from web loads it into data lake and datawarehouse transforms it using dbt and finally , a dashboard to visualize the data using looker studio, the pipeline is orchestrated using prefect
Mg30/pydwt
Modeling tool like DBT to use SQL Alchemy core with a DataFrame interface like
bennyaustin/synapse-dataplatform
A modern data platform implemented on Azure Synapse Analytics using ELT Framework - https://github.com/bennyaustin/elt-framework. Data platform infrastructure provisioned using https://github.com/bennyaustin/iac-synapse-dataplatform
longNguyen010203/FDE-Course-2024-W4-DBT
💻💛Fundamental Data Engineering Course 2024 Week4 Learn DBT Transform Data with Models, Macro, ELT-Pipeline with Dagster 🌎
jgrove90/ufo-deltalake
🛸 This project showcases an Extract, Load, Transform (ELT) pipeline built with Python, Apache Spark, Delta Lake, and Docker. The objective of the project is to scrape UFO sighting data from NUFORC and process it through the Medallion architecture to create a star schema in the Gold layer that is ready for analysis.
stellar/stellar-dbt-public
Public DBT instance to aid in data transformation for analytics purposes
jgrove90/rick-and-morty-deltalake
🍺 A data engineering project showcasing an ELT pipeline using modern technologies such as Delta-rs, and Apache Airflow.
longNguyen010203/ECommerce-ELT-Pipeline
🌄📈📉 A Data Engineering Project 🌈 that implements an ELT data pipeline using Dagster, Docker, Dbt, Polars, Snowflake, PostgreSQL. Data from kaggle website 🔥
vigneshSs-07/Google-Cloud-Professional-Data-Engineer-ACompleteGuide
This Repo contains all study, lab and supportive materials for Udemy course on "Google Cloud Professional Data Engineer - A Complete Guide".
arunp77/SQL
SQL, Databases, warehouses, Data lake, cloud storage, MYSQL, Data Pipeline
jackmulligan-ire/ppr-pipeline
Irish Property Price Register transformed into a data warehouse via an EtLT pipeline.
judeleonard/e-commerce_activity_tracking
This is an ELT data pipeline setup to track the activities of an e-commerce website based on orders, reviews, deliveries and shipment date. This project utilized technologies like Airflow, AWS RDS-Postgres, Python etc.
kayazay/zomato-restaurant-analytics
An end to end ELT project that uses data from the Zomato Restaurant, an Indian multinational restaurant aggregator and food delivery company. The project extracts data from Kaggle dataset, loads it into Snowflake tables, then is transformed and modelled in dbt Labs.
MettaSurendhar/DataEngineeringProject
Data Engineering project which involves ETL using PostgreSQL and Python
nadyavoynich/Instacart
A deep dive into North American grocery e-commerce behaviour based on Instacart's open dataset. [ELT, EDA, ML clustering]
Suprame4/Data_Engineering_Projects
Data engineering projects
hvignolo87/superside-challenge
Resolution of the Superside challenge for the Lead Data Engineer role
JaleesMoeen/Crowdfunding_ETL
Builded an ETL pipeline using Python, Pandas, Python dictionary methods and regular expressions to ETL data. It involves extracting data from multiple sources, cleaning and transforming the data using Jupyter Notebook with pandas, numpy, and datetime packages, and loading the cleaned data into a relational database using pgAdmin
johnkdunyo/Postgres-ELT-Data-Pipeline
An ELT data pipeline project utilizing docker and postgress (both source and destination dbs)
KSwaviman/Enterprise-ELT-Framework
Enterprise ELT Framework using Airbyte, dbt, Prefect, and Power BI for seamless data extraction, transformation, and visualization. This project showcases a scalable pipeline integrating SQL Server, GCP, and tabular models in Power BI for real-time analytics and business intelligence. Ideal for data engineers and analysts seeking efficient ETL/ELT.
rizkipragustono/ecommerce_elt_project
Modern ELT with Snowflake, dbt, and Star Schema for E-Commerce
tanzealist/AutoImageCaption-CNNvsResNet
"AutoImageCaption-CNNvsResNet" leverages the Flickr 8k Dataset to automate image captioning, comparing CNN+LSTM and ResNet+GRU models using BLEU scores for performance evaluation.
bazuayelewis/olist-store-brazil
This repository contains my capstone project for the Data Engineering course at Altschool Africa. It demonstrates my capability to develop a comprehensive end-to-end ELT process for an organization using tools such as Airflow, dbt, Docker, BigQuery, and PostgreSQL.
erreduarte/worldbank-finance-elt-pipeline
Automated ELT pipeline extracting World Bank Finance data, transforming with Pandas, and loading into Azure SQL Database using Apache Airflow.
ibnufajar1994/elt-data-warehouse
Build and Orchestrate an ELT Data Pipeline Using Luigi
lksprado/DW-ETL-end_to_end
ELT com python, AWS RDS, dbt-core
OtmaneDaoudi/azure-migration-pipeline
ELT pipeline for sales analysis using Azure cloud.
rifqialf/customer-churn-elt
Creation of ELT pipeline in Azure Data Factory & Azure Databricks with PySpark and SQL to build a Power BI dashboard for customer churn insights of a telecommunication company in California, USA
sagarswami01/DE_Project
This repo gives you insights on the data engineering project I built using public datasets from San Francisco open data portal.
fajri-yanti/dwh-amazon
This project aims to optimize the extraction, transformation, and analysis of transactional data for a rapidly growing e-commerce platform. It involves building data pipeline capable of handling large volumes of transactional information, ensuring the data is accurate, structured for insightful analysis.
mfajarandikha/DataEngineering_ELT_Airflow
This repository demonstrates an end-to-end ELT (Extract, Load, Transform) pipeline that extracts data from a source PostgreSQL database, loads it into a destination PostgreSQL database, and performs data transformations using dbt (Data Build Tool).
nabilraihann/ELT-Pipeline-Bike-Store
This repository contains the implementation of an ELT (Extract, Load, Transform) pipeline for a Bike Store dataset using modern data tools. The pipeline integrates Airbyte for data extraction, dbt for data transformation, Airflow for orchestration, and Snowflake as the data warehouse.
SalvatoreAmaddio/PipelineWebsite
This a console line application is an Ad-hoc Solution for a client who needed a way of extracting data from their own website and print them onto a spreadsheet.