elt-pipeline

There are 68 repositories under elt-pipeline topic.

  • dataforgelabs/dataforge-core

    DataForge helps data teams write functional transformation pipelines by leveraging software engineering principles

    Language:PLpgSQL48912

  • Fozan-Talat/divvy-bikeshare-de-project

    An end-to-end data pipeline which extracts divvy bikeshare data from web loads it into data lake and datawarehouse transforms it using dbt and finally , a dashboard to visualize the data using looker studio, the pipeline is orchestrated using prefect

    Language:Python37206

  • Mg30/pydwt

    Modeling tool like DBT to use SQL Alchemy core with a DataFrame interface like

    Language:Python11160

  • bennyaustin/synapse-dataplatform

    A modern data platform implemented on Azure Synapse Analytics using ELT Framework - https://github.com/bennyaustin/elt-framework. Data platform infrastructure provisioned using https://github.com/bennyaustin/iac-synapse-dataplatform

    Language:TSQL7206

  • longNguyen010203/FDE-Course-2024-W4-DBT

    💻💛Fundamental Data Engineering Course 2024 Week4 Learn DBT Transform Data with Models, Macro, ELT-Pipeline with Dagster 🌎

    Language:Python5200

  • jgrove90/rick-and-morty-deltalake

    🍺 A data engineering project showcasing an ELT pipeline using modern technologies such as Delta-rs, and Apache Airflow.

    Language:Python4100

  • jgrove90/ufo-deltalake

    🛸 This project showcases an Extract, Load, Transform (ELT) pipeline built with Python, Apache Spark, Delta Lake, and Docker. The objective of the project is to scrape UFO sighting data from NUFORC and process it through the Medallion architecture to create a star schema in the Gold layer that is ready for analysis.

    Language:Python4101

  • stellar/stellar-dbt-public

    Public DBT instance to aid in data transformation for analytics purposes

    Language:Shell41744

  • arunp77/SQL

    SQL, Databases, warehouses, Data lake, cloud storage, MYSQL, Data Pipeline

    3101

  • kkumyk/server-logs-daily-data-pipeline

    A data engineering project with dbt, Docker, Kestra, Terraform, GCP and Looker.

    Language:HCL310

  • longNguyen010203/ECommerce-ELT-Pipeline

    🌄📈📉 A Data Engineering Project 🌈 that implements an ELT data pipeline using Dagster, Docker, Dbt, Polars, Snowflake, PostgreSQL. Data from kaggle website 🔥

    Language:Python3100

  • vigneshSs-07/Google-Cloud-Professional-Data-Engineer-ACompleteGuide

    This Repo contains all study, lab and supportive materials for Udemy course on "Google Cloud Professional Data Engineer - A Complete Guide".

    Language:Python3103

  • jackmulligan-ire/ppr-pipeline

    Irish Property Price Register transformed into a data warehouse via an EtLT pipeline.

    Language:TypeScript2200

  • judeleonard/e-commerce_activity_tracking

    This is an ELT data pipeline setup to track the activities of an e-commerce website based on orders, reviews, deliveries and shipment date. This project utilized technologies like Airflow, AWS RDS-Postgres, Python etc.

    Language:Python2200

  • kayazay/zomato-restaurant-analytics

    An end to end ELT project that uses data from the Zomato Restaurant, an Indian multinational restaurant aggregator and food delivery company. The project extracts data from Kaggle dataset, loads it into Snowflake tables, then is transformed and modelled in dbt Labs.

    2152

  • nadyavoynich/Instacart

    A deep dive into North American grocery e-commerce behaviour based on Instacart's open dataset. [ELT, EDA, ML clustering]

    Language:Jupyter Notebook2100

  • Suprame4/Data_Engineering_Projects

    Data engineering projects

    Language:Jupyter Notebook2201

  • hvignolo87/superside-challenge

    Resolution of the Superside challenge for the Lead Data Engineer role

    Language:Python1100

  • JaleesMoeen/Crowdfunding_ETL

    Builded an ETL pipeline using Python, Pandas, Python dictionary methods and regular expressions to ETL data. It involves extracting data from multiple sources, cleaning and transforming the data using Jupyter Notebook with pandas, numpy, and datetime packages, and loading the cleaned data into a relational database using pgAdmin

    Language:Jupyter Notebook1100

  • johnkdunyo/Postgres-ELT-Data-Pipeline

    An ELT data pipeline project utilizing docker and postgress (both source and destination dbs)

    Language:Python110

  • KSwaviman/Enterprise-ELT-Framework

    Enterprise ELT Framework using Airbyte, dbt, Prefect, and Power BI for seamless data extraction, transformation, and visualization. This project showcases a scalable pipeline integrating SQL Server, GCP, and tabular models in Power BI for real-time analytics and business intelligence. Ideal for data engineers and analysts seeking efficient ETL/ELT.

    Language:Python1100

  • MettaSurendhar/DataEngineeringProject

    Data Engineering project which involves ETL using PostgreSQL and Python

    Language:Python1100

  • oli2v/flight-radar-gcp

    FlightRadar ELT pipeline on GCP

    Language:HCL1100

  • rizkipragustono/ecommerce_elt_project

    Modern ELT with Snowflake, dbt, and Star Schema for E-Commerce

    Language:Python1100
  • northwind_data_warehouse

    tanhoang1808/northwind_data_warehouse

    This project perform an ELT solution to design a Data Warehouse in Snowflake

    110

  • tanzealist/AutoImageCaption-CNNvsResNet

    "AutoImageCaption-CNNvsResNet" leverages the Flickr 8k Dataset to automate image captioning, comparing CNN+LSTM and ResNet+GRU models using BLEU scores for performance evaluation.

    Language:Jupyter Notebook1100

  • akshay-gera/dbt_bigquery_project

    End to End Data Pipeline Project on Google Cloud Warehouse, DBT Data Modelling and Power BI Data Visualization

    0100

  • fajri-yanti/dwh-amazon

    This project aims to optimize the extraction, transformation, and analysis of transactional data for a rapidly growing e-commerce platform. It involves building data pipeline capable of handling large volumes of transactional information, ensuring the data is accurate, structured for insightful analysis.

    Language:Python0100

  • mfajarandikha/DataEngineering_ELT_Airflow

    This repository demonstrates an end-to-end ELT (Extract, Load, Transform) pipeline that extracts data from a source PostgreSQL database, loads it into a destination PostgreSQL database, and performs data transformations using dbt (Data Build Tool).

    Language:Python00

  • ShaheerKhan200/gads-modern-data-stack

    ELT Batch Pipeline using modern data stack (dbt, postgres, docker, ec2 etc)

    Language:Shell0100

  • sudo-donya/dbt-dwh-example

    0100

  • zythedeveloper/first-elt-project

    This project is based on Justin B. Chau's tutorial on FreeCodeCamp's YouTube channel. It explores the creation and deployment of a custom Extract, Load, and Transform (ELT) pipeline, demonstrating practical data engineering concepts and techniques.

    Language:Python0000

  • DSCmatter/SnowDBT_Pipeline

    This project sets up an ELT pipeline using Snowflake and dbt, following best practices for modular and scalable data transformations.

  • KwiecinskiMateusz/Project1_ELT_Pipeline

    ELT pipeline transforming raw CSV data into a structured MySQL dataset for potential analysis and visualization

  • ndomah/ELT-Pipeline

    Simple ELT pipeline using dbt, Snowflake, and Apache Airflow.

    Language:Python

  • paty-oliveira/carris-data-pipeline

    Repository for Extraction, Loading and Transformation of Carris data.

    Language:Python