extract-transform-load
There are 88 repositories under extract-transform-load topic.
Youtube_Data_Harvesting_and_Warehousing
Domain : Social Media | Extracting data using Youtube API and storing it on MongoDB then Transforming it to a relational databaselike MySQL. For getting various info about youtube channels.
build-a-monster
Built an app that allows you to build your own monster and receive a predicted challenge rating and encounter difficulty by choosing the monster's characteristics and ability ratings as well as providing your party's size and average level.
surfs_up
Application of Python database toolkits, such as SQLAlchemy and Flask, for performing data analytics and visualization as part of the Extract, Transform, and Load (ETL) process.
nlp-job-postings
Natural language processing of job postings in order to gain insight into the data science job market.
Steam_statistics_ETL
This pipeline can be used to collect statistical information about all games, distributed through the Steam platform.
Twitter-ETL-of-Elections-PoliceBrutality-HateSpeech-Data
This Twitter ETL project is aimed at providing data to support UN SDG number 16. The project is directed at providing data to generate actionable insights to stakeholders; regarding the 2022 Presidential Elections, Police Brutality, and Propagation of Hate Speech on Twitter
ETL-Sales_Analysis_Report---MySQL-PowerBI
This repo explains how ETL can be done in MySQL and PowerBi to generate insights!
The-Music-has-Changed-WEBSIDE
We going to examine two data sets relate with the music Industry. We want Extract, transform and load this in order to identify insides and trend about the music Industry.
ETL-Project
This project aims to create an ETL pipeline from energy consumption data.
klipper
A Web and API Development Platform build over Symfony
landgrab
Geospatial data hoarding system
sqlserver-copy-java
A Java utility to copy data from a SQL Server table, save it to CSV files, and upload to another SQL Server database.
faa
FAA Airline On-Time Performance Data
ETL_vs_ELT
Comparison between ETL (Extract, Transform, Load) and ELT (Extract, Load, Transform)
AtliQ-Grands-Hospitality-Insights-using-Power-BI
AtliQ Grands hotel Data Analysis using Power BI
regtab
Regtab is a Java library for data extraction from arbitrary tables represented in machine-readable formats
near-real-time-data-warehouse-prototype-for-electronics-business-chain-using-java-and-mysql
This repository comprises the design, implementation, and analysis of a near real-time data warehouse prototype for an electronics business chain, utilising a multi-threaded Extract, Transform, Load (ETL) pipeline leveraging the efficient HYBRIDJOIN algorithm implemented with Java and MySQL on customer sales data.
Supply-Chain-ETL
Data Engineering Project on Supply Chain ETL. Creating a dynamic ADF pipeline to ingest both Full Load and Incremental Load data from SQL Server and then transform these datasets based on medallion architecture using Databricks.
Inventory_Management_Dashboard
This project provides Inventory Management using Power BI, extremely useful for Warehouse/ In-plant Inventory Managers to effectively control the Inventory levels and also maintain the Service Levels.
Human-Activity-Prediction
This project focuses on using sensor data to predict human activity and is based on the ExtraSensory dataset, created by Ph.D. students and staff at the Department of Electrical and Computer Engineering, University of California, San Diego.
Crowdfunding_ETL
This project takes the crowd funding data provided in excel files through Extract Transform and Load (ETL) process and makes it available in a relational database for further usage.
Ciencia-de-dados-Pandas
Este espaço é dedicado para treinar minhas habilidades em ciência de dados, concentrando-se principalmente no aproveitamento da biblioteca Pandas para manipulação e análise de dados.
Water-Quality-DW-on-Oracle-Database
This is an Oracle DB Data Warehouse and ETL implementation on specially formatted Water Quality dataset from DEFRA, UK
machine-learning-monitoring-with-evidently
ML Monitoring with EvidentlyAI
Data_Pipeline
Implemented ETL projects with interactive Streamlit UI for user-friendly data extraction, transformation, and loading tasks
Google_BI_Professional
This certification focuses on in-demand skills like data modeling, data visualization, and dashboarding and reporting.
NYC-TLC-Data-Engineering
NYC TLC Data Analysis using Python, GCP Storage, Compute Engine, Mage Data Pipeline Tool, BigQuery, and Looker Studio. Aims to extract insights from the dataset for informed decisions and deeper operational understanding.
Extract-Transform-Load-Process-Techniques
This repository contains code for comparing the performance of three different ELT (Extract, Load, Transform) methods on CSV files of different sizes. The three methods are implemented in Python using different approaches and libraries, and their execution times are compared and plotted for analysis.
CHM090Efficacy
Final Code from the CHM090 Efficacy Project
Crowdfunding-ETL
Extract, transform, and load crowdfunding data from csv to postgres database in order to perform an analysis using SQL on the loaded database.
ETL-Pipline
Building a fully scalable ETL (Extract, Transform, Load) pipeline to handle large volumes of transaction data for a café business.
Crowdfunding-ETL
Application of Python libraries, like Pandas, and their useful functions for performing efficient Extract, Transform, and Load (ETL) process.
wordle-ETL
ETL for Wordle game
powerbi-reports
A repository for my Power BI reports.
bookscrap
Student project #1 - Web scraping, use Python basics to create a program that automate the process of extracting, transform and load data from the online library "Books to Scrape".