Pinned Repositories
Csv-To-Parquet-And-Data-Reporting-Pipeline
This project extracts data having 800k records from CSV in the data factory and convert it to parquet based data and finally create a PowerBI report of that parquet based data.
Data-Cleaning-Pipeline-ETL
This project extracts data from Azure datalake gen 2 storage, transforming it and then transferring it to SQL database.
Data-Filtering-Pipeline-ETL
This Project Extracts supply chain data from csv file having 180k records and more than 40 columns from the Azure Datalake Gen2 storage account and do some dataanalysis with Python(Pandas) to find the top 3 countries and filtered the data for top 3 countries and finally transferred it to 3 files in datalake again by creating ETL pipeline in ADF.
Datalake-To-Database-Via-DataBricks
This project extracts data from Datalake and then transfer to Azure SQL Database via Azure DataBricks in Python(Pyspark).
MuhammadHasaanWahid's Repositories
MuhammadHasaanWahid/Csv-To-Parquet-And-Data-Reporting-Pipeline
This project extracts data having 800k records from CSV in the data factory and convert it to parquet based data and finally create a PowerBI report of that parquet based data.
MuhammadHasaanWahid/Data-Cleaning-Pipeline-ETL
This project extracts data from Azure datalake gen 2 storage, transforming it and then transferring it to SQL database.
MuhammadHasaanWahid/Data-Filtering-Pipeline-ETL
This Project Extracts supply chain data from csv file having 180k records and more than 40 columns from the Azure Datalake Gen2 storage account and do some dataanalysis with Python(Pandas) to find the top 3 countries and filtered the data for top 3 countries and finally transferred it to 3 files in datalake again by creating ETL pipeline in ADF.
MuhammadHasaanWahid/Datalake-To-Database-Via-DataBricks
This project extracts data from Datalake and then transfer to Azure SQL Database via Azure DataBricks in Python(Pyspark).