The aim of this project is to build a data warehouse and a data mart to:
- Automate daily API data-extraction in AWS for stock prices
- Integrate and transform the daily data along with other associated financial data (e.g. valuation ratio, earnings and income statement, etc.)
- Load the integrated, clean,and transformed data to a Data Warehouse and a star schema for specific technical analyses
- Present the analyses in a visualization format using a Tableau dashboard.
The goals achieved in this project are:
- Designed ETL pipeline and performed data modeling using SSIS to integrate data into facts and dimensions.
- Automated extraction of data from multiple data sources using APIs, AWS Lambda and CloudWatch, and exported it to S3.
- Accomplished data manipulation, profiling, cleansing, integration and price history tracking using SSIS packages.