/TokyoAnalyzeUsingAzure

I Have created simple mini project of Azure using its services

Primary LanguageJupyter Notebook

TokyoAnalyzeUsingAzure

Created simple mini project of Azure using its services.We have uploaded Our demo video Link on Google Drive https://drive.google.com/drive/folders/1LiRQW23ER5gb2zd4HnNld_2uvQ2MWKKd?usp=sharing Short Description About Project Project Topic : Olympic Data Analysis Project Project report outlines the process of analyzing Tokyo Olympic data using various Azure services. The team, consisting of Ms. Mansi Jadhav, Mr. Jayesh Waingankar, and Mr. Yogesh Pawar, leveraged services like Azure Data Factory, Azure Synapse Analytics, Azure Databricks, and App Registration. The report includes detailed steps, challenges faced, and lessons learned throughout the project. This project is an Olympic data analysis and reporting solution built using Azure services. In this project 4 azure services are used that are Azure Storage Account, Azure Data Factory, Azure Databricks, and Azure Synapse Analytics Workspace.

Overview

This project demonstrates the analysis of Olympic data using a set of Azure services to perform ETL, data processing, and reporting. The project showcases the following key components: 1)Azure Storage Account: Used for data storage and retrieval. 2)Azure Data Factory: Orchestrates and schedules data workflows. 3)Azure Databricks: Performs data transformations and analysis. 4)Azure Synapse Analytics Workspace: Stores and manages data, and provides a platform for advanced analytics.

Prerequisites

Before you can use this project, ensure you have the following prerequisites:

An Azure subscription. Appropriate permissions to create and manage Azure resources. Azure Data Factory, Azure Databricks, and Azure Synapse Analytics provisioned and configured. Python and the necessary libraries installed locally for Databricks development. PowerBI Desktop to visualise the data which we have processed Dataset for analysis In this Project we have Dataset and we have done analyzation and visualization of the data. Dataset is provided in the github.

Diagram of Data Factory :-image

Here we have container and in that we have directory raw and transformed data image

Here we create data factory image

Console of data factory image

Preview Data of one of the dataset image

Pipeline created image

Azure Databricks console here we will write code for transformation image

Azure Synapse Analyze where we will write our SQL query image

image

This is the viualization of the data using Powerbi. image

This project report showcases a well-structured approach to utilizing Azure services for data analysis, providing a valuable resource for others interested in similar endeavors.