/Apache-Beam-data-engineering

Apache Beam data engineering

Primary LanguageJupyter Notebook

D3.js Visualization

Utilized the D3.js library to create interactive visualizations for an e-commerce purchase history dataset generated using ChatGPT. The visualizations include advanced features such as zooming, hovering, and data point circles.

Features

  • Stacked Bar Charts
  • Line Charts
  • Zooming In and Out
  • Hover Effects
  • Data Point Circles
  • and more...

Auto EDA with Sweetviz

Performed Automated Exploratory Data Analysis (Auto EDA) on the "Most Streamed Spotify Songs 2023" Kaggle dataset using Sweetviz. Sweetviz generated insightful visualizations and statistics for data exploration.

Features

  • Automatic Data Analysis
  • Visualizations for Data Insights
  • Comparisons between Datasets
  • Detailed Statistical Summaries
  • and more...

Apache Beam Data Processing

Implemented and showcased various features of Apache Beam for data processing tasks. This includes the use of composite transforms, pipeline I/O, triggers, windowing, and ParDo functions.

Features

  • Composite Transforms
  • Pipeline I/O
  • Triggers
  • Windowing
  • ParDo Transformations