/DIC_LAB_2

Primary LanguageJupyter Notebook

Big Data Architecture for Analysing the Current trends in Any Topic

Demo

Code and Deliverables

Example: Current Trends in Europe

The Task of the Project:

  1. first write the python scripts to collect the Data From NYT And Twitter API.
  2. Write the the python Script to crawl the web scraping data from common crawl Repositories.
  3. Provision the AWS GLU ETL pipeline and AWS EMR cluster for Map Reduce jobs.
  4. Design the interactive Web interface for Visualising the Big Data.
  5. Design the AWS Architecture to convert the project in to Production Ready state.

Architecture Diagram