/Hadoop-Spark-Simple-System

Primary LanguageHTML

This project implement an application based on Hadoop+Spark for movie comments sentiment analysis.

Team Project: The code module for the details of code design, and report for this project process. and my efforts are data analysis, data visualization, system establishing.
There are mainly 4 parts of this project, they are:
1. data crawling
2. data analysis
3. set up the Distributed File Storage System on the virtual environment(HDFS).
4. set up the Distributed File Processing System on the computer, run the program on the environment (In this project: Hadoop and Spark)
5. data processing