In this project, we will use the Movielens dataset and based on the instruction below to build a Spark Movie Recommendation Service using Flask.
We will further implement serveral different Movie Recommendation Algorithms and port the backend service on AWS.
Our final step is to build a web site like MoiveLens to recommend the films to the user using the backend service on the cloud.
Here is what we have to do:
Understand how Spark works
https://spark.apache.org/docs/latest/
https://github.com/jadianes/spark-py-notebooks
Try to set up Spark cluster manually on IBM vm cluster.
Understand how to create Spark cluster on AWS.
https://github.com/amplab/spark-ec2
Start up Spark locally and finish building the service based on instruction below(Collaborative Filtering)
https://www.codementor.io/jadianes/building-a-recommender-with-apache-spark-python-example-app-part1-du1083qbw https://www.codementor.io/jadianes/building-a-web-service-with-apache-spark-flask-example-app-part2-du1083854 https://databricks-training.s3.amazonaws.com/movie-recommendation-with-mllib.html
Port the service to AWS.
Build more algorithms using Spark.
Build Web App.