log8415-tp2

Instruction to run the code

To seamlessly navigate through the AWS setup and execute the log8415-tp2 scripts, follow these steps: Begin by updating the ~/.aws/credentials file with your AWS credentials.
Next, clone the project repository from https://github.com/decorJim/log8415-tp2.git onto your local machine.
Move into the log8415-tp2/infra/ directory and initialize Terraform with terraform init, followed by applying it using terraform apply.
Once the instance is created in your AWS account, connect to it as a root user.
Inside the instance, clone the same project repository or use scp to secure copy the project from your local machine to the instance.
The following sections explain how to run the code for the 3 experiments which are Hadoop vs Linux, Hadoop vs Spark and Friend suggestions

Hadoop vs. Linux

To execute the Hadoop vs Linux experiment, navigate to the log8415-tp2/hadoop_vs_linux/ directory and ensure executables permissions by running sudo chmod +x configuration.sh and sudo chmod +x hadoop_vs_linux.sh.
After, run the hadoop_vs_linux.sh script.
The previous command should output a file named hadoop_vs_linux_comparison.txt which contain the chrono time for each execution. Also, you can find in this same directory the output of the two algorithms which are the lists of words with their number of occurrences.

Hadoop vs. Spark

To execute the Hadoop vs Linux experiment, navigate to the log8415-tp2/hadoop_vs_spark/ directory and ensure executables permissions by running sudo chmod +x configuration.sh and sudo chmod +x hadoop_vs_spark.sh.
After, run the hadoop_vs_spark.sh script and follow the prompts displayed on the screen, checking all options and pressing 'enter'.
The previous commands should output a file named hadoop_vs_spark_comparison.txt which contain the chrono time for each execution. Also, you can find in this same directory the output of the two algorithms which are the lists of words with their number of occurrences.

Friend suggestions

To execute the Friend suggestions experiment, navigate to the log8415-tp2/friend_suggestions/ directory and ensure executables permissions by running sudo chmod +x configuration.sh and sudo chmod +x friend_suggestions.sh.
After, run the friend_suggetions.sh script.
The previous commands should output a directory called ouput which contain the results of the algorithm which are the friend suggestions.

References used in the project