A web crawler written in scala.
Use the following command to run the program, this script looks for the sbt , if it is present then run's it and if not
then first downloads it and then run the program.
./run.sh
We used Akka for developing this solution.As it can easily be scaled by using the Router and supervisor.
To reduce the complexity the url's that do not contain wiprodigital.com are not visited.
With more time we can add the functionality of scaling it using Router and Supervisor. With more time we can also add the test cases for akka actors and add the functionality for getting the images reference as well.