This is a companion repo for this blogpost
It contains two Spark applications: a hello-world and a Travelling Salesman Problem solver.
- Run
./start-spark-cluster.sh
. This will build the Spark Docker image and then start a 3-node Spark cluster using docker-compose. - Run
sbt assembly
to build the uber-jar containing both apps. - Run
./open-spark-UIs.sh
to open the various Spark UIs in your browser, so you can see what's going on. - Run either
./run-hello-world.sh
or./run-travelling-salesman.sh
and inspect the terminal output.