Instructions on building and running the Spark Docker image on Centos7, with samba volume.
docker volume create --driver local
--opt type=cifs
--opt device=//SAMBA_SERVER_IP/PATH_TO_DATA/data
--opt o=username=username,password=password VOLUME_NAME
- Build the image worker image
docker-compose build
- Start up a Spark master and n workers
docker-compose up --scale worker=n
- Too destroy the cluster
docker-compose down
- Build the worker image
docker-compose -f worker.yml build
- Start up a Spark workers
docker-compose -f worker.yml --scale worker=n
- To destroy the cluster of workers
docker-compose -f worker.yml down