rockthejvm/spark-essentials

Error after running : "sudo docker-compose up --scale spark-worker=3"

Closed this issue · 1 comments

bellow the error when running "sudo docker-compose up --scale spark-worker=3"

Starting spark-cluster_spark-master_1 ... done
Starting spark-cluster_spark-worker_1 ... done
Starting spark-cluster_spark-worker_2 ... done
Starting spark-cluster_spark-worker_3 ... done
Attaching to spark-cluster_spark-master_1, spark-cluster_spark-worker_1, spark-cluster_spark-worker_2, spark-cluster_spark-worker_3
spark-master_1 | /start-master.sh: line 5: /spark/sbin/spark-config.sh: No such file or directory
spark-master_1 | /start-master.sh: line 7: /spark/bin/load-spark-env.sh: No such file or directory
spark-master_1 | /start-master.sh: line 15: cd: /spark/bin: No such file or directory
spark-worker_2 | /start-worker.sh: line 3: /spark/sbin/spark-config.sh: No such file or directory
spark-worker_2 | /start-worker.sh: line 4: /spark/bin/load-spark-env.sh: No such file or directory
spark-worker_2 | /start-worker.sh: line 12: /spark/sbin/../bin/spark-class: No such file or directory
spark-worker_3 | /start-worker.sh: line 3: /spark/sbin/spark-config.sh: No such file or directory
spark-worker_3 | /start-worker.sh: line 4: /spark/bin/load-spark-env.sh: No such file or directory
spark-worker_1 | /start-worker.sh: line 3: /spark/sbin/spark-config.sh: No such file or directory
spark-worker_1 | /start-worker.sh: line 4: /spark/bin/load-spark-env.sh: No such file or directory
spark-worker_1 | /start-worker.sh: line 12: /spark/sbin/../bin/spark-class: No such file or directory
spark-worker_3 | /start-worker.sh: line 12: /spark/sbin/../bin/spark-class: No such file or directory
spark-cluster_spark-master_1 exited with code 1
spark-cluster_spark-worker_2 exited with code 127
spark-cluster_spark-worker_1 exited with code 127
spark-cluster_spark-worker_3 exited with code 127

Did building the images work?
Can you try removing the containers and the images and build them again?