daithiocrualaoich/spark-vagrant-vm

Had issues w/ Spark setup

Opened this issue · 0 comments

I was getting:
==> node-4: cp:
==> node-4: cannot create regular file `/usr/local/spark/conf'
==> node-4: : No such file or directory
==> node-4: creating spark environment variables
==> node-4: Running provisioner: shell...
node-4: Running: /var/folders/kr/2sll_9v94c96886lth6ptf4m0000gp/T/vagrant-shell20141124-5424-jox9fp.sh
==> node-4: setup hadoop slaves
==> node-4: modifying spark slaves
==> node-4: adding node3
==> node-4: /tmp/vagrant-shell: line 22: /usr/local/spark/conf/slaves: No such file or directory
==> node-4: adding node4

So i changed in scripts/setup-spark.sh:
ln -s /usr/local/$SPARK_VERSION /usr/local/spark
to
ln -s /usr/local/$SPARK_VERSION-bin-hadoop2 /usr/local/spark