Error when accessing hdfs file from remote spark driver
jrabary opened this issue · 1 comments
jrabary commented
Thank you for sharing the result of your project. I'm trying play the hadoop spark workbench and I got some trouble when I want to access the data stored in the hdfs file.
My setting is as follow:
- I set up the hadoop-spark-workbench in a physical server inside our internal network.
- I run a remote spark-shell from my laptop by specifying the master address.
When I try to load a data on the hdfs for example using the following line:
val data = sparkContext.textFile("hdfs://namenode/data.txt")
I get a connection refused error even if the port 8082
is forwarded in the docker.
jrabary commented
Finally get it work. I forwarded the wrong port!