amplab/spark-ec2

Hadoop 2.6 support for Spark 1.6

anfeng opened this issue · 4 comments

Is there any reason that you only support Hadoop 2.4 for Spark 1.6?
It will be lovely if you could also support Hadoop 2.6.

WIgor commented

Actually, I've made some dirty hacks here https://github.com/grammarly/spark-ec2 for my own use. Not so complex and it works, in my case. I needed in s3a support that was included to hadoop 2.6.

@WIgor would you be interested in sending a PR for this ?

WIgor commented

@shivaram Yes, I thought to update this to latest spark version. And then will send PR.

propose a PR #88