Chapter 10 LeftOuterShuffleRegionJoin issue
DaHanII opened this issue · 6 comments
There will be an error when the code is at **LeftOuterShuffleRegionJoin(bHg19Data.value.sequences1000000, sc)**and it shows
error: too many arguments for method apply: (leftRdd: org.apache.spark.rdd.RDD[(org.bdgenomics.adam.models.ReferenceRegion, T)], rightRdd: org.apache.spark.rdd.RDD[(org.bdgenomics.adam.models.ReferenceRegion, U)])(implicit evidence$7: scala.reflect.ClassTag[T], implicit evidence$8: scala.reflect.ClassTag[U])org.bdgenomics.adam.rdd.LeftOuterShuffleRegionJoin[T,U] in object LeftOuterShuffleRegionJoin
So how to fix it
How are you causing this error? sounds like a compile error, but one caused by changing the version of bdgenomics in use or something.
This is what I got following the code
It's not clear what you are running, or how. The code compiles as-is from the repository, and should run fine in the shell. It seems like you have mismatched versions of Spark or a library.
I run it on school machine and line by line of RunTFPrediciton.scala on github by starting adam-shell. I don't think it is the mismatched version issue since spark version is 2 and scala is 2.11. I git Adam and build following the book's instruction(copy and paste in the virtual machine).I also look at Adam source code and for LeftOuterShuffleRegionJoin I find it changes the API: case class LeftOuterShuffleRegionJoin[T: ClassTag, U: ClassTag](leftRdd: RDD[(ReferenceRegion, T)], rightRdd: RDD[(ReferenceRegion. So I think that's why it shows such error.
You are running adam-shell, and I am guessing you are using a newer version than what's in the book. The code in the book assumes you're using the code in the book/repo, and spark-shell.