Stratio/Spark-MongoDB

Getting Error java.lang.NoClassDefFoundError: org/apache/spark/sql/Dataset

erarjit opened this issue · 0 comments

I am using example provided on in the doc.

I started spark shell using below
bin/spark-shell --packages com.stratio.datasource:spark-mongodb_2.11:0.12.0

Then executed below code

 import org.apache.spark.sql._
 import com.mongodb.casbah.{WriteConcern => MongodbWriteConcern}
 import com.stratio.datasource.mongodb._
 import com.stratio.datasource.mongodb.config._
 import com.stratio.datasource.mongodb.config.MongodbConfig._

val sparkSession = SparkSession.builder().getOrCreate()
case class Student(name: String, age: Int)
val dataFrame: DataFrame = sparkSession.createDataFrame(sc.parallelize(List(Student("Torcuato", 27),    Student("Rosalinda", 34))))
val saveConfig = MongodbConfigBuilder(Map(Host -> List("localhost:27017"), Database -> "highschool", Collection ->"students", SamplingRatio -> 1.0, WriteConcern -> "normal", SplitSize -> 8, SplitKey -> "_id"))
dataFrame.saveToMongodb(saveConfig.build)

But at the last line I am getting below error.

scala> dataFrame.saveToMongodb(saveConfig.build)
java.lang.NoClassDefFoundError: org/apache/spark/sql/Dataset
  at     com.stratio.datasource.mongodb.MongodbDataFrame.saveToMongodb(MongodbDataFrame.scala:36)
  ... 56 elided
Caused by: java.lang.ClassNotFoundException: org.apache.spark.sql.Dataset
  at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
  ... 57 more