YotpoLtd/metorikku

Need to add dependencies in built.sbt

sourav2050 opened this issue · 3 comments

sourav2050 commented

Hi,
You need to add below two dependencies in built.sbt file at below your ibraryDependencies.
I am using Spark version 2.4.4 and Scala version 2.11.12.

libraryDependencies += "org.apache.spark" %% "spark-core" % "2.4.4"
libraryDependencies += "org.apache.spark" %% "spark-sql" % "2.4.4"

As without these two dependencies the code is throwing error NoClassDefFoundError for SparkSession, Spark Listener and logger log4j.

lyogev commented

What IDE Are you using? (as these are included as provided)

sourav2050 commented

I am running my code in Intellij. Although those dependencies are added in your built.sbt but still that throwing error with class not found. I added above two dependencies below your library dependencies and code ran fine. Does it matter on the basis of IDE?

lyogev commented

image
Use this feature in intellij to get the provided scoped dependencies

  • 👍1