crakjie/sbt-spark-plugin

What if my main class is not named StreamingApp?

christobill opened this issue · 2 comments

sbt clean package submit
.....
java.lang.ClassNotFoundException: StreamingApp
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:278)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:633)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:169)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:192)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:111)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

The problem comes from SparkPlugin.scala:
val cmd = sparkHome.value+"/bin/spark-submit --class StreamingApp .....

This issue is addressed with this commit a756ba5
Use the version 1.0.1 addSbtPlugin("com.github.crakjie" % "sbt-spark-plugin" % "1.0.1")

If this still don't work configure your self the sbt key mainClass (which should be automatically set by sbt )

Thank you for your prompt answer.