Vegas with Livy inside Zeppelin
maziyarpanahi opened this issue · 2 comments
maziyarpanahi commented
Hi,
I am using Zeppelin (0.7.2) with Livy (0.4.0-incubating - Spark 2.2). Since Livy doesn't support %dep I have to load the package by config:
livy.spark.jars.packages org.vegas-viz:vegas_2.11:0.3.11
When I do the mentioned it always breaks my RPC:
WARN RSCClient: Client RPC channel closed unexpectedly.
WARN RSCClient: Error stopping RPC.
io.netty.util.concurrent.BlockingOperationException: DefaultChannelPromise@3eea3abd(uncancellable)
at io.netty.util.concurrent.DefaultPromise.checkDeadLock(DefaultPromise.java:390)
at io.netty.channel.DefaultChannelPromise.checkDeadLock(DefaultChannelPromise.java:157)
at io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:251)
at io.netty.channel.DefaultChannelPromise.await(DefaultChannelPromise.java:129)
at io.netty.channel.DefaultChannelPromise.await(DefaultChannelPromise.java:28)
at io.netty.util.concurrent.DefaultPromise.sync(DefaultPromise.java:218)
at io.netty.channel.DefaultChannelPromise.sync(DefaultChannelPromise.java:117)
at io.netty.channel.DefaultChannelPromise.sync(DefaultChannelPromise.java:28)
at org.apache.livy.rsc.rpc.Rpc.close(Rpc.java:307)
at org.apache.livy.rsc.RSCClient.stop(RSCClient.java:232)
at org.apache.livy.rsc.RSCClient$2$1.onSuccess(RSCClient.java:129)
at org.apache.livy.rsc.RSCClient$2$1.onSuccess(RSCClient.java:123)
at org.apache.livy.rsc.Utils$2.operationComplete(Utils.java:108)
at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:680)
at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:567)
at io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:406)
at io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:82)
at io.netty.channel.AbstractChannel$CloseFuture.setClosed(AbstractChannel.java:956)
at io.netty.channel.AbstractChannel$AbstractUnsafe.doClose0(AbstractChannel.java:608)
at io.netty.channel.AbstractChannel$AbstractUnsafe.close(AbstractChannel.java:586)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.closeOnRead(AbstractNioByteChannel.java:71)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:158)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
at java.lang.Thread.run(Thread.java:748)
17/09/26 18:24:17 INFO RSCClient: Failing pending job 322a9264-ac0c-409b-961f-9b0cb98b215b due to shutdown.
17/09/26 18:24:17 INFO InteractiveSession: Failed to ping RSC driver for session 14. Killing application.
17/09/26 18:24:17 INFO InteractiveSession: Stopping InteractiveSession 14...
17/09/26 18:24:17 INFO YarnClientImpl: Killed application application_1505910137586_0055
17/09/26 18:24:17 INFO InteractiveSession: Stopped InteractiveSession 14.
When I remove the config (Vegas) it works fine. Is there anyway I can use Vegas with Livy?
Many thanks
maziyarpanahi commented
Just to add more details, looking at Livy logs I can see that the package is downloaded successfully:
17/09/27 17:15:10 INFO LineBufferedStream: stdout: org.vegas-viz#vegas_2.11 added as a dependency
17/09/27 17:15:10 INFO LineBufferedStream: stdout: :: resolving dependencies :: org.apache.spark#spark-submit-parent;1.0
17/09/27 17:15:10 INFO LineBufferedStream: stdout: confs: [default]
17/09/27 17:15:11 INFO LineBufferedStream: stdout: found org.vegas-viz#vegas_2.11;0.3.10 in central
17/09/27 17:15:12 INFO LineBufferedStream: stdout: found org.vegas-viz#vegas-macros_2.11;0.3.10 in central
17/09/27 17:15:12 INFO LineBufferedStream: stdout: found org.scala-lang#scala-reflect;2.11.8 in central
17/09/27 17:15:12 INFO LineBufferedStream: stdout: found org.typelevel#macro-compat_2.11;1.1.1 in central
17/09/27 17:15:12 INFO LineBufferedStream: stdout: found com.github.julien-truffaut#monocle-core_2.11;1.1.0 in central
17/09/27 17:15:13 INFO LineBufferedStream: stdout: found org.scalaz#scalaz-core_2.11;7.1.1 in central
17/09/27 17:15:13 INFO LineBufferedStream: stdout: found org.scala-lang.modules#scala-parser-combinators_2.11;1.0.2 in central
17/09/27 17:15:13 INFO LineBufferedStream: stdout: found org.scala-lang.modules#scala-xml_2.11;1.0.2 in central
17/09/27 17:15:13 INFO LineBufferedStream: stdout: found io.circe#circe-core_2.11;0.7.0 in central
17/09/27 17:15:13 INFO LineBufferedStream: stdout: found io.circe#circe-numbers_2.11;0.7.0 in central
17/09/27 17:15:13 INFO LineBufferedStream: stdout: found org.typelevel#cats-core_2.11;0.9.0 in central
17/09/27 17:15:13 INFO LineBufferedStream: stdout: found org.typelevel#cats-macros_2.11;0.9.0 in central
17/09/27 17:15:13 INFO LineBufferedStream: stdout: found com.github.mpilquist#simulacrum_2.11;0.10.0 in central
17/09/27 17:15:13 INFO LineBufferedStream: stdout: found org.typelevel#machinist_2.11;0.6.1 in central
17/09/27 17:15:13 INFO LineBufferedStream: stdout: found org.typelevel#cats-kernel_2.11;0.9.0 in central
17/09/27 17:15:13 INFO LineBufferedStream: stdout: found io.circe#circe-generic_2.11;0.7.0 in central
17/09/27 17:15:13 INFO LineBufferedStream: stdout: found com.chuusai#shapeless_2.11;2.3.2 in central
17/09/27 17:15:13 INFO LineBufferedStream: stdout: found io.circe#circe-parser_2.11;0.7.0 in central
17/09/27 17:15:13 INFO LineBufferedStream: stdout: found io.circe#circe-jawn_2.11;0.7.0 in central
17/09/27 17:15:13 INFO LineBufferedStream: stdout: found org.spire-math#jawn-parser_2.11;0.10.4 in central
17/09/27 17:15:13 INFO LineBufferedStream: stdout: found com.github.julien-truffaut#monocle-macro_2.11;1.1.0 in central
17/09/27 17:15:13 INFO LineBufferedStream: stdout: found org.scalafx#scalafx_2.11;8.0.92-R10 in central
17/09/27 17:15:13 INFO LineBufferedStream: stdout: found org.webjars.bower#vega-lite;1.2.0 in central
17/09/27 17:15:13 INFO LineBufferedStream: stdout: found org.webjars.bower#vega;2.6.5 in central
17/09/27 17:15:13 INFO LineBufferedStream: stdout: [2.6.5] org.webjars.bower#vega;[2.6.0,3)
17/09/27 17:15:13 INFO LineBufferedStream: stdout: found org.webjars.bower#d3;3.5.17 in central
17/09/27 17:15:13 INFO LineBufferedStream: stdout: [3.5.17] org.webjars.bower#d3;[3.5.6,4)
17/09/27 17:15:13 INFO LineBufferedStream: stdout: downloading https://repo1.maven.org/maven2/org/vegas-viz/vegas_2.11/0.3.10/vegas_2.11-0.3.10.jar ...
17/09/27 17:15:13 INFO LineBufferedStream: stdout: [SUCCESSFUL ] org.vegas-viz#vegas_2.11;0.3.10!vegas_2.11.jar (103ms)
17/09/27 17:15:13 INFO LineBufferedStream: stdout: downloading https://repo1.maven.org/maven2/org/vegas-viz/vegas-macros_2.11/0.3.10/vegas-macros_2.11-0.3.10.jar ...
17/09/27 17:15:13 INFO LineBufferedStream: stdout: [SUCCESSFUL ] org.vegas-viz#vegas-macros_2.11;0.3.10!vegas-macros_2.11.jar (5ms)
17/09/27 17:15:14 INFO LineBufferedStream: stdout: :: resolution report :: resolve 3859ms :: artifacts dl 189ms
17/09/27 17:15:14 INFO LineBufferedStream: stdout: :: modules in use:
17/09/27 17:15:14 INFO LineBufferedStream: stdout: com.chuusai#shapeless_2.11;2.3.2 from central in [default]
17/09/27 17:15:14 INFO LineBufferedStream: stdout: com.github.julien-truffaut#monocle-core_2.11;1.1.0 from central in [default]
17/09/27 17:15:14 INFO LineBufferedStream: stdout: com.github.julien-truffaut#monocle-macro_2.11;1.1.0 from central in [default]
17/09/27 17:15:14 INFO LineBufferedStream: stdout: com.github.mpilquist#simulacrum_2.11;0.10.0 from central in [default]
17/09/27 17:15:14 INFO LineBufferedStream: stdout: io.circe#circe-core_2.11;0.7.0 from central in [default]
17/09/27 17:15:14 INFO LineBufferedStream: stdout: io.circe#circe-generic_2.11;0.7.0 from central in [default]
17/09/27 17:15:14 INFO LineBufferedStream: stdout: io.circe#circe-jawn_2.11;0.7.0 from central in [default]
17/09/27 17:15:14 INFO LineBufferedStream: stdout: io.circe#circe-numbers_2.11;0.7.0 from central in [default]
17/09/27 17:15:14 INFO LineBufferedStream: stdout: io.circe#circe-parser_2.11;0.7.0 from central in [default]
17/09/27 17:15:14 INFO LineBufferedStream: stdout: org.scala-lang#scala-reflect;2.11.8 from central in [default]
17/09/27 17:15:14 INFO LineBufferedStream: stdout: org.scala-lang.modules#scala-parser-combinators_2.11;1.0.2 from central in [default]
17/09/27 17:15:14 INFO LineBufferedStream: stdout: org.scala-lang.modules#scala-xml_2.11;1.0.2 from central in [default]
17/09/27 17:15:14 INFO LineBufferedStream: stdout: org.scalafx#scalafx_2.11;8.0.92-R10 from central in [default]
17/09/27 17:15:14 INFO LineBufferedStream: stdout: org.scalaz#scalaz-core_2.11;7.1.1 from central in [default]
17/09/27 17:15:14 INFO LineBufferedStream: stdout: org.spire-math#jawn-parser_2.11;0.10.4 from central in [default]
17/09/27 17:15:14 INFO LineBufferedStream: stdout: org.typelevel#cats-core_2.11;0.9.0 from central in [default]
17/09/27 17:15:14 INFO LineBufferedStream: stdout: org.typelevel#cats-kernel_2.11;0.9.0 from central in [default]
17/09/27 17:15:14 INFO LineBufferedStream: stdout: org.typelevel#cats-macros_2.11;0.9.0 from central in [default]
17/09/27 17:15:14 INFO LineBufferedStream: stdout: org.typelevel#machinist_2.11;0.6.1 from central in [default]
17/09/27 17:15:14 INFO LineBufferedStream: stdout: org.typelevel#macro-compat_2.11;1.1.1 from central in [default]
17/09/27 17:15:14 INFO LineBufferedStream: stdout: org.vegas-viz#vegas-macros_2.11;0.3.10 from central in [default]
17/09/27 17:15:14 INFO LineBufferedStream: stdout: org.vegas-viz#vegas_2.11;0.3.10 from central in [default]
17/09/27 17:15:14 INFO LineBufferedStream: stdout: org.webjars.bower#d3;3.5.17 from central in [default]
17/09/27 17:15:14 INFO LineBufferedStream: stdout: org.webjars.bower#vega;2.6.5 from central in [default]
17/09/27 17:15:14 INFO LineBufferedStream: stdout: org.webjars.bower#vega-lite;1.2.0 from central in [default]
17/09/27 17:15:14 INFO LineBufferedStream: stdout: :: evicted modules:
17/09/27 17:15:14 INFO LineBufferedStream: stdout: org.scala-lang#scala-reflect;2.11.6 by [org.scala-lang#scala-reflect;2.11.8] in [default]
17/09/27 17:15:14 INFO LineBufferedStream: stdout: ---------------------------------------------------------------------
17/09/27 17:15:14 INFO LineBufferedStream: stdout: | | modules || artifacts |
17/09/27 17:15:14 INFO LineBufferedStream: stdout: | conf | number| search|dwnlded|evicted|| number|dwnlded|
17/09/27 17:15:14 INFO LineBufferedStream: stdout: ---------------------------------------------------------------------
17/09/27 17:15:14 INFO LineBufferedStream: stdout: | default | 26 | 4 | 2 | 1 || 25 | 2 |
17/09/27 17:15:14 INFO LineBufferedStream: stdout: ---------------------------------------------------------------------
17/09/27 17:15:14 INFO LineBufferedStream: stdout: :: retrieving :: org.apache.spark#spark-submit-parent
17/09/27 17:15:14 INFO LineBufferedStream: stdout: confs: [default]
17/09/27 17:15:14 INFO LineBufferedStream: stdout: 2 artifacts copied, 23 already retrieved (2438kB/18ms)
maziyarpanahi commented
Probably the number of JARS and size of some of them are not suitable for Livy. I fixed this by putting the JARS on HDFS.