Compilation Issue - Spark 2.3.0 and Hadoop - 2.7.3
userguy opened this issue · 1 comments
[success] Total time: 0 s, completed Aug 10, 2020 2:16:54 PM
[info] Updating {file:/home/drelephant/dr-elephant/}root...
[info] Resolving org.apache.spark#spark-core_2.10;2.3.0 ...
error] SERVER ERROR: Bad Gateway url=http://repo.typesafe.com/typesafe/releases/org/apache/spark/spark-core_2.10/2.3.0/spark-core_2.10-2.3.0.pom
[warn] module not found: org.apache.spark#spark-core_2.10;2.3.0
[warn] ==== local: tried
[warn] /home/hdfs/.ivy2/local/org.apache.spark/spark-core_2.10/2.3.0/ivys/ivy.xml
[warn] ==== activator-launcher-local: tried
[warn] /home/hdfs/drelephant/activator-dist-1.3.12/repository/org.apache.spark/spark-core_2.10/2.3.0/ivys/ivy.xml
[warn] ==== public: tried
[warn] https://repo1.maven.org/maven2/org/apache/spark/spark-core_2.10/2.3.0/spark-core_2.10-2.3.0.pom
[warn] ==== typesafe-releases: tried
[warn] https://repo.typesafe.com/typesafe/releases/org/apache/spark/spark-core_2.10/2.3.0/spark-core_2.10-2.3.0.pom
[warn] ==== typesafe-ivy-releasez: tried
[warn] https://repo.typesafe.com/typesafe/ivy-releases/org.apache.spark/spark-core_2.10/2.3.0/ivys/ivy.xml
[warn] ==== Typesafe Releases Repository: tried
[warn] http://repo.typesafe.com/typesafe/releases/org/apache/spark/spark-core_2.10/2.3.0/spark-core_2.10-2.3.0.pom
[info] Resolving org.apache#apache;7 ...
[error] SERVER ERROR: Bad Gateway url=http://repo.typesafe.com/typesafe/releases/org/apache/apache/7/apache-7.jar
[error] SERVER ERROR: Bad Gateway url=http://repo.typesafe.com/typesafe/releases/org/apache/commons/commons-parent/17/commons-parent-17.jar
[info] Resolving org.eclipse.jetty#jetty-parent;14 ...
[error] SERVER ERROR: Bad Gateway url=http://repo.typesafe.com/typesafe/releases/org/eclipse/jetty/jetty-parent/14/jetty-parent-14.jar
[error] SERVER ERROR: Bad Gateway url=http://repo.typesafe.com/typesafe/releases/org/mortbay/jetty/jetty-parent/10/jetty-parent-10.jar
[error] SERVER ERROR: Bad Gateway url=http://repo.typesafe.com/typesafe/releases/org/mortbay/jetty/project/6.1.26/project-6.1.26.jar
[info] Resolving net.java#jvnet-parent;1 ...
[error] SERVER ERROR: Bad Gateway url=http://repo.typesafe.com/typesafe/releases/com/sun/jersey/jersey-project/1.9/jersey-project-1.9.jar
[info] Resolving org.slf4j#slf4j-parent;1.7.10 ...
[error] SERVER ERROR: Bad Gateway url=http://repo.typesafe.com/typesafe/releases/org/slf4j/slf4j-parent/1.7.10/slf4j-parent-1.7.10.jar
[info] Resolving com.google#google;5 ...
[error] SERVER ERROR: Bad Gateway url=http://repo.typesafe.com/typesafe/releases/com/google/google/5/google-5.jar
[error] SERVER ERROR: Bad Gateway url=http://repo.typesafe.com/typesafe/releases/com/google/inject/guice-parent/3.0/guice-parent-3.0.jar
[error] SERVER ERROR: Bad Gateway url=http://repo.typesafe.com/typesafe/releases/com/google/inject/extensions/extensions-parent/3.0/extensions-parent-3.0.jar
[info] Resolving org.sonatype.forge#forge-parent;6 ...
[error] SERVER ERROR: Bad Gateway url=http://repo.typesafe.com/typesafe/releases/org/sonatype/forge/forge-parent/6/forge-parent-6.jar
[info] Resolving asm#asm-parent;3.1 ...
[error] SERVER ERROR: Bad Gateway url=http://repo.typesafe.com/typesafe/releases/asm/asm-parent/3.1/asm-parent-3.1.jar
[info] Resolving com.sun.jersey#jersey-project;1.9 ...
[error] SERVER ERROR: Bad Gateway url=http://repo.typesafe.com/typesafe/releases/com/sun/jersey/contribs/jersey-contribs/1.9/jersey-contribs-1.9.jar
[info] Resolving org.apache.hadoop#hadoop-yarn;2.7.3 ...
[error] SERVER ERROR: Bad Gateway
Hi @userguy , the dependecy is wrong as Spark-Core_2.3 is not compatible with Scala 2.10.
Kindly try this branch: https://github.com/ShubhamGupta29/dr-elephant/tree/feature_spark2.3, I will merge this branch soon into master. This would work for you, just don't change any version as the default version for this branch is 2.3.4. I am closing this ticket, feel free to re-open if there is some issue.