请问i你这个项目现在还在维护么,我在apache/submarine的项目下进行的编译,但是在spark3.0.1的版本下无法编译通过,报错:
Closed this issue · 1 comments
[INFO] Compiling 38 Scala sources and 6 Java sources to /home/mahao/submarine-master/submarine-security/spark-security/target/scala-2.12/classes ...
[ERROR] /home/mahao/submarine-master/submarine-security/spark-security/src/main/scala/org/apache/spark/sql/catalyst/optimizer/SubmarineDataMaskingExtension.scala:236: not enough arguments for method apply: (child: org.apache.spark.sql.catalyst.plans.logical.LogicalPlan, correlated: Boolean)org.apache.spark.sql.catalyst.plans.logical.Subquery in object Subquery.
Unspecified value parameter correlated.
[ERROR] /home/mahao/submarine-master/submarine-security/spark-security/src/main/scala/org/apache/spark/sql/catalyst/optimizer/SubmarineDataMaskingExtension.scala:236: wrong number of arguments for pattern org.apache.spark.sql.catalyst.plans.logical.Subquery(child: org.apache.spark.sql.catalyst.plans.logical.LogicalPlan,correlated: Boolean)
[ERROR] /home/mahao/submarine-master/submarine-security/spark-security/src/main/scala/org/apache/spark/sql/catalyst/optimizer/SubmarineRowFilterExtension.scala:66: not enough arguments for method apply: (child: org.apache.spark.sql.catalyst.plans.logical.LogicalPlan, correlated: Boolean)org.apache.spark.sql.catalyst.plans.logical.Subquery in object Subquery.
Unspecified value parameter correlated.
[ERROR] /home/mahao/submarine-master/submarine-security/spark-security/src/main/scala/org/apache/spark/sql/catalyst/optimizer/SubmarineRowFilterExtension.scala:65: wrong number of arguments for pattern org.apache.spark.sql.catalyst.plans.logical.Subquery(child: org.apache.spark.sql.catalyst.plans.logical.LogicalPlan,correlated: Boolean)
[ERROR] /home/mahao/submarine-master/submarine-security/spark-security/src/main/scala/org/apache/spark/sql/catalyst/optimizer/SubmarineSparkRangerAuthorizationExtension.scala:55: not found: type ShowDatabasesCommand
[ERROR] /home/mahao/submarine-master/submarine-security/spark-security/src/main/scala/org/apache/spark/sql/execution/SubmarineShowDatabasesCommand.scala:23: object ShowDatabasesCommand is not a member of package org.apache.spark.sql.execution.command
[ERROR] /home/mahao/submarine-master/submarine-security/spark-security/src/main/scala/org/apache/spark/sql/execution/SubmarineShowDatabasesCommand.scala:27: not found: type ShowDatabasesCommand
[ERROR] /home/mahao/submarine-master/submarine-security/spark-security/src/main/scala/org/apache/spark/sql/catalyst/optimizer/SubmarineSparkRangerAuthorizationExtension.scala:147: not found: type SetDatabaseCommand
[ERROR] /home/mahao/submarine-master/submarine-security/spark-security/src/main/scala/org/apache/spark/sql/catalyst/optimizer/SubmarineSparkRangerAuthorizationExtension.scala:150: not found: type ShowDatabasesCommand
[ERROR] /home/mahao/submarine-master/submarine-security/spark-security/src/main/scala/org/apache/spark/sql/hive/PrivilegesBuilder.scala:31: object PersistedView is not a member of package org.apache.spark.sql.execution.command
[ERROR] /home/mahao/submarine-master/submarine-security/spark-security/src/main/scala/org/apache/spark/sql/hive/PrivilegesBuilder.scala:122: value tableIdentifier is not a member of org.apache.spark.sql.catalyst.analysis.UnresolvedRelation
[ERROR] /home/mahao/submarine-master/submarine-security/spark-security/src/main/scala/org/apache/spark/sql/hive/PrivilegesBuilder.scala:206: type mismatch;
found : Option[Seq[String]]
required: Seq[String]
[ERROR] /home/mahao/submarine-master/submarine-security/spark-security/src/main/scala/org/apache/spark/sql/hive/PrivilegesBuilder.scala:208: type mismatch;
found : Option[Seq[String]]
required: Seq[String]
[ERROR] /home/mahao/submarine-master/submarine-security/spark-security/src/main/scala/org/apache/spark/sql/hive/PrivilegesBuilder.scala:255: not found: value PersistedView
[ERROR] /home/mahao/submarine-master/submarine-security/spark-security/src/main/scala/org/apache/spark/sql/hive/PrivilegesBuilder.scala:331: not found: type SetDatabaseCommand
[ERROR] /home/mahao/submarine-master/submarine-security/spark-security/src/main/scala/org/apache/spark/sql/hive/PrivilegesBuilder.scala:331: value databaseName is not a member of org.apache.spark.sql.catalyst.plans.logical.LogicalPlan
[ERROR] 16 errors found
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 9.462 s
[INFO] Finished at: 2020-11-03T15:34:29+08:00
[INFO] Final Memory: 173M/3452M
[INFO] ------------------------------------------------------------------------
[WARNING] The requested profile "spark-2.4.7" could not be activated because it does not exist.
[ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:4.4.0:compile (scala-compile-first) on project submarine-spark-security: Execution scala-compile-first of goal net.alchim31.maven:scala-maven-plugin:4.4.0:compile failed. CompileFailed -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
其中的scala版本我使用的是2.12.10,同样的情况下使用spark2.4.7是可以编译通过的,我还没有集成到我的spark环境里面去,因为我们现在计划上spark3.X的版本。所以请问您,相关spark-ranger的这个项目,您有计划支持spark3.X么
这个项目已经托管submarine社区了,我个人暂时没有时间做3.x的支持,有需要的话可以直接参与社区