NVIDIA/spark-rapids

[AUDIT] [SPARK-49653][SQL] Single join for correlated scalar subqueries

Opened this issue · 1 comments

amahussein commented

apache/spark@3c81f07

This Spark PR introduces a new join type LeftSingle that triggered changes in the query execution

  • sql/core/src/main/scala/org/apache/spark/sql/execution/joins/ShuffledJoin.scala
  • sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashJoin.scala
  • sql/core/src/main/scala/org/apache/spark/sql/execution/joins/BroadcastNestedLoopJoinExec.scala
  • sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/PlanAdaptiveSubqueries.scala

The PR adds single join implementation to support correlated scalar subqueries and introduces a new config spark.sql.optimizer.scalarSubqueryUseSingleJoin

revans2 commented

I don't think that the link above is for this issue. The PR is apache/spark#48145 which appears to correspond to apache/spark@3c81f07 instead

  • 👍1