NVIDIA/spark-rapids

[BUG] parse_url throws exception if partToExtract is invalid while Spark returns null

Closed this issue · 0 comments

Describe the bug
If the partToExtract is not valid, gpu throws exception while cpu just returns null.

Steps/Code to reproduce bug

scala> val df = spark.read.parquet("kaggle_url_dataset")
df: org.apache.spark.sql.DataFrame = [url: string]
scala> spark.time(df.selectExpr("parse_url(url, 'protocol')").show())
24/10/25 17:44:39 WARN GpuOverrides:
!Exec <CollectLimitExec> cannot run on GPU because the Exec CollectLimitExec has been disabled, and is disabled by default because Collect Limit replacement can be slower on the GPU, if huge number of rows in a batch it could help by limiting the number of rows transferred from GPU to CPU. Set spark.rapids.sql.exec.CollectLimitExec to true if you wish to enable it
  @Partitioning <SinglePartition$> could run on GPU
  *Exec <ProjectExec> will run on GPU
    *Expression <Alias> parse_url(url#0, protocol, false) AS parse_url(url, protocol)#13 will run on GPU
      *Expression <ParseUrl> parse_url(url#0, protocol, false) will run on GPU
    *Exec <FileSourceScanExec> will run on GPU

24/10/25 17:44:39 ERROR Executor: Exception in task 2.0 in stage 4.0 (TID 9)
java.lang.IllegalArgumentException: Invalid partToExtract: GPU_SCALAR StringType Some(protocol) None
	at org.apache.spark.sql.rapids.GpuParseUrl.doColumnar(GpuParseUrl.scala:76)
	at org.apache.spark.sql.rapids.GpuParseUrl.$anonfun$columnarEval$2(GpuParseUrl.scala:107)
	at com.nvidia.spark.rapids.Arm$.withResourceIfAllowed(Arm.scala:84)
	at org.apache.spark.sql.rapids.GpuParseUrl.$anonfun$columnarEval$1(GpuParseUrl.scala:104)
	at com.nvidia.spark.rapids.Arm$.withResourceIfAllowed(Arm.scala:84)
	at org.apache.spark.sql.rapids.GpuParseUrl.columnarEval(GpuParseUrl.scala:103)
	at com.nvidia.spark.rapids.RapidsPluginImplicits$ReallyAGpuExpression.columnarEval(implicits.scala:34)
	at com.nvidia.spark.rapids.GpuAlias.columnarEval(namedExpressions.scala:110)
	at com.nvidia.spark.rapids.RapidsPluginImplicits$ReallyAGpuExpression.columnarEval(implicits.scala:34)
	at com.nvidia.spark.rapids.GpuProjectExec$.$anonfun$project$1(basicPhysicalOperators.scala:122)
	at com.nvidia.spark.rapids.RapidsPluginImplicits$MapsSafely.$anonfun$safeMap$1(implicits.scala:220)
	at com.nvidia.spark.rapids.RapidsPluginImplicits$MapsSafely.$anonfun$safeMap$1$adapted(implicits.scala:217)
	at scala.collection.immutable.List.foreach(List.scala:431)
.......
.......

scala> spark.conf.set("spark.rapids.sql.enabled", "false")

scala> spark.time(df.selectExpr("parse_url(url, 'protocol')").show())
+------------------------+
|parse_url(url, protocol)|
+------------------------+
|                    null|
|                    null|
|                    null|
|                    null|
|                    null|
|                    null|
|                    null|
|                    null|
|                    null|
|                    null|
|                    null|
|                    null|
|                    null|
|                    null|
|                    null|
|                    null|
|                    null|
|                    null|
|                    null|
|                    null|
+------------------------+
only showing top 20 rows

Time taken: 493 ms

Expected behavior
Behaviour should match.