Support spark 3.5
melin opened this issue · 2 comments
melin commented
Support spark 3.5
davidrabinowitz commented
The spark-3.4-bigquery connector can be used on Spark 3.5. Are there any specific spark 3.5 features you need to be supported?
melin commented
The spark-3.4-bigquery connector can be used on Spark 3.5. Are there any specific spark 3.5 features you need to be supported?
We're excited about some of the improvements in Spark 3.5, e.g.:
Allow ShuffleDriverComponent to declare if shuffle data is reliably stored SPARK-42689
Allow V2 writes to indicate advisory shuffle partition size SPARK-42779
Any plan on support Spark 3.5?