apache/datafusion-comet

Support array extraction

Closed this issue · 0 comments

What is the problem the feature request solves?

Support for extracting elements from an array. Spark has two expressions for doing this: GerArrayItem and ElementAt. These have a lot in common, with only minor differences for zero or one based, negative indices, and a default value, so they can probably be combined into a single DataFusion expression.

Describe the potential solution

Currently DataFusion has an ArrayElement expression which acts like ElementAt, but without the fail on error ANSI support and default value support, which is only needed for the SplitPart Spark expression via runtime replacement. It makes more sense to implement a custom function to support these behaviors, as well as also handle the GetArrayItem functionality that is zero based and doesn't support negative indices.

Additional context

No response