dealing with structs within transform
Closed this issue · 2 comments
Gauravshah commented
Thank you for adding this library, helpful to go via api route.
I am struggling to use transform for array<struct<cat:Long,ctg:string>>
transform(
col(qualifiedFieldName),
x=>{
x.cat
},
"el"
)
this throws compile time exception
transform(
col(qualifiedFieldName),
x=>{
col("el.cat")
},
"el"
)
throws org.apache.spark.sql.AnalysisException: cannot resolve '`el.aid`' given input columns
yruslan commented
Accessing elements of a struct inside an array is a common use case. Here is an example:
transform(
col(qualifiedFieldName),
x => {
x.getField("cat")
},
"el"
)
Gauravshah commented
thanks @yruslan