linkedin/coral

Extract_union doesnt return struct

aastha25 opened this issue · 0 comments

PR #409 enabled the conversion of single union type hive field schema, such as uniontype<array<string>>, to array<string> in Coral's SqlNode & RelNode representation.

However, it breaks extract_union contract of returning only structs, 2- manages this new contract in another unrelated operator (DOT) with a method to guess the context as reported here . Hence, it needs to be revisited & should support the following scenarios across engines:

for a table foo: {a - int, b - uniontype<string>, c - uniontype<c1 - int, c2 - string, c3 - uniontype<string>>}

(1) select b from foo; // string
(2) select extract_union(b) from foo; // {tag_0:string}
(3) select extract_union(a) from foo; // {tag_0:string}
(4) select extract_union(c) from foo; // {c1: int, c2: string, c3: string}
(5) select extract_union(c).c3 from foo; // string
(6) select extract_union(c.c3) from foo; // {tag_0:string}