Is it possible to support sparksql parser，so that coral can support sparksql to hive/trino?
ImTangYun opened this issue · 4 comments
From document I know that,now coral support:
Hive to Trino
Hive to Spark
Trino to Spark
Spark to other is not support, but many company mainly use sparksql, it may be very useful to to support sparksql to others
Spark SQL is compatible with Hive QL so using the coral-hive module for Spark queries can be a good start. Is there a specific difference between Spark SQL and Hive QL that you want to see supported?
Spark SQL is compatible with Hive QL so using the
coral-hivemodule for Spark queries can be a good start. Is there a specific difference between Spark SQL and Hive QL that you want to see supported?
case 1:
SELECT
event.event_info [ 'key1' ]
FROM
test.test1
limit 1
with event an complex type: event struct<event_info map<string, string>>
get error:
org.apache.calcite.runtime.CalciteContextException: At line 0, column 0: Table 'event' not found
case 2:
SELECT
uid,
percentile_approx(score, ARRAY(0.5, 0.9, 0.99)) AS pa
FROM
tmp.test2
GROUP BY
uid
ORDER BY
uid ASC
get error:
CalciteContextException: At line 0, column 0: Expression 'score' is not being grouped
case 3:
create table if not exists test.test4 as
select * from test.test3
get error:
Unhandled Hive AST token 665 TOK_CREATETABLE
case 4:
select location,count(1) from tmp.room
where location = 1
group by 1
get error:
At line 0, column 0: Expression 'location' is not being grouped
There also have some other error
Spark SQL is compatible with Hive QL
I don't think it's fully compatible. For instance, Spark's raw string literal r'...' and ILIKE ar unsupported in Hive.