Don't drop annotations in source data
Opened this issue · 3 comments
See also this similar issue in PartiQL-Rust: partiql/partiql-lang-rust#371
Description
PartiQL drops all annotations in source data, but it shouldn't. Annotations include important context needed for downstream applications to know how to process the query results. In effect, PartiQL is corrupting the data that's fed into it. This will likely become a big issue for my team's PartiQL product in the next few months.
To Reproduce
Welcome to the PartiQL shell!
Typing mode: LEGACY
Using version: 0.9.2-f3b92a82
PartiQL> select * from `[ { foo: { bar: some_annotation::42 } } ]`
|
===' from source
<<
{
'foo': {
'bar': 42
}
}
>>
---
OK!
As shown above, the value 42 is missing the annotation.
Expected Behavior
The result of the query above should be:
<<
{
'foo': {
'bar': some_annotation::42
}
}
>>
PartiQL drops all annotations in source data, but it shouldn't.
The PartiQL Specification does not include annotations in the data model; meaning the expected result isn't a valid PartiQL Value (Page 5, Figure 1).
<<
{
'foo': {
'bar': some_annotation::42
}
}
>>
We have had a similar ask to add annotations to the data model, but we will need to do the necessary specification work before extending the data model.
A paragraph or two below figure 1 it also states:
PartiQL’s data model extends SQL to Ion’s type system to cover schema-less and nested data.
Which I have always taken to mean that PartiQL's type system is a proper superset of Ion's, thus, anything you can represent in Ion can also be represented in PartiQL values. (But not necessarily the inverse.) Hence why I filed this issue.
As discussed offline, PartiQL cannot be a super-set of Ion without annotation tokens on values. This will require an RFC for spec additions and clarifications, then the feature work to support it.