Add Nested data type support
slvrtrn opened this issue · 2 comments
Hi @slvrtrn!
I'm willing to work on adding support for nested data types.
My approach would be to flatten the nested structures, such as objects. So, I would choose the Record<K, V> TS type to support the Nested data type, because both have the same goal, to handle structured data.
If we assume the explanation, then we face incompatibility with the Map(K, V) data type. According to the official docs, the Map(K, V) data type is supported by the Record<K, V> TS type.
This means that I won’t be able to differentiate between Map and Nested when attempting to flatten them.
In my opinion, the Map(K, V) data type should be supported by the Map<K, V> TS type. There are a couple of advantages to supporting it as a TS Map:
- It would create semantic coherence between the Clickhouse data type and the TS type.
- The TS Map API is better suited for handling data map structures than the TS Record type.
What do you think about supporting Clickhouse Map with TS Map?
Many thanks
Hi @antoniovizuete, thanks for your interest in the project.
IIRC, JS Maps introduction will likely require additional decoding effort when working with JSON* types. Right now, it just does JSON.stringify/parse (see the source), but with JS Maps, we will likely need to replace it with a better codec. I guess we just wanted to keep it as simple as possible.
Another question, for datasets with large maps, do we expect significant serialization/deserialization performance difference between the current implementation (plain objects) and JS Maps?
As for Nested, I don't think the problem is in the Record<K, V> type that we use; for tests, I could not figure out a proper way to insert records with Nested - again, IIRC, the inserted rows had empty columns no matter what without the actual Nested object.