R crashes when reading nested data
alanpaulkwan opened this issue · 5 comments
I can provide (confidentially) a sample of data that is originated from a Parquet dataset that had very complicted nesting.
Thanks, that would be helpful to understand your problem. Could you even reduce it to a minimal subset of observations?
I didn't see your reply. If you want I could PM you credentials to one of my public clickhouse servers
Basically I discovered this trying to read Open Street Maps which has super nested data.
myclickhousewrapper("select * from osm.planet_history limit 1")
*** caught segfault ***
address (nil), cause 'memory not mapped'
Traceback:
1: select(conn@ptr, statement)
2: dbSendQuery(conn, statement, ...)
3: dbSendQuery(conn, statement, ...)
4: .local(conn, statement, ...)
5: dbGetQuery(con, q2, ...)
6: dbGetQuery(con, q2, ...)
7: queryClickhouseDb(q = q, fix = fix, con_fn = getConClickhouseServer3, ...)
8: queryClickhouseServer3("select * from osm.planet_history limit 1")
Possible actions:
1: abort (with core dump, if enabled)
2: normal R exit
3: exit R without saving workspace
4: exit R saving workspace
Selection:
Due to time constraints, we cannot provide free private support. Please provide a minimal example that demonstrates the issue. If your source is Open Street Maps, there should not be any license issues, right?
I understand, I was just trying to produce a replicable example by showing you a live database. I thought that was the easiest method.
What format of replicable example would you want? The underlying data, a schema, and a command to parse the file in?
If your problem is related to parquet, please provide a minimal Parquet file.