There is a transient "Unhandled memory fault at #x0." error when calling `make-arrow-schema-new`
kat-co opened this issue · 2 comments
This does not happen consistently, and some testing suggests that it's random. This leads me to believe it's some kind of race-condition. It can be worked around by calling the function again with the same parameters.
The utility function schema-from-object calls this, and as a temporary work-around, I plan on making it retry this call if it fails.
Possibly related: I wasn't able to figure out how to free a GArrowSchema entirely when using purely native code. I've filed a question in the Apache Arrow issue tracker; I wonder if it may turn out to be a bug.
I think we've pinned this down. The issue was that even though the pointer to a GList was in foreign-memory and thus immune from garbage collection, the data being held by a GList node was being allocated from cl-gobject-introspection which ensures foreign memory is freed when the Lisp handle is GCed. So when we were creating objects to store in a GList, they were sometimes getting GCed before make-arrow-schema-new could enumerate the GList.
I've fixed this by introducing a data-structure which couples the lifetime of a GList and the data it contains. I should have a PR tomorrow.