UWHustle/hustle

User-Defined Type in Arrow

GindaChen opened this issue · 3 comments

support custom data types (i.e. IP addrress was the running example at the time)

@johnchronis Im actually pretty confident that we can use the arrow::Extention type in arrow to implement IP Address. In fact, this mysterious Extention type is designed for user-defined type. The extention type can, in itself, defines its relationship with other types (ideally, of course, using what we have proposed ini #94).

In fact, arrow had a test IPv4 type in their code base:
https://github.com/apache/arrow/blob/2d140c3eeecca3ff7823edc8c9562ebd6a1c336a/cpp/src/arrow/array/array_view_test.cc#L55-L78

We can open up an issue to address user-defined type - will need to experiment a little bit on this.

Originally posted by @GindaChen in #28 (comment)

That is interesting, it seems like arrow has the necessary mechanisms to handle user-defined types and operations on them. So it seems we can extend the set of supported types in the future :)

Good digging @johnchronis!

Close this conversation as we are pretty sure arrow can do user-defined type in the current version. @johnchronis New issues are welcomed when you have a list of UDTs in mind.