DiskFrame/disk.frame

Support for Big-endian platforms: how to test?

Opened this issue · 3 comments

fst currently is broken for Big-endian platforms. Is it possible to forego it, or is it essential to the core functionality?

Yes it is essential for now and foreseeable future. It's well documented that fst doesn't work on big-endian though.

What do you hope to use disk.frame for? Would {arrow} suffice for your needs?

I don't feel the need to invest in disk.frame if {arrpw} with parquet support is good enough

@xiaodaigh

It's well documented that fst doesn't work on big-endian though

In fact I tried to build fst today and run tests, and everything works fine on Big-endian. No idea how that claim on CRAN originated, but it is misleading.

However turned out arrow wants a prebuilt lib, which does not exist for PPC, obviously. So now stuck with another dependency.

P. S. I am adding R packages into Macports, and for that I need all archs to be working.

UPD. arrow fixed, R-arrow has a minor issue, but soon I can build disk.frame on PPC :)

@xiaodaigh Ok, I have built everything, including data.frame. However since there are no tests, I have no idea if it works correctly. (My concern is primarily endianness.) How may I check it?