ruby-numo/numo-narray

Loading Binary Data

ankane opened this issue · 5 comments

ankane commented

Hi, I'm working on loading Numpy .npy and .npz files into an NArray (which maybe we can include in this project at some point). The data can be read into Ruby with:

arr = byte_str.unpack("C*")

And later converted with:

Numo::UInt8[*arr]

However, it'd be great if there was a way to read directly into an NArray for efficiency (also, the current approach causes a stack level too deep error with large arrays - see #139 for more details).

Here's the code and test data.

mrkn commented

@ankane I guess Numo::NArray.from_string is appropriate for your purpose if you can detect the value type of deserialized arrays.

masa16 commented

Yes, Numo::UInt8.from_string(byte_str) is faster for conversion from binary string.

ankane commented

Not sure how I missed that (facepalm). I appreciate the help.

For future readers, you need to call it on a specific type, as calling it on Numo::NArray raises uninitialized constant Numo::NArray::ELEMENT_BYTE_SIZE.

ankane commented

Here's the gem, just fyi: https://github.com/ankane/npy

  • 🎉2