Vindaar/nimhdf5

H5T_STD_I32LE is converted to int64. and data becomes garbage

hkrish opened this issue · 6 comments

hkrish commented

If I try echo h5ToNimType(H5T_STD_I32LE) nimhdf5 prints dkInt64

Since I am trying to read an array of int32 values (I have no control over how the data is generated, it is from Matlab), this produces wrong values.
It seems I can recover the right values by iterating over what nimhdf5 returns with it.bitand(0xffffffff), it.shr(32) and recover the original values, but this is a bit "hacky".

The h5dump tool prints the correct datatype and values

DATASET ... {
      DATATYPE  H5T_STD_I32LE
      DATASPACE  SIMPLE { ( 1, 109 ) / ( 1, 109 ) }
      ...

Is it possible to fix this?

Thank you

hkrish commented

For now this works for me,

proc forceReadAs[T: SomeInteger](mat: H5FileObj, path: string, dtype: typedesc[T]): seq[T] =
  let dset = mat.datasets[path]
  dset.dtype = $dtype
  return dset[T]
Vindaar commented

Thanks for the issue!

Yes, that should be easy to fix. I misunderstood the HDF5 mappings for the STD_* types and their NATIVE_* counterparts.

I'll push a fix in a few minutes.

Vindaar commented

Once #36 is merged, let me know whether it works correctly please.

hkrish commented

Sure thing.
Thank you very much for such a quick fix.

Vindaar commented

Sorry for the auto close due to the merge.

New version is pushed.

hkrish commented

Great. It correctly maps int32 now.
Thank you.

  • 🎉1