ncar-xdev/xpersist

changes in precision on write changes answers

Opened this issue · 0 comments

Computation often defaults to double precision, but xarray writes data with encoding['dtype']. If encoding['dtype'] has been inherited from a previous file read, for instance, and does not match DataArray.values.dtype, xarray will truncate precision when writing the file.

This means that using an xpersist cache file might change answers of a computation, which is not desirable.

How to fix?

  • Write data at full precision (DataArray.values.dtype)
  • Retain the encoding information in the file and repopulate it on read.

cc @klindsay28