Bug with saving masked arrays (inconsistent fill value dtype)
Opened this issue · 1 comments
When a cube with a masked array is saved, if the masked array does not have a fill value specified, it will use the numpy default value which hass dtype np.float64. This may be inconsistent with the dtype of the array itself, and when the array is filled (here) it will be filled with a value that has the dtype of the array (e.g. np.float32). This will then be inconsistent with the fill value described on this line which will then cause the data to be saved incorrectly.
Changing this line to the following seems to fix this problem:
fill_value = float(np.array(cube.data.fill_value, dtype=cube.data.dtype))
it was an obstacle for me. I try to convert NetCDF to grib2 and in the case when 32-bit NetCDF-file doesn't contain missing_value, np.ma.MaskedArray chooses default float fill_value (1e20), which is converting incorrectly (np.float64(np.float32(1e20)) != np.float64(1e20)) and bit Map (from 7 grib-section) isn't filling.
Eventually, I fixed it by recreating MaskedArray:
cube.data = np.ma.MaskedArray(cube.data.data, fill_value=cub.data.fill_value)