Assigning `f8` 2d data to a table can lead to corrupt output
Opened this issue · 2 comments
braingram commented
Let's assume we have a schema for TableModel
that contains
one_column_table:
fits_hdu: ONECOL
datatype:
- name: col
datatype: float64
If we assign data to the model as follows, the produced file is corrupt:
fn = tmp_path / "test.fits"
write_model = TableModel()
arr = np.arange(16 * 32, dtype='f8').reshape((16, 32))
write_model.one_column_table = arr
np.testing.assert_equal(arr, write_model.one_column_table['col'])
write_model.save(fn)
with TableModel(fn) as read_model:
np.testing.assert_equal(arr, read_model.one_column_table['col'])
The above code fails on TableModel(fn)
with:
E astropy.utils.exceptions.AstropyUserWarning: ERROR loading embedded ASDF: non-ASCII characters are present in the FITS file header and have been replaced by "?" characters
This can be further reduced to just astropy:
from astropy.io import fits
import numpy as np
hdulist = fits.HDUList()
hdulist.append(fits.PrimaryHDU())
arr = np.arange(32 * 16).reshape((32, 16)).astype(dtype=[('col', 'f8')])
hdulist.append(fits.BinTableHDU())
hdulist[-1].data = arr
hdulist.writeto('foo.fits', overwrite=True)
with fits.open('foo.fits') as ff:
ff.info()
Running the above code results in:
UnicodeDecodeError: 'ascii' codec can't decode byte 0x80 in position 2: ordinal not in range(128)
and
OSError: Header missing END card.
Note that in all cases the error doesn't occur until the data is read. No errors are issued on write.
braingram commented
Using f4
does not produce the same error but does produce a file with an incorrectly shaped array ((32)
instead of (32, 16)
).