Writing a new file with compressed data with only 1 entry

[pootle]

I'm trying to take an existing file, update some header fields and then save old data with updated header to a new file with compression. Using compression, the file has 2 HDUs, the first is empty, but confuses other apps.

import fitsio
import numpy as np

fitsio.write
('testout.fits', np.zeros((5,4)), clobber=True, compress='GZIP_2')
checkfits = fitsio.FITS('testout.fits')
print('all at once', len(checkfits))
print(checkfits[0].read())
checkfits.close()
fitsio.write('testout.fits', np.zeros((5,4)), clobber=True)
checkfits = fitsio.FITS('testout.fits')
print('all at once no compression', len(checkfits)) 
checkfits.close()

gives

all at once 2
None
all at once no compression 1

I have tried using

afits = fitsio.FITS('testout.fits', 'rw')
...

but I can only set compression by calling afits.write(...) which creates another HDU

[esheldon]

Thanks for the report. Looks like the write works, but the object is not being updated after write. It works after leaving a context, see below

import fitsio
import numpy as np

data = np.random.uniform(size=(5, 4))

for comp in (None, 'GZIP_2'):
    print('-' * 70)
    print(f'testing compress {comp}')
    fname = f'test-{comp}.fits'
    with fitsio.FITS(fname, 'rw', clobber=True) as fits:
        fits.write(data, compress=comp)
        print('test from object')
        rdata = fits[0].read()
        print(rdata)
        if rdata is not None:
            assert np.all(rdata == data)
        else:
            print('is None')

    print('test after close')
    rdata = fitsio.read(fname)
    assert np.all(rdata == data)

[pootle]

Ah! that wasn't quite the problem I'm having. It's the fact I get 2 HDU's in the file when using compression:

import fitsio
import numpy as np

data = np.random.uniform(size=(5, 4))

for comp in (None, 'GZIP_2'):
    print('-' * 70)
    print(f'testing compress {comp}')
    fname = f'test-{comp}.fits'
    with fitsio.FITS(fname, 'rw', clobber=True) as fits:
        fits.write(data, compress=comp)
        print('test from object')
        rdata = fits[0].read()
        print(rdata)
        if rdata is not None:
            assert np.all(rdata == data)
        else:
            print('data is None')

    print('test after close')
    rdata = fitsio.read(fname)
    assert np.all(rdata == data)
    with fitsio.FITS(fname, 'r') as fits:
        for xctr, anhdu in enumerate(fits):
            rdata=anhdu.read()
            if rdata is None:
                print('hdu %d is None' % xctr)
            elif np.all(rdata == data):
                print('hdu %d is OK' % xctr)
            else:
                print('hdu %d is %s' % (xctr, rdata))

gives me:

----------------------------------------------------------------------
testing compress None
test from object
[[0.91227526 0.03923437 0.76965836 0.15452345]
 [0.28061116 0.19342035 0.72939983 0.96776926]
 [0.4501779  0.2932473  0.19273068 0.26375251]
 [0.75963652 0.07441869 0.82223672 0.19808283]
 [0.6705528  0.16009362 0.15806931 0.13130748]]
test after close
hdu 0 is OK
----------------------------------------------------------------------
testing compress GZIP_2
test from object
None
data is None
test after close
hdu 0 is None
hdu 1 is OK

[beckermr]

2 hdus with compression is expected. That's how the tile compressed image standard is implemented in fits.

[pootle]

Ah!, must be software that can't cope with compressed files then - or I confused it somehow, I'll have another go!
thanks very much.

[esheldon]

There is still the issue that the fits object isn't being updated after the write

[esheldon]

actually never mind, this is the same thing, due to data being in the second hdu