wcs-addpv.py causes python segfault when used with astropy-2.0.16

[kirxkirx]

The script stopped working (presumably) after I upgraded my system from astropy-1 to astropy-2. I've added import faulthandler; faulthandler.enable() at the beginning of the script to see the traceback:

$ ./wcs-addpv.py wcs_Sgr4_2019-11-4_15-17-19_001.fts                                              
WARNING: FITSFixedWarning: RADECSYS= 'FK5 ' / Equatorial coordinate system 
the RADECSYS keyword is deprecated, use RADESYSa. [astropy.wcs.wcs]
Fatal Python error: Segmentation fault

Current thread 0x00007f75ec52b740 (most recent call first):
  File "/usr/lib64/python3.6/site-packages/astropy/wcs/wcs.py", line 645 in fix
  File "/usr/lib64/python3.6/site-packages/astropy/wcs/wcs.py", line 500 in __init__
  File "lib/wcs-addpv.py", line 136 in addpv
  File "lib/wcs-addpv.py", line 287 in main
  File "lib/wcs-addpv.py", line 298 in <module>
Segmentation fault

I get the same error if I switch from python3.6 to python2.7. The problem can be fixed by commenting out lines 643-651 in astropy/wcs/wcs.py

        if self.wcs is not None:
            self._fix_scamp()
            fixes = self.wcs.fix(translate_units, naxis)
            for key, val in six.iteritems(fixes):
                if val != "No change":
                    warnings.warn(
                        ("'{0}' made the change '{1}'.").
                        format(key, val),
                        FITSFixedWarning)

I'm reporting the issue here instead of astropy because I don't fully understand the interactions between wcs-addpv.py and astropy. (sorry!) Can you please check if the problem can be fixed on the sippv side and if not, file an upstream bug report to astropy providing a more meaningful description of the problem?

The files needed to reproduce the error (the TAN-SIP FITS image, the traceback-enabled version of wcs-addpv.py and the modified astropy/wcs/wcs.py that avoids the segfault) may be found here.

[evertrol]

Thanks for the report!

I haven't looked at this code in several years, so I'm not sure if I can find (easily) what could be wrong.

I did, however, just try with Python 3.8 and AstroPy 4.0 (just out). And that combination gave no problems. Or at least, no errors: I can't guarantee that the solution is correrct (I have used your example file; thanks for that).
So, this may be a problem in AstroPy that has been fixed since 2.0.16.

Would there be any chance for you to try and upgrade? I don't expect it's the Python version that makes the difference, more the AstroPy version (or possibly an underlying NumPy version, but the error doesn't indicate that either).
Perhaps just with a virtual-env or a Conda environment for testing purposes at first.

For the record, here are the (new) values the script added to the header of your test FITS file:

PV1_1    = 1.0
PV2_1    = 1.0
PV1_2    = -5.6896282994755e-08
PV2_2    = 1.66269651518153e-06
PV1_4    = 4.70195409013084e-05
PV2_4    = -0.00016526154191509
PV1_5    = -0.00014503167230489
PV2_5    = 1.44848477163303e-05
PV1_6    = -4.0576187723794e-05
PV2_6    = -4.3890942374917e-05

[evertrol]

I have just tested with a Conda environment with Python 3.6.10 and AstroPy 2.0.16, with no failures.
Could you check your other dependencies?
For me, the important ones are:

python == 3.6.10
astropy == 2.0.16
numpy == 1.18.1
scipy = 1.4.1

It could also be something to do with the BLAS or ATLAS version, which scipy will probably use under the hood when running minimize.

[kirxkirx]

Thanks for the quick reply (and for the very useful script, of course)!
The versions of the related packages that I had the problem with are

python == 3.6.9
astropy == 2.0.16
numpy == 1.16.5
scipy == 1.1.0

I've upgraded to astropy == 3.2.3 and can confirm that there is no segmentation fault happening with this version of astropy. So, I guess this was an incompatibility of the script with something related to a particular version of astropy that is now outdated (but that does not explain why the error could not be reproduced on your system)? Anyhow, I guess the issue can be closed as I cannot reproduce the problem with a newer astropy.

[evertrol]

Glad it's solved.

I just tried a Conda environment with your package versions, and still can't reproduce the problem, so it would be something specific to your system. Why an AstroPy upgrade would solve that, I have no idea.