run_derive_calib: poylfit poorly conditioned warnings
Opened this issue · 1 comments
bmiszalski commented
When running run_derive_calib on the test dataset classical_20230922 I get these warnings:
/usr/local/lib/python3.11/dist-packages/pywifes/wifes_calib.py:574: RankWarning: Polyfit may be poorly conditioned
temp_best_calib = numpy.polyfit(temp_full_x, temp_full_y, polydeg)
/usr/local/lib/python3.11/dist-packages/pywifes/wifes_calib.py:604: RankWarning: Polyfit may be poorly conditioned
best_calib = numpy.polyfit(full_x, full_y, polydeg)
/usr/local/lib/python3.11/dist-packages/pywifes/wifes_calib.py:607: RankWarning: Polyfit may be poorly conditioned
best_calib = numpy.polyfit(full_x, full_y, polydeg)
It may be worth investigating these further.
Full output:
Deriving sensitivity function
cube_fn_list =['/code/WiFeS/classical_20230922/reduc_blue/OBK-124128-WiFeS-Blue-UT20230922T090650-1.p08.fits']
Found star HD200654
i_mid 1987 <class 'int'>
/usr/local/lib/python3.11/dist-packages/pywifes/wifes_calib.py:574: RankWarning: Polyfit may be poorly conditioned
temp_best_calib = numpy.polyfit(temp_full_x, temp_full_y, polydeg)
/usr/local/lib/python3.11/dist-packages/pywifes/wifes_calib.py:604: RankWarning: Polyfit may be poorly conditioned
best_calib = numpy.polyfit(full_x, full_y, polydeg)
/usr/local/lib/python3.11/dist-packages/pywifes/wifes_calib.py:607: RankWarning: Polyfit may be poorly conditioned
best_calib = numpy.polyfit(full_x, full_y, polydeg)
best_calib
poly
None
[-5.24319378e-87 9.27170120e-83 -2.87821729e-79 -2.07717069e-75
9.83936597e-73 5.56474206e-68 2.46468957e-64 -1.38920254e-61
-7.50710660e-57 -4.19689941e-53 -5.38499094e-50 8.39401439e-46
6.92422748e-42 2.04175926e-38 -6.94012109e-35 -1.02223388e-30
-4.17392776e-27 5.54172444e-24 1.52346626e-19 6.02521323e-16
-1.84368992e-12 -2.46812350e-08 -2.21053038e-06 7.87764729e-01
-2.62218764e+03 2.60966260e+06]
run_derive_calib: All done in 0.0 seconds.
CMartinezLombilla commented
We've been exploring this issue and it's likely due to the high-degree polynomial used in the fitting (polydeg=30
) in derive_wifes_calibration()
. Fitting a polynomial of such a degree will likely lead to numerical instability and overfitting.