calibrate_photometry

[mcoughlin]

Perhaps you should have an early stopping criteria in the calibrate_photometry?

33 objects pass initial quality cuts
Iteration 0 : 33 / 35 - rms 0.45 0.45 - normed 13.73 2.62 - scale 4.02 4.02 - rms 0.02
Iteration 1 : 29 / 35 - rms 0.35 0.36 - normed 39.99 4.43 - scale 1.53 1.53 - rms 0.02
Iteration 2 : 19 / 35 - rms 0.30 0.17 - normed 211.11 3.47 - scale 1.00 1.00 - rms 0.02
Iteration 3 : 17 / 35 - rms 0.29 0.07 - normed 13.57 21.22 - scale 0.63 0.63 - rms 0.00
Iteration 4 : 14 / 35 - rms 0.30 0.08 - normed 0.86 1.55 - scale 0.55 0.55 - rms 0.00
Iteration 5 : 13 / 35 - rms 0.29 0.04 - normed 0.37 1.63 - scale 0.23 0.23 - rms 0.00
Iteration 6 : 12 / 35 - rms 0.29 0.02 - normed 0.20 347.96 - scale 0.00 0.00 - rms 0.00
/Users/mcoughlin/opt/anaconda3/envs/skyportal/lib/python3.9/site-packages/statsmodels/regression/linear_model.py:1671: RuntimeWarning: divide by zero encountered in double_scalars
/Users/mcoughlin/opt/anaconda3/envs/skyportal/lib/python3.9/site-packages/statsmodels/regression/_tools.py:121: RuntimeWarning: divide by zero encountered in double_scalars
Future exception was never retrieved
future: <Future finished exception=ZeroDivisionError('float division by zero')>
Traceback (most recent call last):
File "/Users/mcoughlin/opt/anaconda3/envs/skyportal/lib/python3.9/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/Users/mcoughlin/Code/ZTF/skyportal-branches/kao/skyportal/skyportal/handlers/api/internal/image_analysis.py", line 710, in
lambda: reduce_image(
File "/Users/mcoughlin/Code/ZTF/skyportal-branches/kao/skyportal/skyportal/handlers/api/internal/image_analysis.py", line 348, in reduce_image
m = pipeline.calibrate_photometry(
File "/Users/mcoughlin/Code/ZTF/stdpipe-branches/astropy_columns/stdpipe/stdpipe/pipeline.py", line 324, in calibrate_photometry
m = photometry.match(obj['ra'], obj['dec'], obj[obj_col_mag],
File "/Users/mcoughlin/Code/ZTF/stdpipe-branches/astropy_columns/stdpipe/stdpipe/photometry.py", line 618, in match
C = sm.RLM(zero[idx]/total_err[idx], (X[idx].T/total_err[idx]).T).fit()
File "/Users/mcoughlin/opt/anaconda3/envs/skyportal/lib/python3.9/site-packages/statsmodels/robust/robust_linear_model.py", line 301, in fit
results = RLMResults(self, wls_results.params,
File "/Users/mcoughlin/opt/anaconda3/envs/skyportal/lib/python3.9/site-packages/statsmodels/robust/robust_linear_model.py", line 413, in init
self.cov_params_default = self.bcov_scaled
File "pandas/_libs/properties.pyx", line 36, in pandas._libs.properties.CachedProperty.get
File "/Users/mcoughlin/opt/anaconda3/envs/skyportal/lib/python3.9/site-packages/statsmodels/robust/robust_linear_model.py", line 450, in bcov_scaled
return k ** 2 * (1 / self.df_resid * ss_psi * self.scale ** 2) /
ZeroDivisionError: float division by zero

[karpov-sv]

Should be "fixed" in 1be4be1 - the matching now should gracefully fail if it does not have enough data points for performing the fit at some iteration.

[mcoughlin]

@karpov-sv Confirmed!