--data-rebin-time: corrected data comes out looking Very Wrong
Closed this issue · 5 comments
Running a D (leakage) solution with --data-rebin-time 8 (because the solint is infinite in time, and there's just a bit too much data to load raw), I get perfectly sensible-looking solutions and chi-squares, but a horrible corrected data:
Rerunning the same solutions with apply-only (and no rebinning), things look fine:
Since in rebin mode the output data is meant to be simply broadcast out over the rebin, this is a sign of some arithmetic bug...
Run is at /net/simon/projects/OldDevils/cc-polcal/solve-D.cc-out.5.
Oh, and when I set --weight-column None, I get cosmologically large chi-sq values. This MS just keeps on giving (bugs).
The latter is due to #417.
The former seems to be fixed by the https://github.com/ratt-ru/CubiCal/tree/mk-polcal debugging spree.
Nope, false dawn! Still getting crap in the output. Suspect it's the re-emergence of casacore/python-casacore#130.
The scenario only arises with the D-solve parset. My theory is that the difference is, in this scenario we have one very large tile, due to timeint being infinite, and there being a fair chunk of 1939 data in the MS. (By contrast, the X-solve data is one small scan on 3C286. And when I apply the D and X solutions, I use smaller chunks.)
When I check in the debugger, the data & flags in memory look consistent (I check for high unflagged amplitudes, i.e. (data>100)&~flag, this returns nothing).
However shadems still plots crap as per #415 (comment), and trying to look at the data in the table with python-casacore, I run into casacore/python-casacore#130. So I don't trust the reading or the writing at this point.
Yes, the above guess is correct! With the fix for #424 in place, corrected data post-solve looks perfectly fine:
Note that this is better than the second plot in #415 (comment). So the leakage solutions are better, no crazy channels. Probably makes sense: casacore/python-casacore#130 would have affected the input data being read too, so the previous solutions would have been off.
How has radio astronomy ever worked? 😱
With the casacore bug worked around, the leakage solutions are a thing of beauty:
| before | after |
|---|---|
 |
 |