error message 'ValueError: not enough values to unpack (ecpected 4, got 3)' in rmclean3d
cainrinkMac opened this issue · 7 comments
Hello,
I ran rmsynth3d on my data (J1227 FITS) with no issues. Running rmclean3d on the dirty FITS produced an error which traced back to a ValueError in RMtools_3D. I haven't yet found a solution to this issue. I've run the tools on a very similar dataset (3c286 FITS), with no problems.
williamslab@macid32377:~$ rmsynth3d /mnt/data/Research/CASApipeline/J1227RM/J1227.cube0.CTq.fits /mnt/data/Research/CASApipeline/J1227RM/J1227.cube0.CTu.fits /mnt/data/Research/CASApipeline/J1227RM/J1227.cube0.CT.freqfile -v
Reading /mnt/data/Research/CASApipeline/J1227RM/J1227.cube0.CTq.fits ...
done.
Dimensions of the input cube are: NAXIS1 = 100 NAXIS2 = 100 NAXIS3 = 87 NAXIS4 = 1
Dimensions of the input array are: (87, 100, 100)
Reading /mnt/data/Research/CASApipeline/J1227RM/J1227.cube0.CTu.fits ...
done.
Dimensions of the input cube are: NAXIS1 = 100 NAXIS2 = 100 NAXIS3 = 87 NAXIS4 = 1
Dimensions of the input array are: (87, 100, 100)
PhiArr = -1213.14 to 1213.14 by 24.26 (101 chans).
Weight type is 'uniform'.
Running RM-synthesis by channel.
[========================================] 100%
Calculating 1D RMSF and replicating along X & Y axes.
> RM-synthesis completed in 0.39 seconds.
Saving the dirty FDF, RMSF and ancillary FITS files.
> /mnt/data/Research/CASApipeline/J1227RM/FDF_real_dirty.fits
WARNING: VerifyWarning: Card is too long, comment will be truncated. [astropy.io.fits.card]
> /mnt/data/Research/CASApipeline/J1227RM/FDF_im_dirty.fits
> /mnt/data/Research/CASApipeline/J1227RM/FDF_tot_dirty.fits
> /mnt/data/Research/CASApipeline/J1227RM/RMSF_real.fits
> /mnt/data/Research/CASApipeline/J1227RM/RMSF_im.fits
> /mnt/data/Research/CASApipeline/J1227RM/RMSF_tot.fits
> /mnt/data/Research/CASApipeline/J1227RM/RMSF_FWHM.fits
> /mnt/data/Research/CASApipeline/J1227RM/FDF_maxPI.fits
> /mnt/data/Research/CASApipeline/J1227RM/FDF_peakRM.fits
williamslab@macid32377:~$ rmclean3d /mnt/data/Research/CASApipeline/J1227RM/FDF_tot_dirty.fits /mnt/data/Research/CASApipeline/J1227RM/RMSF_tot.fits -v
Err: 'phi2Arr_radm2' and 'dirtyFDF' are not the same length.
Traceback (most recent call last):
File "/home/williamslab/.local/bin/rmclean3d", line 8, in <module>
sys.exit(main())
File "/home/williamslab/.local/lib/python3.10/site-packages/RMtools_3D/do_RMclean_3D.py", line 424, in main
cleanFDF, ccArr, iterCountArr, residFDF, headtemp = run_rmclean(fitsFDF = args.fitsFDF[0],
File "/home/williamslab/.local/lib/python3.10/site-packages/RMtools_3D/do_RMclean_3D.py", line 111, in run_rmclean
cleanFDF, ccArr, iterCountArr, residFDF = \
ValueError: not enough values to unpack (expected 4, got 3)
williamslab@macid32377:~$
Hi!
Thanks very much for including the data, that's made it much easier to investigate. Unfortunately, I can't reproduce the bug -- I ran the same data and same parameters, and it ran to completion. But I do understand why the ValueError is raised; that's not a desired outcome and I will fix that, but it's ultimately a distraction. Also there's a mistype in the Err: ... line that I'll fix. But neither gets to the actual problem.
The root problem seems to be that it seems to have created a mis-sized phiArr (array of Faraday depths for all slices in the FDF). I have no idea how that could happen (since it should be generated directly from the FITS header), and that particular bit of code hasn't changed in a long time.
Could you send me the FDF and RMSF outputs that you got from rmsynth3d, so I can compare them to mine? You could also try updating to the newest version of RM-Tools (that's always safe advice, right?) if you haven't already, but since the part of the code relevant to this bug hasn't changed recently so I don't expect that to make a difference.
The only other idea I have right now is to completely delete the rmsynth3d outputs and regenerate them (in case something went weird while making them), if you haven't tried that yet.
Cheers,
Cameron
Thanks for replying! I'll attach the FDF and RMSF outputs here.
I should be on the most recent version of RM-Tools, I made sure to install the newest one when I started the project. I've tried deleting and remaking the rmsynth3d files, but that didn't work. I've also tried remaking the original inputs and starting there, but that also didn't fix it.
Because you weren't able to replicate the error, it could be an issue with my computer (no idea what that issue would be though), as its had a few really random bugs recently.
Thanks for your help!
Cain
P.S: accidentally clicked the wrong comment button, and unintentionally closed issue.
Hi Cain:
Yeah, I'm completely stumped -- the FDF is identical to mine within machine precision.
We have a sample size of 2 -- one seeing the bug and one not, so let's get a tie breaker to see who's machine is weird (yours or mine). @AlecThomson, do you have a few minutes to try running it yourself to see if you can reproduce the error?
Cheers,
Cameron
Hello,
I'm stumped also. I got the error again today after restarting my computer, so at least its consistent. Some good news is that RM-Tools is working smoothly on my other fields, the error seems isolated to this data. A third party would be great, good idea.
Thanks,
Cain
Hi both, I will grab the data and give this a go.
@cainrinkMac - just to sanity check how did you go about obtaining the latest version of RM-Tools?
Ok, I also cannot reproduce the error. I suspect that is because the @cainrinkMac's version is out of date.
To be completely sure I would do something like:
pip install git+https://github.com/CIRADA-Tools/RM-Tools --upgrade --force-reinstall
or
pip uninstall RM-Tools && pip install git+https://github.com/CIRADA-Tools/RM-Tools
Quick update, I had a conda env with an older version and I did reproduce the error. Updating to the latest version fixes the issue.