Cannot reshape array of size ******......After calculating the FDF anf RMSF

[arpan-52]

I am running the rmsynth3d on a Q, U, I spectral cube with 4 frequency planes starting at 950 MHz to 1200 MHz. After slicing the images, all the time I am getting the same error.

#######

ustre/aoc/observers/nm-13972/anaconda3/envs/38data/lib/python3.8/site-packages/RMutils/util_RM.py:169: RuntimeWarning: invalid value encountered in multiply
pCube = (dataQ + 1j * dataU) * weightArr[:, np.newaxis]
Reading pl_4_8_I_E.fits ...
done.
Dimensions of the input cube are: NAXIS1 = 2000 NAXIS2 = 2001 NAXIS3 = 5 NAXIS4 = 1
Dimensions of the input array are: (5, 2001, 2000)
Reading pl_4_8_Q_E.fits ...
done.
Dimensions of the input cube are: NAXIS1 = 2000 NAXIS2 = 2001 NAXIS3 = 5 NAXIS4 = 1
Dimensions of the input array are: (5, 2001, 2000)
Reading pl_4_8_U_E.fits ...
done.
Dimensions of the input cube are: NAXIS1 = 2000 NAXIS2 = 2001 NAXIS3 = 5 NAXIS4 = 1
Dimensions of the input array are: (5, 2001, 2000)
PhiArr = -50.00 to 50.00 by 0.10 (1001 chans).
Weight type is 'uniform'.
Calculating 1D RMSF and replicating along X & Y axes.

RM-synthesis completed in 89.33 seconds.
Saving the dirty FDF, RMSF and ancillary FITS files.
Traceback (most recent call last):
File "/lustre/aoc/observers/nm-13972/anaconda3/envs/38data/bin/rmsynth3d", line 8, in
sys.exit(main())
File "/lustre/aoc/observers/nm-13972/anaconda3/envs/38data/lib/python3.8/site-packages/RMtools_3D/do_RMsynth_3D.py", line 965, in main
writefits(
File "/lustre/aoc/observers/nm-13972/anaconda3/envs/38data/lib/python3.8/site-packages/RMtools_3D/do_RMsynth_3D.py", line 423, in writefits
FDFcube = np.reshape(FDFcube, [FDFcube.shape[0]] + output_axes)
File "<array_function internals>", line 200, in reshape
File "/lustre/aoc/observers/nm-13972/anaconda3/envs/38data/lib/python3.8/site-packages/numpy/core/fromnumeric.py", line 298, in reshape
return _wrapfunc(a, 'reshape', newshape, order=order)
File "/lustre/aoc/observers/nm-13972/anaconda3/envs/38data/lib/python3.8/site-packages/numpy/core/fromnumeric.py", line 57, in _wrapfunc
return bound(*args, **kwds)
ValueError: cannot reshape array of size 4006002000 into shape (1001,5,2001,2000)

#######

[Cameron-Van-Eck]

Hi Arpan:

I can try to take a look at this early next week. Could you post one of the FITS headers, since that can help with reproducing the bug?

Thanks,
Cameron

[arpan-52]

Hi Cameron,

Let me share all the files with you. Thanks for the help.

f.txt
I.txt
Q.txt
U.txt

Thanks:)
Arpan

[Cameron-Van-Eck]

Hi Arpan:

The FITS headers were exactly the clue I needed: they're missing critical keywords that RM-Tools uses to determine the dimensionality of the data. Specifically, there's no CRTYPE3 = FREQ key, which would tell RM-Tools to use the 3rd axis as the frequency axis (it's interesting that this problem doesn't emerge until the write step -- I would have expected it to cause problems at the read step).

I can think of two solutions:

1. Add that key to the FITS headers (strictly speaking it only needs to be in the Stokes Q header, I think).
2. Remove the degenerate 4th axis from the header (set NAXIS = 3 and remove NAXIS4 = 1). The write function defaults to using the last axis as the frequency axis, so it should work then.

Cheers,
Cameron

P.S.- I think you're sampling far to densely in Faraday depth, and not far enough out, given your frequencies. I'd recommend Phi_max ~= 1000, and dPhi ~ 10 rad/m2, to start with. With only 5 channels, you're going to have significant sidelobe structure in the RMSF.