convert_msv2_to_processing_set SPECTRUM AttributeError
Closed this issue · 8 comments
Version: xradio-0.0.33
convert_msv2_to_processing_set. gets an AttributeError when trying to convert several MSv2s to zarr processing sets. I used the default partition_scheme. Only 3 of 9 test datasets were successfully converted with this version of xradio (previous conversions worked). One example is a casaguides msv2 downloaded from http://casa.nrao.edu/Data/EVLA/SNRG55/SNR_G55_10s.tar.gz.
File "/home/groot/anaconda3/envs/gui_env/lib/python3.11/site-packages/xradio/vis/convert_msv2_to_processing_set.py", line 100, in convert_msv2_to_processing_set
convert_and_write_partition(
File "/home/groot/anaconda3/envs/gui_env/lib/python3.11/site-packages/xradio/vis/_vis_utils/_ms/conversion.py", line 763, in convert_and_write_partition
np.ones(xds.SPECTRUM.shape, dtype=np.float64),
^^^^^^^^^^^^
File "/home/groot/anaconda3/envs/gui_env/lib/python3.11/site-packages/xarray/core/common.py", line 286, in getattr
raise AttributeError(
AttributeError: 'Dataset' object has no attribute 'SPECTRUM'
@pford The issue is that SNR_G55_10s.tar.gz has a WEIGHT_SPECTRUM column but it is invalid. I have added a fix and stakeholder test based on SNR_G55_10s.ms. Can you please check if branch 210-convert_msv2_to_processing_set-spectrum-attributeerror works for your datasets?
File "/home/groot/casa/github/casagui/casagui/io/_vis_io.py", line 41, in get_processing_set
convert_msv2_to_processing_set(
File "/home/groot/anaconda3/envs/gui_env/lib/python3.11/site-packages/xradio/vis/convert_msv2_to_processing_set.py", line 100, in convert_msv2_to_processing_set
convert_and_write_partition(
File "/home/groot/anaconda3/envs/gui_env/lib/python3.11/site-packages/xradio/vis/_vis_utils/_ms/conversion.py", line 773, in convert_and_write_partition
create_data_variables(
File "/home/groot/anaconda3/envs/gui_env/lib/python3.11/site-packages/xradio/vis/_vis_utils/_ms/conversion.py", line 596, in create_data_variables
xds = get_weight(
^^^^^^^^^^^
File "/home/groot/anaconda3/envs/gui_env/lib/python3.11/site-packages/xradio/vis/_vis_utils/_ms/conversion.py", line 624, in get_weight
create_attribute_metadata(col, main_column_descriptions)
^^^^^^^^^^^^^^^^^^^^^^^^
NameError: name 'main_column_descriptions' is not defined
Is this using SNR_G55_10s and what parameters are you giving the convert function?
Using SNR_G55_10s.ms and setting in_file and out_file parameters only. main_column_descriptions is not passed in or set in get_weight.
Full traceback:
Traceback (most recent call last):
File "/home/groot/anaconda3/envs/gui_env/lib/python3.11/site-packages/xradio/vis/_vis_utils/_ms/conversion.py", line 572, in create_data_variables
read_col_conversion(
File "/home/groot/anaconda3/envs/gui_env/lib/python3.11/site-packages/xradio/vis/_vis_utils/_ms/_tables/read.py", line 1243, in read_col_conversion
col_dtype = np.array(tb_tool.col(col)[0]).dtype
~~~~~~~~~~~~~~~~^^^
File "/home/groot/anaconda3/envs/gui_env/lib/python3.11/site-packages/casacore/tables/tablecolumn.py", line 270, in getitem
return self.getcell(sei[0])
^^^^^^^^^^^^^^^^^^^^
File "/home/groot/anaconda3/envs/gui_env/lib/python3.11/site-packages/casacore/tables/tablecolumn.py", line 127, in getcell
return self._table.getcell(self._column, rownr)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/groot/anaconda3/envs/gui_env/lib/python3.11/site-packages/casacore/tables/table.py", line 959, in getcell
return self._getcell(columnname, rownr)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Table DataManager error: Invalid operation: TSM: no array in row 0 of column WEIGHT_SPECTRUM in /home/groot/tarballs/SNR_G55_10s.ms/table.f19
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "", line 1, in
File "/home/groot/casa/github/casagui/casagui/plots/_vis_raster.py", line 42, in init
self._ps, self._vis_path = get_processing_set(vis)
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/groot/casa/github/casagui/casagui/io/_vis_io.py", line 41, in get_processing_set
convert_msv2_to_processing_set(
File "/home/groot/anaconda3/envs/gui_env/lib/python3.11/site-packages/xradio/vis/convert_msv2_to_processing_set.py", line 100, in convert_msv2_to_processing_set
convert_and_write_partition(
File "/home/groot/anaconda3/envs/gui_env/lib/python3.11/site-packages/xradio/vis/_vis_utils/_ms/conversion.py", line 773, in convert_and_write_partition
create_data_variables(
File "/home/groot/anaconda3/envs/gui_env/lib/python3.11/site-packages/xradio/vis/_vis_utils/_ms/conversion.py", line 596, in create_data_variables
xds = get_weight(
^^^^^^^^^^^
File "/home/groot/anaconda3/envs/gui_env/lib/python3.11/site-packages/xradio/vis/_vis_utils/_ms/conversion.py", line 624, in get_weight
create_attribute_metadata(col, main_column_descriptions)
^^^^^^^^^^^^^^^^^^^^^^^^
NameError: name 'main_column_descriptions' is not defined
It is strange that this error is not occurring for the split version of SNR_G55_10s.ms used in the tests. I have pushed a fix.
I was able to convert all of my MS datasets. Thanks @Jan-Willem!
Thank you for your patience and checking.