The processing set partition can not be written back to disk using `xarray.dataset.to_zarr`
Opened this issue · 1 comments
When processing set is read using read_processing_set, we get xarray Datasets corresponding to each partition.
Each dataset contains the data from the MAIN zarr group. The dataset may also contain data from ANTENNA and POINTING groups, stored as attributes.
In all our test datasets, both antenna_xds and pointing_xds were present as attributes.
Xarray's Dataset has a method to_zarr() which can be used to store the dataset in a zarr file format.
Calling to_zarr() on any of the read datasets throws this error
TypeError: Invalid attribute in Dataset.attrs.
To explore this issue further, using the xarray.open_zarr(), we read only the MAIN zarr group inside a partition of the processsing set, which also returned an xarray Dataset object.
When we called dataset.to_zarr() on this MAIN dataset, it was stored in a zarr file successfully.