catalog-builder broken after addition of grid.zarr variables in AWS
bonnland opened this issue · 3 comments
bonnland commented
The catalog builder is crashing when it tries to parse the recently added grid variables on AWS. The folder structure is not expected:
['ncar-cesm-lens', 'lnd', 'monthly', 'cesmLE-RCP85-FSNO.zarr']
['ncar-cesm-lens', 'lnd', 'monthly', 'cesmLE-RCP85-H2OSNO.zarr']
['ncar-cesm-lens', 'lnd', 'monthly', 'cesmLE-RCP85-QRUNOFF.zarr']
['ncar-cesm-lens', 'lnd', 'monthly', 'cesmLE-RCP85-RAIN.zarr']
['ncar-cesm-lens', 'lnd', 'monthly', 'cesmLE-RCP85-SNOW.zarr']
['ncar-cesm-lens', 'lnd', 'monthly', 'cesmLE-RCP85-SOILLIQ.zarr']
['ncar-cesm-lens', 'lnd', 'monthly', 'cesmLE-RCP85-SOILWATER_10CM.zarr']
['ncar-cesm-lens', 'ocn', 'grid.zarr', '.zattrs']
Traceback (most recent call last):
File "catalog-builder.py", line 42, in <module>
build_catalog()
File "catalog-builder.py", line 27, in build_catalog
_, experiment, variable = path_components[-1].split('.')[0].split('-')
ValueError: not enough values to unpack (expected 3, got 1)
bonnland commented
@andersy005 Does the grid.zarr folder need to go in ocn/monthly? Or is it intended to cover a broader group of datasets than this?
andersy005 commented
Since the grid variables don't depend on time, it makes sense to just put the grid.zarr
store under ncar-cesm-lens/ocn/static