cannot access ds_names

[aradhakrishnanGFDL]

I am testing wheeler khiladis on am5 test data. I am pretty sure my runtime configuration is not playing nice, but need help understanding what's missing from the following error message from the MDTF.

Also why does the first line of the stdout read "from cesm"? The convention is set as CMIP in my runtime config, the data is GFDL post-processed format.
Config can be found here accessible from GFDL /home/a1r/testing/runtime_config_am5test_wheeler.jsonc

mdtf_framework.py -f ~/testing/runtime_config_am5test_wheeler.jsonc
Translating POD variables from cesm to cmip
Preprocessing data for EOF_500hPa
Querying /home/a1r/github/noaa-gfdl/catalogs/c96L65_am5f7b10r0_amip30_0814.json for variable zg500 for case atmos_cmip.
WARNING: /home/oar.gfdl.mdtf/miniconda3/envs/_MDTF_base/lib/python3.12/site-packages/intake_esm/_search.py:50: UserWarning: This pattern is interpreted as a regular expression, and has match groups. To actually get the groups, use str.extract.
  mask = df[column].str.contains(value, regex=True, case=True, flags=0)

Variable <zg500> data starts at hour 12
Variable <zg500> data ends at hour 12
Units for 'time' on var 'zg500' found in dataset; setting to 'days since 1979-01-01 00:00:00'.
CRITICAL: **********************************************************************
Uncaught exception:
Traceback (most recent call last):
  File "/home/a1r/MDTF-diagnostics/src/preprocessor.py", line 1291, in parse_ds
    ds = self.parser.parse(var, ds)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/a1r/MDTF-diagnostics/src/xr_parser.py", line 1369, in parse
    self.reconcile_variable(var, ds)
  File "/home/a1r/MDTF-diagnostics/src/xr_parser.py", line 1197, in reconcile_variable
    self.reconcile_names(tv, ds, tv.name)
  File "/home/a1r/MDTF-diagnostics/src/xr_parser.py", line 917, in reconcile_names
    if len(ds_names) == 1:
           ^^^^^^^^
UnboundLocalError: cannot access local variable 'ds_names' where it is not associated with a value

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/a1r/MDTF-diagnostics/mdtf_framework.py", line 244, in <module>
    exit_code = main(prog_name='MDTF-diagnostics')
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/oar.gfdl.mdtf/miniconda3/envs/_MDTF_base/lib/python3.12/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/oar.gfdl.mdtf/miniconda3/envs/_MDTF_base/lib/python3.12/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/oar.gfdl.mdtf/miniconda3/envs/_MDTF_base/lib/python3.12/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/oar.gfdl.mdtf/miniconda3/envs/_MDTF_base/lib/python3.12/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/oar.gfdl.mdtf/miniconda3/envs/_MDTF_base/lib/python3.12/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/a1r/MDTF-diagnostics/mdtf_framework.py", line 200, in main
    cat_subset = data_pp.process(cases, ctx.config, model_paths.MODEL_WORK_DIR)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/a1r/MDTF-diagnostics/src/preprocessor.py", line 1329, in process
    var_xr_dataset = self.parse_ds(v, case_xr_dataset)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/a1r/MDTF-diagnostics/src/preprocessor.py", line 1293, in parse_ds
    raise util.chain_exc(exc, f"parsing dataset metadata", util.DataPreprocessEvent)
  File "/home/a1r/MDTF-diagnostics/src/util/exceptions.py", line 58, in chain_exc
    raise new_exc_class(new_msg) from exc
src.util.exceptions.DataPreprocessEvent: Caught exception while parsing dataset metadata: UnboundLocalError("cannot access local variable 'ds_names' where it is not associated with a value").

[aradhakrishnanGFDL]

Note- This seems to be tied to that error message. I see different errors when I adjust the time to include MM, and those errors seem like a progress.

          "startdate": "2003",
          "enddate": "2004"

[wrongkindofdoctor]

@aradhakrishnanGFDL There's an issue with the date range function that I found with similar tests that the EOF500_hPa POD is triggering. I also need to refine how the translation handles 4D vs 3D variables when levels are requested when defining the variable_id passed to the catalog query.

The message about the translation is referring to the convention expected by the POD vs the dataset convention specified at runtime. W-K expects CESM standard data, and the AM5 dataset is CMIP, so the varlist object parsed from POD settings will be converted to CMIP before the catalog query.

[wrongkindofdoctor]

@aradhakrishnanGFDL I fixed the ds_names issue, and can run EOF500hPa with your test catalog for 1980-1985. However, the catalog contains several entries with empty strings for standard_name, including ua200, so the query will still fail for W-K.

[aradhakrishnanGFDL]

Thanks @wrongkindofdoctor! This issue can be closed.

The csv mappings may need an update for the standard names. I will check and have a separate issue or PR for that.