test albedofb with ESM4 output failed
Wen-hao-Dong opened this issue · 4 comments
Bug Severity
- 1 = Minor problem that does not affect total framework functionality (e.g., computation error in a POD, problem with logging output, or an issue on a single system
- 2 = Major problem that affects overall functionality, but that does not occur for all users (e.g., problems installing the framework with a specific Conda version, a framework option that causes one or more PODs to fail, or missing/incompatible Python modules).
- 3 = Catastrophic problem that occurs frequently for multiple users and/or on multiple systems (e.g.,framework consistently fails to install on multiple systems, or one or more PODs continuously fails after running successfully)
Describe the bug
When testing this POD with ESM4 output, data request for <#Ohgu:albedofb.areacella> failed since area info is stored in atmos_cmip.static.nc with the name called area. This is different from areacella set in fieldlist_CMIP.jsonc.
While data request for <#OhjX:albedofb.tas> <#Ohss:albedofb.rsdt> <#Ohzr:albedofb.rsds> <#OhFU:albedofb.rsut> <#OhP8:albedofb.rsus> failed due to "Too many iterations in select_data() for <#NBkk:ESM4_historical_D1>."
Steps To Reproduce
I am using the updated version and have submitted the job using the following command: sbatch ./mdtf_slurm -f sites/NOAA_GFDL/default_gfdl.jsonc --site NOAA_GFDL -v
Environment
Describe the system environment:
- OS: [e.g. RHEL 7.2]
- Conda version: [the
conda infocommand will provide the installation information] - branch name and link: [e.g., test0716, https://github.com/Wen-hao-Dong/MDTF-diagnostics/tree/test0716]
Log information and/or terminal output
/archive/wnd/MDTF-GFDL/STDO/slurm_MDTF_diags.24885969_4294967294.out
There is probably an issue with the standard_name or similar in the area data file you are using.
If you provide the path to your test repo, I can try replicating the issue and testing the fix on my end--probably requires another cell_area entry to the CMIP fieldlist, and possibly another modifier.
Note that the variable area does not appear in the variable list when I search the llnl cmip6 database, and that it seems like it is best practice to use areacello or areacella when defining grid cell area for ocean and atmospheric grids, respectively (I know this is not in your control, but is something to pass on the people in charge of defining the output data).
Here is the path to my test repo: /archive/wnd/MDTF-GFDL/mdtf/MDTF-diagnostics
The branch name and link are: test0716, https://github.com/Wen-hao-Dong/MDTF-diagnostics/tree/test0716
Yes, the name area in not in the CMIP variable list. It is always saved in the static file for the GFDL model output. I will take your suggestion for the subsequent frepp test.
@Wen-hao-Dong I have a workaround on my fork in a test branch where I've added pod_env_vars to set a cell area file and variable name that will be subbed for the areacella variable (now optional) if it is not present. This requires the user to copy the desired static file to a directory with read access from PPAN before running the package.
However, there are issues with the input data types in the ESM4 dataset being xrarray DataArrays, and the coord operations on rsus and rsds fail in the albedo_calcs.py module.
I have some test lines in albedofb_calcs.py to try and debug the problem, but am going to set this aside since this is a POD/data-related issue, not a framework problem.
Someone familiar with using xarray to process climate model data might be able to help resolve this problem faster. If you are able to find a fix before I do, please create a branch with the new code, and submit a PR to the NOAA_GFDL main branch for review.
@Wen-hao-Dong This issue has been resolved. Please pull updates from the main branch into your test branch, and follow the updated instructions in the albedofb documentation for incorporating non-CMIP-standard cell area files (e.g., atmos.static.nc) into the POD via the new pod_env_vars in the settings file.