E3SM-Project/e3sm_diags

[Bug]: Fix `_get_instance_of_param_class()` conditional resulting in param object attributes not being correctly copied

Closed this issue · 0 comments

What happened?

This commit E3SM-Project/e3sm_diags@48426aa (#755) changes if type(p) == cls_type to if isinstance(p, cls_type). Apparently, both of these conditionals are not the same (source) and can return different boolean values. I changed if type(p) == cls_type because it is considered bad practice (Flake8 E721).

As a result, in v2.10.0, the results_dir config is not being copied from the first parameter (param) to the second parameter (qbo_param). This causes results_dir to be blank which cascades to the FileNotFoundError: [Errno 2] No such file or directory: '' and FileNotFoundError: [Errno 2] No such file or directory: '/prov/e3sm_diags_run.log'.

What did you expect to happen? Are there are possible answers you came across?

The fix is to change if isinstance(p, cls_type) to if type(p) is cls_type. I will open a separate PR for this and get a new e3sm_diags RC release out.

Minimal Complete Verifiable Example (MVCE)

import os
import numpy
from e3sm_diags.parameter.core_parameter import CoreParameter
from e3sm_diags.parameter.qbo_parameter import QboParameter

from e3sm_diags.run import runner

short_name = "v2.LR.historical_0201"
test_ts = "ts"
start_yr = int("1850")
end_yr = int("1851")
num_years = end_yr - start_yr + 1
ref_start_yr = 1985

param = CoreParameter()

# Model
param.test_name = "v2.LR.historical_0201"
param.short_test_name = short_name

# Output dir
param.results_dir = "model_vs_obs_1850-1851"

# Additional settings
param.run_type = "model_vs_obs"
param.diff_title = "Model - Observations"
param.output_format = ["png"]
param.output_format_subplot = []
param.multiprocessing = False
param.num_workers = 24
# param.fail_on_incomplete = True
params = [param]
qbo_param = QboParameter()
qbo_param.test_data_path = test_ts
qbo_param.test_name = short_name
qbo_param.test_start_yr = start_yr
qbo_param.test_end_yr = end_yr
qbo_param.ref_start_yr = ref_start_yr
ref_end_yr = ref_start_yr + num_years - 1
if ref_end_yr <= 2014:
    qbo_param.ref_end_yr = ref_end_yr
else:
    qbo_param.ref_end_yr = 2014

# Obs
qbo_param.reference_data_path = (
    "/lcrc/group/e3sm/diagnostics/observations/Atm/time-series/"
)

params.append(qbo_param)

# Run
runner.sets_to_run = ["qbo"]
runner.run_diags(params)

Relevant log output

===== RUN E3SM DIAGS =====

2023-12-21 20:06:05,915 [ERROR]: run.py(run_diags:37) >> Error traceback:
Traceback (most recent call last):
  File "/lcrc/soft/climate/e3sm-unified/base/envs/e3sm_unified_1.9.2rc2_chrysalis/lib/python3.10/site-packages/e3sm_diags/run.py", line 35, in run_diags
    main(final_params)
  File "/lcrc/soft/climate/e3sm-unified/base/envs/e3sm_unified_1.9.2rc2_chrysalis/lib/python3.10/site-packages/e3sm_diags/e3sm_diags_driver.py", line 362, in main
    os.makedirs(parameters[0].results_dir, 0o755)
  File "/lcrc/soft/climate/e3sm-unified/base/envs/e3sm_unified_1.9.2rc2_chrysalis/lib/python3.10/os.py", line 225, in makedirs
    mkdir(name, mode)
FileNotFoundError: [Errno 2] No such file or directory: ''
Traceback (most recent call last):
  File "/lcrc/group/e3sm/ac.forsyth2/zppy_test_complete_run_output/unified_1.9.2rc2/v2.LR.historical_0201/post/scripts/tmp.447067.MErK/e3sm.py", line 53, in <module>
    runner.run_diags(params)
  File "/lcrc/soft/climate/e3sm-unified/base/envs/e3sm_unified_1.9.2rc2_chrysalis/lib/python3.10/site-packages/e3sm_diags/run.py", line 38, in run_diags
    move_log_to_prov_dir(final_params[0].results_dir)
  File "/lcrc/soft/climate/e3sm-unified/base/envs/e3sm_unified_1.9.2rc2_chrysalis/lib/python3.10/site-packages/e3sm_diags/logger.py", line 104, in move_log_to_prov_dir
    shutil.copy(LOG_FILENAME, provenance_dir)
  File "/lcrc/soft/climate/e3sm-unified/base/envs/e3sm_unified_1.9.2rc2_chrysalis/lib/python3.10/shutil.py", line 417, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/lcrc/soft/climate/e3sm-unified/base/envs/e3sm_unified_1.9.2rc2_chrysalis/lib/python3.10/shutil.py", line 256, in copyfile
    with open(dst, 'wb') as fdst:
FileNotFoundError: [Errno 2] No such file or directory: '/prov/e3sm_diags_run.log'
srun: error: chr-0493: task 0: Exited with exit code 1

Anything else we need to know?

Related to E3SM-Project/zppy#538

Environment

v2.10.0 and main