[Bug]: `UnboundLocalError: local variable 'params_results' referenced before assignment`

[tomvothecoder]

What happened?

run_diags() now sets the params_results variable and uses the results_dir attribute to move the log file to.

However, if diagnostic run(s) fail, params_results will not be set resulting in the UnboundLocalError: local variable 'params_results' referenced before assignment

Related code:

e3sm_diags/e3sm_diags/run.py

Lines 48 to 94 in 9e14ff8

	def run_diags(
	self, parameters: List[CoreParameter], use_cfg: bool = True
	) -> List[CoreParameter]:
	"""Run a set of diagnostics with a list of parameters.
	Parameters
	----------
	parameters : List[CoreParameter]
	A list of parameters defined through the Python API.
	use_cfg : bool, optional
	Also run diagnostics using a `.cfg` file, by default True.
	* If True, run all sets using the list of parameters passed in
	this function and parameters defined in a .cfg file (if
	defined), or use the .cfg file(s) for default diagnostics. This
	is the default option.
	* If False, only the parameters passed via ``parameters`` will be
	run. The sets to run are based on the sets defined by the
	parameters. This makes it easy to debug a few sets instead of
	all of the debug sets too.
	Returns
	-------
	List[CoreParameter]
	A list of parameter objects with their results.
	Raises
	------
	RuntimeError
	If a diagnostic run using a parameter fails for any reason.
	"""
	params = self.get_run_parameters(parameters, use_cfg)
	if params is None or len(params) == 0:
	raise RuntimeError(
	"No parameters we able to be extracted. Please "
	"check the parameters you defined."
	)
	try:
	params_results = main(params)
	except Exception:
	logger.exception("Error traceback:", exc_info=True)
	move_log_to_prov_dir(params_results[0].results_dir)
	return params_results

This error is technically not a bug because it indicates diagnostic sets failed. However, it is unintended behavior because e3sm_diags still completes its execution (the except statement) even with failed runs.

What did you expect to happen? Are there are possible answers you came across?

e3sm_diags should complete its execution even with failed runs

Minimal Complete Verifiable Example (MVCE)

import os
import numpy
from e3sm_diags.parameter.core_parameter import CoreParameter
from e3sm_diags.parameter.qbo_parameter import QboParameter

from e3sm_diags.run import runner

short_name = "v2.LR.historical_0201"
test_ts = "ts"
start_yr = int("1850")
end_yr = int("1851")
num_years = end_yr - start_yr + 1
ref_start_yr = 1985

param = CoreParameter()

# Model
param.test_name = "v2.LR.historical_0201"
param.short_test_name = short_name

# Output dir
param.results_dir = "model_vs_obs_1850-1851"

# Additional settings
param.run_type = "model_vs_obs"
param.diff_title = "Model - Observations"
param.output_format = ["png"]
param.output_format_subplot = []
param.multiprocessing = False
param.num_workers = 24
# param.fail_on_incomplete = True
params = [param]
qbo_param = QboParameter()
qbo_param.test_data_path = test_ts
qbo_param.test_name = short_name
qbo_param.test_start_yr = start_yr
qbo_param.test_end_yr = end_yr
qbo_param.ref_start_yr = ref_start_yr
ref_end_yr = ref_start_yr + num_years - 1
if ref_end_yr <= 2014:
    qbo_param.ref_end_yr = ref_end_yr
else:
    qbo_param.ref_end_yr = 2014

# Obs
qbo_param.reference_data_path = (
    "/lcrc/group/e3sm/diagnostics/observations/Atm/time-series/"
)

params.append(qbo_param)

# Run
runner.sets_to_run = ["qbo"]
runner.run_diags(params)

Relevant log output

===== RUN E3SM DIAGS =====

2023-12-21 14:25:12,848 [ERROR]: run.py(run_diags:90) >> Error traceback:
Traceback (most recent call last):
  File "/home/ac.forsyth2/miniconda3/envs/e3sm_diags_20231221/lib/python3.10/site-packages/e3sm_d\
iags/run.py", line 88, in run_diags
    params_results = main(params)
  File "/home/ac.forsyth2/miniconda3/envs/e3sm_diags_20231221/lib/python3.10/site-packages/e3sm_d\
iags/e3sm_diags_driver.py", line 363, in main
    os.makedirs(parameters[0].results_dir, 0o755)
  File "/home/ac.forsyth2/miniconda3/envs/e3sm_diags_20231221/lib/python3.10/os.py", line 225, in\
 makedirs
    mkdir(name, mode)
FileNotFoundError: [Errno 2] No such file or directory: ''
Traceback (most recent call last):
  File "/lcrc/group/e3sm/ac.forsyth2/zppy_test_complete_run_output/unified_1.9.2rc2/v2.LR.histori\
cal_0201/post/scripts/tmp.447022.wCBq/e3sm.py", line 53, in <module>
    runner.run_diags(params)
  File "/home/ac.forsyth2/miniconda3/envs/e3sm_diags_20231221/lib/python3.10/site-packages/e3sm_d\
iags/run.py", line 92, in run_diags
    move_log_to_prov_dir(params_results[0].results_dir)
UnboundLocalError: local variable 'params_results' referenced before assignment
[WARNING] yaksa: 10 leaked handle pool objects
srun: error: chr-0496: task 0: Exited with exit code 1

Anything else we need to know?

Related to E3SM-Project/zppy#538

Environment

main branch