[Refactor]: CDAT Migration - Add test run scripts and results
Closed this issue · 2 comments
tomvothecoder commented
Is your feature request related to a problem?
We need a standard structure and process to streamline regression testing. We will be using perlmutter as our test machine.
1. Create a shared directory that stores results of test scripts for main and branches (@tomvothecoder) -- DONE
- Root directory:
/global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/ - Example branch results directory:
/global/cfs/projectsdirs/e3sm/e3sm_diags_cdat_test/655-zonal-mean-2d/branch
2. Run run_v2_9_0_all_sets_E3SM_machines.py on main to generate baseline results for testing (@tomvothecoder) -- DONE
-
Directory:
/global/cfs/projectdirs/e3sm/e3sm_diags_cdat_test/main -
Add template test script to cdat-migration-fy24 with configurable
results_dir
Describe the solution you'd like
No response
Describe alternatives you've considered
No response
Additional context
No response
tomvothecoder commented
If the second step isn't what we need:
3. (OPTIONAL) Set up test scripts for each set (@chengzhuzhang)
Steps:
- Open a PR with these scripts to merge to
cdat-migration-fy24- Need to store intermediary files and metrics files for comparison
- Store scripts under
auxiliary_tools/cdat_regression_testing/run_scripts - Think about test script methodology:
- Best to test all possible variables, maybe with 1 season, regridding is variable dependent
- Or select a subset of variables to speed up testing process
Needed for:
- polar
- cosp_histogram
- zonal_mean_xy
- zonal_mean_2d and zonal_mean_2d_stratosphere
- area_mean_time_series
- annual_cycle_zonal_mean
- meridional_mean_2d
- enso_diags
- qbo
- streamflow
- diurnal_cycle
- arm_diags
- tc_analysis
- aerosol_aeronet
- aerosol_budget
Run test scripts using main branch (TBD)
- Store the results in the shared directory from 1.
- NOTE: we need to address #759 to have results more closely aligned with Xarray code
- Follow https://github.com/E3SM-Project/e3sm_diags/wiki/CDAT-Migration-FY24-%E2%80%90-General-Guide
tomvothecoder commented
I just added the template run script to cdat-migration-fy24 (related commit). This issue is now complete.