NCAR/container-dtc-nwp

Setup GitHub Actions to automate the testing of the Sandy case for each feature branch commit and/or pull request.

Closed this issue · 4 comments

JohnHalleyGotway commented

Automating the testing of the Sandy case via GHA will make new development, especially when updating software versions, more efficient. Consider leveraging the existing docker_run_case.ksh script for this work. Note that this should just be an initial capability and can be enhanced to either run more efficiently, diff results from previous runs, or run more than a single case.

JohnHalleyGotway commented

@JohnHalleyGotway and @michelleharrold met on 1/19/22 to brainstorm and decided the following:

  • When should the tests run?
    • Only for pull requests initially.
    • For commits to all branches that include a special keyword. Meaning, it's disabled by default.
    • Consider supporting this type of keyword: ci-run-case-{NAME}
  • What test(s) should be run?
    • For PR's, always run Sandy.
    • For commits to all branches, look for the ci-run-case-{NAME} keyword and run the specified case.
  • What code should be tested?
    • Check what's changed.
      • If components/base has changed, rebuild all images locally. METviewer and python depend on Dockerfile_simple while all others depend on Dockerfile. If the entrypoint script changes, then all should be rebuilt. For now, keep it simpler and just rebuild everything for any changes in the base directory.
      • If components/{COMPONENT_NAME} has changed (not base) then rebuild that image locally.
      • If not, pull dtcenter/{COMPONENT_NAME}:latest from DockerHub.
  • What defines a successful run?
    • Just all the commands running without returning bad status.

Could consider defining a "workflow_dispatch" test event for changes to the common community container.

Change docker_run_case.ksh to set VERSION = latest unless the version is specified via an argument on the command line.

JohnHalleyGotway commented

@fossell I made some progress on this in the feature/issue_48_gha branch over the last week, but don't have it finished yet. I will work more on it starting on 7/5.

  • 👍1
JohnHalleyGotway commented

@TatianaBurek I've been working on setting up GitHub actions for this repository.
Here's the most recent GHA run which has a little green checkmark to indicate success. However, downloading the log files and looking in run_sandy.log, you'll see that the METviewer loading step fails. I'm particularly concerned about the ERROR messages shown below.

Creating mysql_mv ... 
Creating mysql_mv ... done
Creating metviewer ... 
Creating metviewer ... done
ERROR 2005 (HY000): Unknown MySQL server host 'mysql_mv' (2)
ERROR 2005 (HY000): Unknown MySQL server host 'mysql_mv' (2)
ERROR 2005 (HY000): Unknown MySQL server host 'mysql_mv' (2)
----  MVLoad  ----

Version: 4.0.2

Begin time: 2022-08-07 04:02:13
Parsing: /data/load_mv_sandy.xml

Aug 07, 2022 4:02:13 AM org.apache.tomcat.jdbc.pool.ConnectionPool init
SEVERE: Unable to create initial connections of pool.
java.sql.SQLNonTransientConnectionException: Could not connect to address=(host=mysql_mv)(port=3306)(type=master) : Socket fail to connect to host:mysql_mv, port:3306. mysql_mv

I'm trying to decide whether I should continue debugging this behavior or just remove METviewer from the GHA testing.

If this is something you've seen and have recommendations for getting past it, please let me know. Or if you have no idea and suspect it may be challenging to make work, please let me know that.

Thanks for the advice!

JohnHalleyGotway commented

Included logic to automatically rebuild images when the contents of the corresponding component directories have changed. Here's an example GHA run:

Run .github/jobs/set_job_controls.sh
Modified files (git diff --name-only 46[8](https://github.com/NCAR/container-dtc-nwp/runs/7773863585?check_suite_focus=true#step:3:9)165[9](https://github.com/NCAR/container-dtc-nwp/runs/7773863585?check_suite_focus=true#step:3:10)41e1fe122c5ddbfc8e4922076dfd5810e):
.github/jobs/set_job_controls.sh components/upp/Dockerfile

Build UPP since components/upp has changed.

Job Control Settings:
branch_name=feature_issue_48_gha
build_base=false
build_wps_wrf=false
build_gsi=false
build_upp=true
build_python=false
build_met=false
build_metviewer=false
run_sandy=true