dtcenter/METplus

Update Truth: For dtcenter/MET#2967

Closed this issue · 1 comments

JohnHalleyGotway commented

Describe Expected Changes

This work in the MET repository switches from storing integer contingency table counts to storing double-precision sums of weights. This is needed to compute weighted contingency table statistics (such as area-weighted).

During this development, a bug was uncovered in the computation of the Odds-Ratio-Skill-Score where integer overflow was occurring. Switching to double-precision variables solves the problem. However, differences occur in the MET and METplus Use Case output files where this fix appears in the ORSS, ORSS_NCL, and ORSS_NCU columns of the CTS and NBRCTS line types.

Define the Metadata

Title

  • Define the Title of this issue as Update Truth: For dtcenter/{REPO}#{PR_NUMBER} to indicate the repository and pull request that warranted this issue.

Assignee

  • Select engineer(s) or no engineer required
  • Select scientist(s) or no scientist required

Milestone and Projects

  • Select Milestone as the next official version if updating truth data for the develop branch OR select next METplus-Wrappers-X.Y.Z bugfix version if updating truth data for a main_vX.Y branch.
  • If updating truth data for the develop branch, select the METplus-Wrappers-X.Y.Z Development project OR if updating truth for a main_vX.Y branch, select the Coordinated METplus-X.Y Support project

Update Truth Checklist

  • Review the GitHub Actions workflow that was triggered by the PR merge
    • If no differences were found, note this in a comment.
    • If all of the differences are expected, note this in a comment.
      Include any details of how the review was performed.
    • If unexpected differences are found, the following instructions can
      help uncover potential explanations. If none of these apply and the
      source of the differences cannot be determined, contact the
      METplus wrappers lead engineer (@georgemccabe) for assistance.
      • Search for other open issues that have the label type: update truth
        applied by clicking on the label on this issue. Coordinate with the
        author of these issues to ensure all diffs are properly reviewed.
      • Check if any additional GitHub Actions testing workflows have been
        triggered since the workflow that corresponds to this issue was run.
        Review the latest run to ensure that there are no diffs that are
        unrelated to this issue.
      • If the incorrect differences are caused by the changes from the
        issue that warranted this issue, consider reverting the PR and
        re-opening the issue.
    • Iterate until one of the above conditions apply.
  • Approve the update of the truth data
    • Contact the METplus wrappers lead engineer (@georgemccabe) or
      backup lead (@jprestop) to let them know that the truth data can
      be updated.
  • Update the truth data.
    This should be handled by a METplus wrappers engineer.
    See the instructions to update the truth data
    for more info.
  • Close this issue.
JohnHalleyGotway commented

This METplus testing.yml workflow was launched by the recent dtcenter/MET#2967 update to the develop branch.

Differences are flagged in 5 of the 56 use case groups:

  1. Use Case Tests (clouds:1)
  2. Use Case Tests (clouds:5)
  3. Use Case Tests (s2s_mid_lat:0-2)
  4. Use Case Tests (short_range:7)
  5. Use Case Tests (short_range:8)

I downloaded all the diff files and see that a total of 9 output files are affected:

.//short_range/MODE_fcstFV3_obsGOES_BrightnessTemp/short_range/brightness_temperature/mode_FV3_core_lsm1_020000L_20190521_020000V_000000A_cts_output.txt
.//short_range/MODE_fcstFV3_obsGOES_BrightnessTempObjs/short_range/brightness_temperature/mode/mode_FV3_core_lsm1_010000L_20190521_010000V_000000A_cts_output.txt
.//short_range/MODE_fcstFV3_obsGOES_BrightnessTempObjs/short_range/brightness_temperature/mode/mode_FV3_core_lsm1_020000L_20190521_020000V_000000A_cts_output.txt
.//short_range/MODE_fcstFV3_obsGOES_BrightnessTempObjs/short_range/brightness_temperature/mode/mode_FV3_core_mp1_010000L_20190521_010000V_000000A_cts_output.txt
.//short_range/MODE_fcstFV3_obsGOES_BrightnessTempObjs/short_range/brightness_temperature/mode/mode_FV3_core_mp1_020000L_20190521_020000V_000000A_cts_output.txt
.//s2s_mid_lat/UserScript_fcstGFS_obsERA_Blocking/s2s_mid_lat/UserScript_fcstGFS_obsERA_Blocking/GFS_ERA_IBLS_240000L_CTS_CNT_output.stat
.//s2s_mid_lat/UserScript_fcstGFS_obsERA_Blocking/s2s_mid_lat/UserScript_fcstGFS_obsERA_Blocking/GFS_ERA_Blocks_240000L_CTS_output.stat
.//clouds/GridStat_fcstGFS_obsMERRA2_lowAndTotalCloudFrac/model_applications/clouds/GridStat_fcstGFS_obsMERRA2_lowAndTotalCloudFrac/grid_stat_GFS_to_MERRA2_F36_CloudFracs_NBR_360000L_20210705_000000V_output.stat
.//clouds/GridStat_fcstMPAS_obsSATCORPS_lowAndTotalCloudFrac/model_applications/clouds/GridStat_fcstMPAS_obsSATCORPS_lowAndTotalCloudFrac/grid_stat_MPAS_F36_CloudFracs_NBR_360000L_20200724_120000V_output.stat

I used opendiff to inspect the diffs in all 9 output files and confirmed they're limited to the MODE CTS ORSS column and the STAT CTS ORSS plus CI columns.