Setup cycling test C96_atm3DVar on Gaea-C5
Closed this issue · 7 comments
What new functionality do you need?
Add the ability to run CI test C96_atm3DVar and C96_atm3DVar_extended on Gaea-C5
What are the requirements for the new functionality?
Clone, build, and install obsproc, prepobs, and fit2obs
Run C96_atm3DVar and C96_atm3DVar_extended CI cases
Acceptance Criteria
Successful completion of C96_atm3DVar and C96_atm3DVar_extended CI test on Gaea-C5
Suggest a solution (optional)
No response
Hi @aerorahul @WalterKolczynski-NOAA It appears I don't have permissions to update assignees in NOAA-EMC/global-workflow. Could you please assign this issue to me?
Hi @aerorahul @DavidHuber-NOAA I'm looking for some advice before proceeding...prepobs and obsproc both use hpc-stack to build on Hera, Jet, and Orion. From what I can see, hpc-stack is not installed on gaea and is most likely supported by individual groups. Here are the locations on other HPC:
Hera: /scratch2/NCEPDEV/nwprod/hpc-stack
Jet: /lfs4/HFIP/hfv3gfs/role.epic/hpc-stack
Orion: /apps/contrib/NCEP/libs/hpc-stack
Maybe this is a question for the prepobs/obsproc team but figured I'd ask for your advice first.
fit2obs uses spack-stack so that built on Gaea no problem. Thanks.
@DavidBurrows-NCO
We use the branch dev/gfsv17 for prepobs.
I am not sure which branch is used for obsproc @KateFriedman-NOAA would know.
fit2obs uses develop branch, so that's good.
All of these should be using spack-stack 1.6.0
If you add Gaed modulefiles, please open a PR to those repos and tag @KateFriedman-NOAA for review.
Thanks @aerorahul for the information. prepobs dev/gfsv17 built using spack-stack 1.6.0 with no problem on Gaea, just added standard mod files...Yes, when they are ready, I will open PRs and tag @KateFriedman-NOAA
@DavidBurrows-NCO @aerorahul FYI, we are just using the current obsproc ops tag for v1.1.2 (OT.obsproc.v1.1.2_20221130). We don't currently use any execs from the obsproc package. We use some scripts from obsproc (jobs/JOBSPROC_GLOBAL_PREP then scripts/exglobal_makeprepbufr.sh), which then invoke the prepobs package. We have thus far not needed to make changes to the obsproc package for GFSv17. I do need to move us to obsproc.v1.2 (the newest version in ops) but it hasn't been critical. Hoping to do that soon.
So yeah, @DavidBurrows-NCO we'd only need Gaea updates in our prepobs dev/gfsv17 branch. I'm happy to review any PRs into that branch in the prepobs repo. Please open an issue in the prepobs repo to document the Gaea update into our gfs branch. Thanks and let me know if you have any questions!
@DavidBurrows-NCO Another note...in a prepobs branch, make sure to increment the version in the VERSION file. We can go to 1.0.3 for this update. I'm happy to do a quick test of your branch on one of the other platforms if that would help. Let me know.
Great. Thanks @KateFriedman-NOAA. I created an issue for prepobs here. I'll reach back out here once I run a test...