NOAA-GFDL/SHiELD_build

Features required for building a Python-wrapped version of SHiELD

Closed this issue · 7 comments

spencerkclark commented

Is your feature request related to a problem? Please describe.

At AI2 we have developed a Python-wrapped version of NOAA's FV3GFS model, a close relative of SHiELD. The code is currently maintained here, and described in McGibbon et al. (2021). It has been a powerful tool that has facilitated several published hybrid machine learning research projects. A drawback of it, however, is that while similar to SHiELD, our now aging fork of FV3GFS is not the same, and a number of features lag behind what has been developed and made public in SHiELD. We would like to transition from wrapping FV3GFS to wrapping SHiELD, both for our work at AI2 and for the fact that it would make it more attractive (and intuitive) for users of SHiELD to use our Python-wrapped model. It has been something on the roadmap of @lharris4 and our group for a while.

I have set up a repository for such a wrapper, which includes a PR introducing a basic initial working version: ai2cm/SHiELD-wrapper#1. This is not totally feature-complete (as described in the PR description), but it represents a major first step towards our eventual goal. In addition, as part of this goal, we would like to move away from forking the entire fortran codebase and instead depend directly on the evolving public repositories of the NOAA-GFDL organization, which will keep the Python wrapper up to date. What this means, however, is that any changes to the fortran code and build system required to implement wrapper features must be made in these repositories. This issue is meant to introduce this overall effort, and to discuss how best to incorporate the initial required changes for wrapping SHiELD in Python in a similar way to how we have wrapped FV3GFS.

Initial required changes

In this initial phase, the main changes required were to the SHiELD_build repository (I made a stab at this in this branch, but there may be another preferred way of going about it):

  • We need to be able to build a static library containing the dynamical core and atmos_drivers code, which we can link to when compiling the wrapper.
    • I did this by adding another config option for shield_wrapper, which splits off all but the coupler_main.F90 file into static libraries, before building the executable.
  • We need a way to compile all pieces of the code (FMS, nceplibs, and SHiELD code) as "position independent code," i.e. with the -fPIC flag.
    • I did this by threading through the config option into the FMS, nceplibs, and SHiELD build scripts, and adding the -fPIC flag in the event that config equals "shield_wrapper".

There was also a very minor change needed to declare some variables as public in the atmos_drivers repository (see this small diff).

Ask

Please let me know from the SHiELD / SHiELD_build perspective if things look reasonable enough for me to initiate pull requests from these branches for review. There is no need to review ai2cm/SHiELD-wrapper#1, though you are of course welcome to take a look there for more context (I will wait to merge that until I am no longer pointing to personal forks of the SHiELD_build and atmos_drivers repositories).

I am happy to answer any questions about / discuss this project that come up now or later. Thanks!

Disclaimer

I totally recognize I am posting this issue on the eve of a possible government shutdown, and so I understand that you will likely not be able to look deeply into this issue today (and possibly not for several days into the future).

lharris4 commented
lharris4 commented
spencerkclark commented

Thanks @lharris4, yes, it will be useful to get @bensonr's thoughts.

32-bit/mixed-mode support has indeed crossed my mind, though it will take some time to think about how to best support that in the wrapper, which we have sort of punted on up to this point with FV3GFS. My initial priority, however, will be on implementing the additional features necessary for @tlhsieh's work.

lharris4 commented
spencerkclark commented

Understood -- I created ai2cm/SHiELD-wrapper#2 to track further discussion on that topic, since it should not have any bearing on the initial discussion here.

  • 👍1
bensonr commented

Back to the point from @spencerkclark and @lharris4 regarding FV3 as a library. The typical way FRE compiles up models is to build each component as a static library and link them in when compiling up the driver program (in this case coupler_main.F90). If you have a PR that does this in general and not just for your specific project, I think it would be a welcome update to the SHiELD_build system.

spencerkclark commented

Thanks @bensonr -- that makes sense. Per our discussion yesterday, I made a couple PRs related to these changes, which should be ready for your and / or @laurenchilutti's review when you get a chance: