NOAA-GFDL/pace

Pace refactor

Closed this issue · 9 comments

bensonr commented

@FlorianDeconinck - NDSL, PyFV3, and PySHiELD have been created in the NOAA-GFDL organization. One the current PRs are accepted and merged, I think we can proceed with the plan.

FlorianDeconinck commented

👍

Github doesn't allow me to cross-post issues or move an issue from our GEOS-ESM project into the NOAA-GFDL, so we'll track progress on the GEOS-ESM#41 side of things, and I'll cross post a summary here

bensonr commented

I started to add this info in the NASA PR, but thought maybe you wouldn't want clutter info about the a different org in your PR.

FlorianDeconinck commented

I think it's good info on the issue, I don't segregate it's all Open Source. I would have rather put everything here but Github is not allowing me so we might have those kind of cross-posting issue often

FlorianDeconinck commented

First milestone ndsl.

Done

  • pace.stencils, pace.dsl, pace.util have been moved to ndsl
  • util has been flattened, files grouped into logical directory and cleaned of unused pieces. More cleanup to do.
  • external has moved to ndsl to submodules gt4py and dace
  • pace is now using ndsl as a submodule
  • update_dwind_phys.py, update_atmos_state.py and fv_update_phys.py lives in Physics but should be part of a broader plug & play Physics API that know how to interface with the Dynamical Core. (poke @oelbert)
  • A C12 data bundle for testing restart has been moved to tests/main/data on pace it was living in util before.
  • PACE_LOGLEVEL became NDSL_LOGLEVEL, more of those will follow.

ToDo

  • tracer_variables global variable in gt4py_utils: need an API in ndsl and calls in the model. Stays in ndsl for now, API needed and will be worked on as requirements for GEOS.
  • halo and origin global variables in gt4py_utils: those should have an API with reasonable defaults.
  • Reconnect all GitHub workflows
  • ndsl.restart should move to pace and/or pyFV3

The import structure as "ballooned up" on most files because we were using pace.util as a global importer and we broke that in the refactor for clarity. The good practice around __init__.py is to have a docstrings and a curated list of imports. ndsl is a large middleware, this might not be suitable. Discussion should happen and guidelines updated.

FlorianDeconinck commented

ndsl is in. Much clean up needs to happen but the basics are done and moved in Pace

FlorianDeconinck commented

pyFV3 and pySHiELD have been created and their main branch is now protected (discussion on git strategy to follow)

Pace has been refactored to bring together all the packages.

FlorianDeconinck commented

Remaining task:

  • docs needs splitting
  • Makefile workflow is broken in all repositories
  • docker is broken in all repositories

Clean up on naming and random bits will be an ongoing effort.

FlorianDeconinck commented

New repositories:
ndsl
pyFV3
pySHiELD