NOAA-GFDL/MDTF-diagnostics

Containerization of MDTF

Opened this issue · 2 comments

aradhakrishnanGFDL commented

**One minute to MDTF: usability across all platforms (sorta) **

**Issue to track the container readiness (build as part of CI, run specific tests in CI and from two other platforms) and to record challenges, issues, future directions **

  • Scope of the first deliverable is limited to seeing how an example POD can be run using an MDTF container, and see how it may be built as part of an automated pipeline.

** A developer/contributor mode and an explorer mode from a containerized build may need to come with some best practices: document things learnt**

aradhakrishnanGFDL commented

Use my fork and the main branch for testing:

https://github.com/aradhakrishnanGFDL/MDTF-diagnostics

HOW TO TEST THE MDTF CONTAINER

  1. Install Docker desktop if you don’t have docker already.

  2. From your terminal, do the following:

docker pull public.ecr.aws/c6h4h4s8/mdtf/mdtf-test:latest
The above will pull the latest docker image built in the GitHub actions workflow associated with this container22 branch.

  1. If you haven't already, clone the MDTF github repo. The first cut of the docker image does not include the diagnostics to allow one to dev and contribute to github more swiftly. We may include it later.

  2. You likely have your MDTF sample inputdata in your environment. We will simply point to that and the docker light-weight for this release.
    If you don't have sample input datasets, please follow this documentation here
    general organization of inputdata: https://mdtf-diagnostics.readthedocs.io/en/main/sphinx/start_config.html

sample datasets from an FTP server: https://mdtf-diagnostics.readthedocs.io/en/main/sphinx/start_install.html#obtaining-supporting-data

  1. Ensure inputdata is present at some_path/inputdata. We will use this in the next step

  2. Type this from the terminal, refer 7 and then run with suggested changes.

docker run -it -v /Users/ar46/mdtf/MDTF-diagnostics/diagnostics/:/proj/MDTF-diagnostics/diagnostics/ -v /Users/ar46/mdtf/inputdata/:/proj/inputdata/ -v /Users/ar46/mdtf-dock/wkdir/:/proj/wkdir/ -p 8080:80 public.ecr.aws/c6h4h4s8/mdtf/mdtf-test where,

  1. Replace /Users/ar46/mdtf in /Users/ar46/mdtf/inputdata and /Users/ar46/mdtf/MDTF-diagnostics/diagnostics/ with your path to the inputdata and the diagnostics directory (from the cloned repo)
    Replace /Users/ar46/mdtf-dock/wkdir/ by a working directory in your host machine.
    Replace
    We are mounting volumes to the docker container such that they are available for our MDTF runs.

  2. From the docker container that is now in the "run" mode from the previous step, you may run MDTF!

micromamba activate _MDTF_base
mdtf_framework.py -f /proj/inputdata/tests/docker_test_set1.jsonc
  1. Where do I see the results?
    From your localhost, just open the HTML file from your work directory.
    In this example, I open the following from my web browser:
    /Users/ar46/mdtf-dock/wkdir/MDTF_GFDL.v2.Synthetic_1_10/index.html
aradhakrishnanGFDL commented

including @jkrasting, I have done limited testing on my mac. Someone else other than me testing and improving the instructions would be helpful.