Speed up integration tests on GH Actions
Opened this issue · 0 comments
tomvothecoder commented
The integration tests take awhile to run. Downloading the test files take a significant chunk of the runtime (~3-5 minutes).
There should be a caching mechanism when downloading test files from LCRC. This logic should only download new files or replaces modified files for every build run after the initial one.
Resources:
- Interesting article on the cost and productivity implications with slow build times: https://github.blog/2022-12-08-experiment-the-hidden-costs-of-waiting-on-slow-build-times/
- Integration test script: https://github.com/E3SM-Project/e3sm_diags/blob/main/tests/test.sh
- Script for downloading data: https://github.com/E3SM-Project/e3sm_diags/blob/main/tests/integration/download_data.py
- Possible package for caching downloaded data: https://github.com/fatiando/pooch