popsim-consortium/stdpopsim

Fix CI

Closed this issue · 5 comments

Currently the tests seem to all fail quickly with

EnvironmentNameNotFound: Could not find conda environment: stdpopsim
You can list all discoverable environments with `conda info --envs`.

I have tried bumping the cache number; no luck. Things I would do to debug this are:

  1. update our .yml file, maybe using the tskit one as a template - maybe our actions are out of date
  2. putting in some debugging info into the .yml (e.g. get it to run conda info --envs like it suggests?)
  3. adding the magic stuff that lets you ssh into it - there's an example of how to do this somewhere on the web
nspope commented

CI is working after another cache bump -- for folks' open PRs to use the right cache, though, they'll have to rebase onto main

nspope commented

Spoke to soon!

What seems to be happening is:

  1. We bump cache number, and run CI: the cache isn't found, so conda gets a fresh install,
Cache not found for input keys: Linux-3.8-conda-cbac2d4b1a0192ba33b618f9d0e4afd00393538ef1a85231be810bed0a6a1e53-8

and the test suite runs.

  1. After the test suite finishes, CI tries to store the conda installation in the cache, but (for some reason) the resulting archive is empty,
Post job cleanup.
/usr/bin/tar --posix -cf cache.tzst --exclude cache.tzst -P -C /home/runner/work/stdpopsim/stdpopsim --files-from manifest.txt --use-compress-program zstdmt
Cache Size: ~0 MB (541 B)
Cache saved successfully
Cache saved with key: Linux-3.8-conda-cbac2d4b1a0192ba33b618f9d0e4afd00393538ef1a85231be810bed0a6a1e53-8
  1. A subsequent run of the CI with the same cache number pulls this empty archive:
/usr/bin/tar -tf /home/runner/work/_temp/8a815e20-78ab-49d4-9ae8-1b085b735580/cache.tzst -P --use-compress-program unzstd
../../../.bash_profile
Cache Size: ~0 MB (535 B)
/usr/bin/tar -xf /home/runner/work/_temp/8a815e20-78ab-49d4-9ae8-1b085b735580/cache.tzst -P -C /home/runner/work/stdpopsim/stdpopsim --use-compress-program unzstd
Cache restored successfully
Cache restored from key: Linux-3.10-conda-cbac2d4b1a0192ba33b618f9d0e4afd00393538ef1a85231be810bed0a6a1e53-8

and then the test suite doesn't execute

nspope commented

Found the problem -- for future reference:

- name: find conda
id: find-conda
run: |
echo "name=CONDA::$CONDA" >> $GITHUB_OUTPUT

is supposed to save the $CONDA variable into steps.find-conda.outputs.CONDA. This is used later as the path for the installation to cache. But the syntax is wrong: it should be echo "CONDA=$CONDA" >> $GITHUB_OUTPUT as described here. So, steps.find-conda.outputs.CONDA is empty; the resulting cache is empty; and future runs that try to use the cache don't end up with a stdpopsim installation.

nspope commented

Tests now work in #1520 after a rebase, so tentatively calling this solved

Looks like it has solved it! Nice work.