/setup-miniconda

Set up your GitHub Actions workflow with conda via miniconda

Primary LanguageTypeScriptMIT LicenseMIT

conda-incubator/setup-miniconda

This action sets up a base conda environment by one of:

  • locating the conda installation bundled with the available runners and available in $CONDA
  • installing a specific (or latest) version of

A conda-build-version or mamba-version may be provided to install specific versions of conda or mamba into base

The base condabin/ folder is added to $PATH and shell integration is initialized across all platforms.

By default, this action will then create, and activate, an environment by one of:

  • creating a mostly-empty test environment, containing only the latest python-version and its dependencies
  • creating a test environment described in a given environment-file including:
    • an environment.yml-like file (which can be patched with python-version). Note: the patched environment will be cleaned up unless clean-patched-environment-file: false is given
    • a lockfile

This action correctly handles activation of environments and offers the possibility of automatically activating the test environment on all shells.

Please see the IMPORTANT notes on additional information on environment activation.

Example Overview

Each of the examples below is discussed in a dedicated section below.

Documentation Workflow Status
Basic usage Basic Usage Status
Other shells Other Shells Status
Other options Other Options Status
Channels Channels Status
Custom installer Custom Installer Status
Mamba Mamba Status
Lockfiles Lockfiles Status
Miniforge Miniforge Status
Alternative Architectures Alternative Architectures
Configure conda solver Configure conda solver
Caching packages Caching Example Status
Caching environments Caching Env Example Status
Apple Silicon Apple Silicon

Other Workflows

These are quality control and test workflows, and are not described in depth.

QA Workflow Linting Catch Invalid Enviroments Handle Empty Channels
Workflow Status Linting Status Catch Invalid Environments Status Handle Empty Channels Status

Environment activation

This action will, by default, activate an environment called test and not activate the base environment. This encourages the recommended practice of not installing workflow packages into the base environment and leaving it with only conda (and/or mamba).

Inputs and outputs

For a full list of available inputs and outputs for this action see action.yml.

Use a different environment name or path

You can change the default test environment to have a different name or path by setting the activate-environment input option.

- uses: conda-incubator/setup-miniconda@v3
  with:
    activate-environment: whatever

This will create a named env in $CONDA/envs/whatever, where $CONDA is the path to the infrequently-updated, but very fast to start, "bundled" Miniconda installation.

  • If activate-environment contains either POSIX or Windows slashes, it will be interpreted as a path, or prefix in conda terminology. Use this to avoid "path too long"-style errors, especially on Windows.
  • Self-hosted runners can emulate the "bundled" Miniconda approach by pre-installing a constructor-based installer and ensuring $CONDA is set prior to starting setup-miniconda

Activate base environment

If your specific workflow still needs to activate and use base you will need to do both of:

  • set activate-environment to an empty string
  • set auto-activate-base to true
- uses: conda-incubator/setup-miniconda@v3
  with:
    auto-activate-base: true
    activate-environment: ""

Usage examples

Example 1: Basic usage

This example shows how to set a basic python workflow with conda using the cross-platform available shells: bash and pwsh. In this example an environment named test will be created with the specific python-version installed for each operating system, resulting in 6 build workers.

jobs:
  example-1:
    name: Ex1 (${{ matrix.python-version }}, ${{ matrix.os }})
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false
      matrix:
        os: ["ubuntu-latest", "macos-latest", "windows-latest"]
        python-version: ["3.7", "3.11"]
    steps:
      - uses: conda-incubator/setup-miniconda@v3
        with:
          auto-update-conda: true
          python-version: ${{ matrix.python-version }}
      - name: Conda info
        shell: bash -el {0}
        run: conda info
      - name: Conda list
        shell: pwsh
        run: conda list

Example 2: Other shells

This example shows how to use all other available shells for specific operating systems. In this example we download the latest anaconda version then create and activate a default environment named foo.

jobs:
  example-2-linux:
    name: Ex2 Linux
    runs-on: "ubuntu-latest"
    steps:
      - uses: conda-incubator/setup-miniconda@v3
        with:
          miniconda-version: "latest"
          activate-environment: foo
      - name: Bash
        shell: bash -el {0}
        run: |
          conda info
          conda list
      - name: PowerShell Core
        shell: pwsh
        run: |
          conda info
          conda list

  example-2-mac:
    name: Ex2 Mac
    runs-on: "macos-latest"
    steps:
      - uses: conda-incubator/setup-miniconda@v3
        with:
          miniconda-version: "latest"
          activate-environment: foo
      - name: Sh
        shell: sh -l {0}
        run: |
          conda info
          conda list
      - name: Bash
        shell: bash -el {0}
        run: |
          conda info
          conda list
      - name: PowerShell Core
        shell: pwsh
        run: |
          conda info
          conda list

  example-2-win:
    name: Ex2 Windows
    runs-on: "windows-latest"
    steps:
      - uses: conda-incubator/setup-miniconda@v3
        with:
          miniconda-version: "latest"
          activate-environment: foo
      - name: Bash
        shell: bash -el {0}
        run: |
          conda info
          conda list
      - name: PowerShell
        shell: powershell
        run: |
          conda info
          conda list
      - name: PowerShell Core
        shell: pwsh
        run: |
          conda info
          conda list
      - name: Cmd.exe
        shell: cmd /C CALL {0}
        run: >-
          conda info && conda list

Example 3: Other options

This example shows how to use environment.yml for easier creation of test/build environments and .condarc files for fine grained configuration management. In this example we use a custom configuration file, install an environment from a yaml file, and disable autoactivating the base environment before activating the anaconda-client-env.

jobs:
  example-3:
    name: Ex3 Linux
    runs-on: "ubuntu-latest"
    defaults:
      run:
        shell: bash -el {0}
    steps:
      - uses: actions/checkout@v4
      - uses: conda-incubator/setup-miniconda@v3
        with:
          activate-environment: anaconda-client-env
          environment-file: etc/example-environment.yml
          python-version: 3.5
          condarc-file: etc/example-condarc.yml
          auto-activate-base: false
      - run: |
          conda info
          conda list

Example 4: Conda options

This example shows how to use the channels option and other extra options. The priority will be set by the order of the channels. The following example will result in these priorities (from highest to lowest):

  • conda-forge
  • spyder-ide
  • defaults
jobs:
  example-4:
    name: Ex4 Linux
    runs-on: "ubuntu-latest"
    defaults:
      run:
        shell: bash -el {0}
    steps:
      - uses: actions/checkout@v4
      - uses: conda-incubator/setup-miniconda@v3
        with:
          activate-environment: foo
          python-version: 3.6
          channels: conda-forge,spyder-ide
          allow-softlinks: true
          channel-priority: flexible
          show-channel-urls: true
          use-only-tar-bz2: true
      - run: |
          conda info
          conda list
          conda config --show-sources
          conda config --show

Example 5: Custom installer

Any installer created with the constructor tool (which includes conda) can be used in place of Miniconda. For example, conda-forge maintains additional builds of miniforge for platforms not yet supported by Miniconda. For more details, see Example 10.

Note:

  • Installer downloads are cached based on their full URL: adding some non-functional salt to the URL will prevent this behavior, e.g., #${{ github.run_number }}
jobs:
  example-5:
    name: Ex5 Miniforge for PyPy
    runs-on: "ubuntu-latest"
    defaults:
      run:
        shell: bash -el {0}
    steps:
      - uses: actions/checkout@v4
      - uses: conda-incubator/setup-miniconda@v3
        with:
          installer-url: https://github.com/conda-forge/miniforge/releases/download/4.8.3-2/Miniforge-pypy3-4.8.3-2-Linux-x86_64.sh
          allow-softlinks: true
          show-channel-urls: true
          use-only-tar-bz2: true
      - run: |
          conda info
          conda list
          conda config --show-sources
          conda config --show

Example 6: Mamba

Experimental! Use mamba to enable much faster conda installs. mamba-version accepts a version string x.y (including "*"). It requires you specify conda-forge as part of the channels, ideally with the highest priority.

Notes:

  • If a custom installer provides mamba, it can be prioritized wherever possible (including installing mamba-version) with use-mamba: true.
jobs:
  example-6:
    name: Ex6 Mamba
    runs-on: "ubuntu-latest"
    steps:
      - uses: actions/checkout@v4
      - uses: conda-incubator/setup-miniconda@v3
        with:
          python-version: 3.6
          mamba-version: "*"
          channels: conda-forge,defaults
          channel-priority: true
          activate-environment: anaconda-client-env
          environment-file: etc/example-environment.yml
      - shell: bash -el {0}
        run: |
          conda info
          conda list
          conda config --show-sources
          conda config --show
          printenv | sort
      - shell: bash -el {0}
        run: mamba install jupyterlab

Example 7: Lockfiles

conda list --explicit and conda-lock support generating explicit environment specifications, which skip the environment solution step altogether, as they contain the ordered list of exact URLs needed to reproduce the environment.

This means explicitly-defined environments which:

  • are much faster to install, as several expensive steps are skipped:
    • channels are not queried for their repo data
    • no solver is run
  • are not cross-platform, as the URLs almost always contain platform/architecture information
  • can become broken if any file becomes unavailable

This approach can be useful as part of a larger system e.g., a separate workflow that runs conda-lock for all the platforms needed in a separate job.

jobs:
  example-7:
    name: Ex7 Explicit
    runs-on: "ubuntu-latest"
    defaults:
      run:
        shell: bash -el {0}
    steps:
      - uses: actions/checkout@v4
      - uses: conda-incubator/setup-miniconda@v3
        with:
          auto-update-conda: false
          activate-environment: explicit-env
          environment-file: etc/example-explicit.conda.lock
      - run: |
          conda info
          conda list
          conda config --show-sources
          conda config --show
          printenv | sort

Example 10: Miniforge

Miniforge provides a number of alternatives to Miniconda, built from the ground up with conda-forge packages and with only conda-forge in its default channel(s).

If only miniforge-version is provided then Miniforge3 will be used.

jobs:
  example-10-miniforge:
    name: Ex10 (${{ matrix.os }}, Miniforge)
    runs-on: ${{ matrix.os }}-latest
    strategy:
      matrix:
        os: ["ubuntu", "macos", "windows"]
    steps:
      - uses: actions/checkout@v4
      - uses: conda-incubator/setup-miniconda@v3
        with:
          environment-file: etc/example-environment.yml
          miniforge-version: latest

In addition to Miniforge3 with conda and CPython, for each of its many supported platforms and architectures, additional variants including Mambaforge (which comes pre-installed mamba in addition to conda on all platforms) and Miniforge-pypy3/Mamabaforge-pypy3 (which replace CPython with pypy3 on Linux/MacOS) are available.

jobs:
  example-10-mambaforge:
    name: Ex10 (${{ matrix.os }}, Mambaforge)
    runs-on: ${{ matrix.os }}-latest
    strategy:
      fail-fast: false
      matrix:
        os: ["ubuntu", "macos", "windows"]
        include:
          - os: ubuntu
            environment-file: etc/example-environment-no-name.yml
            miniforge-variant: Mambaforge
            miniforge-version: 4.9.2-4
          - os: windows
            environment-file: etc/example-explicit.Windows.conda.lock
            condarc-file: etc/example-condarc.yml
            miniforge-variant: Mambaforge
    steps:
      - uses: actions/checkout@v4
      - uses: conda-incubator/setup-miniconda@v3
        with:
          condarc-file: ${{ matrix.condarc-file }}
          environment-file: ${{ matrix.environment-file }}
          miniforge-variant: ${{ matrix.miniforge-variant }}
          miniforge-version: ${{ matrix.miniforge-version }}
          use-mamba: true

Example 11: Alternative Architectures

In addition to the default 64-bit builds of Miniconda, 32-bit versions are available for Windows. Note that although some x86 builds are available for Linux and MacOS, these are too old (<4.6) to be supported by this action.

jobs:
  example-11:
    name:
      Ex11 (os=${{ matrix.os }} architecture=${{ matrix.architecture }}
      miniconda-version=${{ matrix.miniconda-version }})
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false
      matrix:
        os: ["windows-latest"]
        architecture: ["x86"]
        miniconda-version: ["latest"]
    steps:
      - uses: actions/checkout@v4
      - uses: conda-incubator/setup-miniconda@v3
        with:
          architecture: ${{ matrix.architecture }}
          miniconda-version: $${{ matrix.miniconda-version }}
          auto-update-conda: true
          python-version: "3.8"

Example 12: Configure conda solver

Set the conda solver plugin to use. Only applies to the conda client, not mamba. Starting with Miniconda 23.5.2 and Miniforge 23.3.1, you can choose between classic and libmamba. Best when combined with auto-update-conda: true.

jobs:
  example-12:
    name: Ex12 (os=${{ matrix.os }} solver=${{ matrix.solver }})
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false
      matrix:
        solver: ["classic", "libmamba"]
        os: ["ubuntu-latest", "windows-latest"]
    steps:
      - uses: actions/checkout@v4
      - uses: conda-incubator/setup-miniconda@v3
        id: setup-miniconda
        continue-on-error: true
        with:
          auto-update-conda: true
          conda-solver: ${{ matrix.solver }}
          python-version: "3.9"

Example 13: Apple Silicon

jobs:
  example-13:
    name: Ex13 (os=${{ matrix.os }})
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false
      matrix:
        os: ["macos-14"]
    steps:
      - uses: actions/checkout@v4
      - uses: ./
        id: setup-miniconda
        continue-on-error: true
        with:
          miniconda-version: latest
      - name: Check arm64
        shell: bash -el {0}
        run: |
          conda install -y python
          python -c "import platform; assert platform.machine() == 'arm64', platform.machine()"

Caching

Caching packages

If you want to enable package caching for conda you can use the cache action using ~/conda_pkgs_dir as path for conda packages.

The cache will use an explicit key for restoring and saving the cache.

This can be based in the contents of files like:

  • setup.py
  • requirements.txt
  • environment.yml
jobs:
  caching-example:
    name: Caching
    runs-on: "ubuntu-latest"
    steps:
      - uses: actions/checkout@v4
      - name: Cache conda
        uses: actions/cache@v3
        env:
          # Increase this value to reset cache if etc/example-environment.yml has not changed
          CACHE_NUMBER: 0
        with:
          path: ~/conda_pkgs_dir
          key:
            ${{ runner.os }}-conda-${{ env.CACHE_NUMBER }}-${{
            hashFiles('etc/example-environment.yml') }}
      - uses: conda-incubator/setup-miniconda@v3
        with:
          activate-environment: anaconda-client-env
          channel-priority: strict
          environment-file: etc/example-environment.yml
          use-only-tar-bz2: true # IMPORTANT: This needs to be set for caching to work properly!

If you are using pip to resolve any dependencies in your conda environment then you may want to cache those dependencies separately, as they are not included in the conda package cache.

Caching environments

The first installation step should setup a Miniconda variant without specifying a environment file.

- name: Setup Mambaforge
  uses: conda-incubator/setup-miniconda@v3
  with:
    miniforge-variant: Mambaforge
    miniforge-version: latest
    activate-environment: anaconda-client-env
    use-mamba: true

It's a good idea to refresh the cache every 24 hours to avoid inconsistencies of package versions between the CI pipeline and local installations. Here we ensure that this happens by adding the current date to the cache key. You can remove the "Get Date" step below if you use a resolved environment file product of conda env export or conda list --explicit.

- name: Get Date
  id: get-date
  run: echo "today=$(/bin/date -u '+%Y%m%d')" >> $GITHUB_OUTPUT
  shell: bash

- name: Cache Conda env
  uses: actions/cache@v3
  with:
    path: ${{ env.CONDA }}/envs
    key:
      conda-${{ runner.os }}--${{ runner.arch }}--${{
      steps.get-date.outputs.today }}-${{
      hashFiles('etc/example-environment-caching.yml') }}-${{ env.CACHE_NUMBER
      }}
  env:
    # Increase this value to reset cache if etc/example-environment.yml has not changed
    CACHE_NUMBER: 0
  id: cache

Keep in mind that hashing etc/example-environment-caching.yml is not the same as hashing a resolved environment file. conda (and mamba) resolves the dependencies declared in the YAML file according to the packages available on the channels at installation time. Since packages are updated all the time, you will not see these changes reflected in the cache until the key gets updated by date.

This means that the same environment file can make your tests pass locally but fail on CI, or the other way around. In that case, reset the cache manually to see if that leads to consistent results, or use a resolved environment file.

Finally, update the environment based on the environment file if the cache does not exist.

- name: Update environment
  run:
    mamba env update -n anaconda-client-env -f
    etc/example-environment-caching.yml
  if: steps.cache.outputs.cache-hit != 'true'

Use a default shell

If you use the same shell for every step in your workflow you don't have to add a shell directive to every step (e.g., shell: bash -el {0} when using bash).

You can add a defaults section and specify the desired directive (e.g., bash -el {0} or equivalent). All steps in the job will then default to using that value.

For other shells, make sure to use the correct shell parameter as the default value. Check the section below for some examples.

For more information see the Github Actions help page.

jobs:
  default-shell:
    name: Default shell
    runs-on: "ubuntu-latest"
    defaults:
      run:
        shell: bash -el {0}
    steps:
      - uses: actions/checkout@v4
      - uses: conda-incubator/setup-miniconda@v3
        with:
          activate-environment: anaconda-client-env
          environment-file: etc/example-environment-caching.yml
      - run: conda info
      - run: conda list
      - run: conda config --show

IMPORTANT

  • Conda activation does not correctly work on sh. Please use bash.
  • Bash shells do not use ~/.profile or ~/.bashrc so these shells need to be explicitly declared as shell: bash -el {0} on steps that need to be properly activated (or use a default shell). This is because bash shells are executed with bash --noprofile --norc -eo pipefail {0} thus ignoring updated on bash profile files made by conda init bash. See Github Actions Documentation and this community thread.
  • Sh shells do not use ~/.profile or ~/.bashrc so these shells need to be explicitly declared as shell: sh -l {0} on steps that need to be properly activated (or use a default shell). This is because sh shells are executed with sh -e {0} thus ignoring updates on bash profile files made by conda init bash. See Github Actions Documentation.
  • Cmd shells do not run Autorun commands so these shells need to be explicitly declared as shell: cmd /C call {0} on steps that need to be properly activated (or use a default shell). This is because cmd shells are executed with %ComSpec% /D /E:ON /V:OFF /S /C "CALL "{0}"" and the /D flag disables execution of Command Processor/Autorun Windows registry keys, which is what conda init cmd.exe sets. See Github Actions Documentation.
  • For caching to work properly, you will need to set the use-only-tar-bz2 option to true.
  • Some options (e.g. use-only-tar-bz2) are not available on the default conda installed on Windows VMs, be sure to use auto-update-conda or provide a version of conda compatible with the option.
  • If you plan to use a environment.yaml file to set up the environment, the action will read the channels listed in the key (if found). If you provide the channels input in the action they must not conflict with what was defined in environment.yaml, otherwise the conda solver might find conflicts which cause very long install times or install failures.

Security / Reproducibility

Security and reproducibility is important especially when workflows deal with secrets. No matter how much individual Github action repositories are secured, git branches and tags are always mutable. It is thus good practice to:

  1. pin the action to a specific sha1 with tag as comment, instead of e.g. using v2 or v2.2.1 (which are mutable tags): uses: conda-incubator/setup-miniconda@9f54435e0e72c53962ee863144e47a4b094bfd35 # v2.3.0 see example
  2. keep the non-human-readable pinning updated to not run behind recent updates and fixes via automation like renovate or dependabot
  3. use conda-lock files, see conda-lock

Project History and Contributing

See the CHANGELOG for project history, or CONTRIBUTING to get started adding features you need.

Similar Actions to work with conda packages

Contributors

Thanks to all the contributors that make this awesome project possible!

Meet our contributors!

Made with contributors-img.

License

The scripts and documentation in this project are released under the MIT License