GPU testing of PRs

[ianthomas23]

This is to document the GPU testing I usually perform for Datashader PRs. Currently there is no automated GPU testing in the GitHub Actions-based CI, so this needs to be performed manually.

Test machine can be local or remote, and needs to have a Nvidia graphics card. Recommended Operating System is Ubuntu, other Linux distros will also work and even Windows via WSL2 but they may be harder to set up. Driver needs to be installed for the graphics card, and the nvidia-utils package is useful as it provides the nvidia-smi command-line tool.

Run nvidia-smi to determine the driver and CUDA version. My results are:

Driver Version: 470.199.02   CUDA Version: 11.4

You need an appropriate conda/mamba environment containing cuDF. This is most easily obtained by installing RAPIDS following the installation guide at https://docs.rapids.ai/install. Opt for the stable release with the same version of CUDA that you are using, and your preferred choice of Python version (generally the later the better). There is other information on the RAPIDS install page to help if you experience problems.

Install Datashader using pip, e.g. from the root directory of your local copy of the datashader github repo:

pip install -ve .[tests]

GPU tests are not run by default but need an environment variable defined. To run the whole test suite on both CPU and GPU use

DATASHADER_TEST_GPU=1 pytest -v datashader/tests

Unless you are dealing with a GPU-specific issue, it is recommended for PRs to get everything working and tests written and passing for the CPU before adding GPU support.

There is a PR in progress to add GPU tests on a self-hosted runner: #1301

[hoxbro]

An update on this: this should now be done with pixi run test-gpu on a Linux machine with Cuda support.

For GPU on the CI maybe we could use something similar to what is described here: https://betatim.github.io/posts/github-action-with-gpu/