Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GPU testing of PRs #1302

Open
ianthomas23 opened this issue Oct 30, 2023 · 1 comment
Open

GPU testing of PRs #1302

ianthomas23 opened this issue Oct 30, 2023 · 1 comment

Comments

@ianthomas23
Copy link
Member

This is to document the GPU testing I usually perform for Datashader PRs. Currently there is no automated GPU testing in the GitHub Actions-based CI, so this needs to be performed manually.

Test machine can be local or remote, and needs to have a Nvidia graphics card. Recommended Operating System is Ubuntu, other Linux distros will also work and even Windows via WSL2 but they may be harder to set up. Driver needs to be installed for the graphics card, and the nvidia-utils package is useful as it provides the nvidia-smi command-line tool.

Run nvidia-smi to determine the driver and CUDA version. My results are:

Driver Version: 470.199.02   CUDA Version: 11.4

You need an appropriate conda/mamba environment containing cuDF. This is most easily obtained by installing RAPIDS following the installation guide at https://docs.rapids.ai/install. Opt for the stable release with the same version of CUDA that you are using, and your preferred choice of Python version (generally the later the better). There is other information on the RAPIDS install page to help if you experience problems.

Install Datashader using pip, e.g. from the root directory of your local copy of the datashader github repo:

pip install -ve .[tests]

GPU tests are not run by default but need an environment variable defined. To run the whole test suite on both CPU and GPU use

DATASHADER_TEST_GPU=1 pytest -v datashader/tests

Unless you are dealing with a GPU-specific issue, it is recommended for PRs to get everything working and tests written and passing for the CPU before adding GPU support.

There is a PR in progress to add GPU tests on a self-hosted runner: #1301

@hoxbro
Copy link
Member

hoxbro commented Aug 26, 2024

An update on this: this should now be done with pixi run test-gpu on a Linux machine with Cuda support.

For GPU on the CI maybe we could use something similar to what is described here: https://betatim.github.io/posts/github-action-with-gpu/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants