The following instructions are for users wishing to build cuGraph from source code. These instructions are tested on supported distributions of Linux, CUDA, and Python - See RAPIDS Getting Started for list of supported environments. Other operating systems might be compatible, but are not currently tested.
The cuGraph package include both a C/C++ CUDA portion and a python portion. Both libraries need to be installed in order for cuGraph to operate correctly.
Compiler:
gccversion 9.3+nvccversion 11.0+cmakeversion 3.20.1+
CUDA:
- CUDA 11.0+
- NVIDIA driver 450.80.02+
- Pascal architecture or better
You can obtain CUDA from https://developer.nvidia.com/cuda-downloads.
To install cuGraph from source, ensure the dependencies are met.
GIT clone a version of the repository
# Set the localtion to cuGraph in an environment variable CUGRAPH_HOME
export CUGRAPH_HOME=$(pwd)/cugraph
# Download the cuGraph repo - if you have a folked version, use that path here instead
git clone https://github.com/rapidsai/cugraph.git $CUGRAPH_HOME
cd $CUGRAPH_HOMECreate the conda development environment
# create the conda environment (assuming in base `cugraph` directory)
# for CUDA 11.0
conda env create --name cugraph_dev --file conda/environments/cugraph_dev_cuda11.0.yml
# for CUDA 11.2
conda env create --name cugraph_dev --file conda/environments/cugraph_dev_cuda11.2.yml
# for CUDA 11.4
conda env create --name cugraph_dev --file conda/environments/cugraph_dev_cuda11.4.yml
# for CUDA 11.5
conda env create --name cugraph_dev --file conda/environments/cugraph_dev_cuda11.5.yml
# activate the environment
conda activate cugraph_dev
# to deactivate an environment
conda deactivate- The environment can be updated as development includes/changes the dependencies. To do so, run:
# Where XXX is the CUDA 11 version
conda env update --name cugraph_dev --file conda/environments/cugraph_dev_cuda11.XXX.yml
conda activate cugraph_devUsing the build.sh script make compiling and installing cuGraph a breeze. To build and install, simply do:
$ cd $CUGRAPH_HOME
$ ./build.sh clean
$ ./build.sh libcugraph
$ ./build.sh cugraphThere are several other options available on the build script for advanced users.
build.sh options:
build.sh [<target> ...] [<flag> ...]
where <target> is:
clean - remove all existing build artifacts and configuration (start over)
uninstall - uninstall libcugraph and cugraph from a prior build/install (see also -n)
libcugraph - build libcugraph.so and SG test binaries
libcugraph_etl - build libcugraph_etl.so and SG test binaries
cugraph - build the cugraph Python package
pylibcugraph - build the pylibcugraph Python package
cpp-mgtests - build libcugraph and libcugraph_etl MG tests. Builds MPI communicator, adding MPI as a dependency.
docs - build the docs
and <flag> is:
-v - verbose build mode
-g - build for debug
-n - do not install after a successful build
--allgpuarch - build for all supported GPU architectures
--buildfaiss - build faiss statically into cugraph
--show_depr_warn - show cmake deprecation warnings
--skip_cpp_tests - do not build the SG test binaries as part of the libcugraph and libcugraph_etl targets
-h - print this text
default action (no args) is to build and install 'libcugraph' then 'cugraph' then 'docs' targets
examples:
$ ./build.sh clean # remove prior build artifacts (start over)
$ ./build.sh libcugraph -v # compile and install libcugraph with verbose output
$ ./build.sh libcugraph -g # compile and install libcugraph for debug
$ ./build.sh libcugraph -n # compile libcugraph but do not install
# make parallelism options can also be defined: Example build jobs using 4 threads (make -j4)
$ PARALLEL_LEVEL=4 ./build.sh libcugraph
Note that the libraries will be installed to the location set in `$PREFIX` if set (i.e. `export PREFIX=/install/path`), otherwise to `$CONDA_PREFIX`.CMake depends on the nvcc executable being on your path or defined in $CUDACXX.
This project uses cmake for building the C/C++ library. To configure cmake, run:
# Set the localtion to cuGraph in an environment variable CUGRAPH_HOME
export CUGRAPH_HOME=$(pwd)/cugraph
cd $CUGRAPH_HOME
cd cpp # enter cpp directory
mkdir build # create build directory
cd build # enter the build directory
cmake .. -DCMAKE_INSTALL_PREFIX=$CONDA_PREFIX
# now build the code
make -j # "-j" starts multiple threads
make install # install the librariesThe default installation locations are $CMAKE_INSTALL_PREFIX/lib and $CMAKE_INSTALL_PREFIX/include/cugraph respectively.
libcugraph uses the RAFT library and there are times when it might be desirable to build against a different RAFT branch, such as when working on new features that might span both RAFT and cuGraph.
For local development, the CPM_raft_SOURCE=<path/to/raft/source> option can be passed to the cmake command to enable libcugraph to use the local RAFT branch.
To have CI test a cugraph pull request against a different RAFT branch, modify the bottom of the cpp/cmake/thirdparty/get_raft.cmake file as follows:
# Change pinned tag and fork here to test a commit in CI
# To use a different RAFT locally, set the CMake variable
# RPM_raft_SOURCE=/path/to/local/raft
find_and_configure_raft(VERSION ${CUGRAPH_MIN_VERSION_raft}
FORK <your_git_fork>
PINNED_TAG <your_git_branch_or_tag>
# When PINNED_TAG above doesn't match cugraph,
# force local raft clone in build directory
# even if it's already installed.
CLONE_ON_PIN ON
)When the above change is pushed to a pull request, the continuous integration servers will use the specified RAFT branch to run the cuGraph tests. After the changes in the RAFT branch are merged to the release branch, remember to revert the get_raft.cmake file back to the original cuGraph branch.
- Install the Python packages to your Python path:
cd $CUGRAPH_HOME
cd python
cd pylibcugraph
python setup.py build_ext --inplace
python setup.py install # install pylibcugraph
cd ../cugraph
python setup.py build_ext --inplace
python setup.py install # install cugraph python bindings
If you already have the datasets:
export RAPIDS_DATASET_ROOT_DIR=<path_to_ccp_test_and_reference_data>If you do not have the datasets:
cd $CUGRAPH_HOME/datasets
source get_test_data.sh #This takes about 10 minutes and downloads 1GB data (>5 GB uncompressed)Run either the C++ or the Python tests with datasets
-
Python tests with datasets
pip install python-louvain #some tests require this package to run cd $CUGRAPH_HOME cd python pytest
-
C++ stand alone tests
From the build directory :
# Run the cugraph tests cd $CUGRAPH_HOME cd cpp/build gtests/GDFGRAPH_TEST # this is an executable file
-
C++ tests with larger datasets
Run the C++ tests on large input:
cd $CUGRAPH_HOME/cpp/build #test one particular analytics (eg. pagerank) gtests/PAGERANK_TEST #test everything make test
Note: This conda installation only applies to Linux and Python versions 3.7/3.8.
You can do a local build and test on your machine that mimics our gpuCI environment using the ci/local/build.sh script.
For detailed information on usage of this script, see here.
It is possible to configure the conda environment to set environmental variables on activation. Providing instructions to set PATH to include the CUDA toolkit bin directory and LD_LIBRARY_PATH to include the CUDA lib64 directory will be helpful.
cd ~/anaconda3/envs/cugraph_dev
mkdir -p ./etc/conda/activate.d
mkdir -p ./etc/conda/deactivate.d
touch ./etc/conda/activate.d/env_vars.sh
touch ./etc/conda/deactivate.d/env_vars.shNext the env_vars.sh file needs to be edited
vi ./etc/conda/activate.d/env_vars.sh
#!/bin/bash
export PATH=/usr/local/cuda-11.0/bin:$PATH # or cuda-11.1 if using CUDA 11.1 and cuda-11.2 if using CUDA 11.2, respectively
export LD_LIBRARY_PATH=/usr/local/cuda-11.0/lib64:$LD_LIBRARY_PATH # or cuda-11.1 if using CUDA 11.1 and cuda-11.2 if using CUDA 11.2, respectivelyvi ./etc/conda/deactivate.d/env_vars.sh
#!/bin/bash
unset PATH
unset LD_LIBRARY_PATH
Python API documentation can be generated from docs directory.
Portions adopted from https://github.com/pytorch/pytorch/blob/master/CONTRIBUTING.md