Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide "--ccache" flag for build.sh #3566

Merged
merged 3 commits into from
Mar 9, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions BUILD.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ As a convenience, a `build.sh` script is provided which can be used to execute t
$ ./build.sh # build the cuML libraries, tests, and python package, then
# install them to $INSTALL_PREFIX if set, otherwise $CONDA_PREFIX
```
For workflows that involve frequent switching among branches or between debug and release builds, it is recommended that you install [ccache](https://ccache.dev/) and make use of it by passing the `--ccache` flag to `build.sh`.
wphicks marked this conversation as resolved.
Show resolved Hide resolved

To build individual components, specify them as arguments to `build.sh`
```bash
Expand All @@ -56,6 +57,7 @@ $ PARALLEL_LEVEL=4 ./build.sh libcuml # build and install libcuml limiting para
$ ./build.sh libcuml -n # build libcuml but do not install
$ ./build.sh prims --allgpuarch # build the ML prims tests for all supported GPU architectures
$ ./build.sh cuml --singlegpu # build the cuML python package without MNMG algorithms
$ ./build.sh --ccache # use ccache to cache compilations, speeding up subsequent builds
```

To run the C++ unit tests (optional), from the repo root:
Expand Down Expand Up @@ -135,6 +137,12 @@ Additionally, to reduce compile times, you can specify a GPU compute capability
$ cmake .. -DCMAKE_INSTALL_PREFIX=$CONDA_PREFIX -DGPU_ARCHS="70"
```

You may also wish to make use of `ccache` to reduce build times when switching among branches or between debug and release builds:

```bash
$ cmake .. -DUSE_CCACHE=ON
```

There are many options to configure the build process, see the [customizing build section](#libcuml-&-libcumlc++).

3. Build `libcuml++` and `libcuml`:
Expand Down
8 changes: 7 additions & 1 deletion build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ ARGS=$*
REPODIR=$(cd $(dirname $0); pwd)

VALIDTARGETS="clean libcuml cuml cpp-mgtests prims bench prims-bench cppdocs pydocs"
VALIDFLAGS="-v -g -n --allgpuarch --buildfaiss --buildgtest --singlegpu --nvtx --show_depr_warn --codecov -h --help "
VALIDFLAGS="-v -g -n --allgpuarch --buildfaiss --buildgtest --singlegpu --nvtx --show_depr_warn --codecov --ccache -h --help "
VALIDARGS="${VALIDTARGETS} ${VALIDFLAGS}"
HELP="$0 [<target> ...] [<flag> ...]
where <target> is:
Expand All @@ -45,6 +45,7 @@ HELP="$0 [<target> ...] [<flag> ...]
--show_depr_warn - show cmake deprecation warnings
--codecov - Enable code coverage support by compiling with Cython linetracing
and profiling enabled (WARNING: Impacts performance)
--ccache - Use ccache to cache previous compilations
-h - print this text

default action (no args) is to build and install 'libcuml', 'cuml', and 'prims' targets only for the detected GPU arch
Expand All @@ -69,6 +70,7 @@ BUILD_ALL_GPU_ARCH=0
SINGLEGPU_CPP_FLAG=""
CUML_EXTRA_PYTHON_ARGS=${CUML_EXTRA_PYTHON_ARGS:=""}
NVTX=OFF
CCACHE=OFF
CLEAN=0
BUILD_DISABLE_DEPRECATION_WARNING=ON
BUILD_CUML_STD_COMMS=ON
Expand Down Expand Up @@ -151,6 +153,9 @@ fi
if hasArg --codecov; then
CUML_EXTRA_PYTHON_ARGS="${CUML_EXTRA_PYTHON_ARGS} --linetrace=1 --profile"
fi
if hasArg --ccache; then
CCACHE=ON
fi
if hasArg clean; then
CLEAN=1
fi
Expand Down Expand Up @@ -199,6 +204,7 @@ if completeBuild || hasArg libcuml || hasArg prims || hasArg bench || hasArg pri
-DBUILD_STATIC_FAISS=${BUILD_STATIC_FAISS} \
-DNVTX=${NVTX} \
-DPARALLEL_LEVEL=${PARALLEL_LEVEL} \
-DUSE_CCACHE=${CCACHE} \
-DNCCL_PATH=${INSTALL_PREFIX} \
-DDISABLE_DEPRECATION_WARNING=${BUILD_DISABLE_DEPRECATION_WARNING} \
-DCMAKE_PREFIX_PATH=${INSTALL_PREFIX} \
Expand Down
8 changes: 8 additions & 0 deletions cpp/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,8 @@ option(NVTX "Enable nvtx markers" OFF)

option(SINGLEGPU "Disable all mnmg components and comms libraries" OFF)

option(USE_CCACHE "Cache build artifacts with ccache" OFF)

set(BUILD_RPATH_USE_ORIGIN "Whether to use relative paths for the build RPATH." TRUE)

set(PARALLEL_LEVEL "" CACHE STRING
Expand Down Expand Up @@ -161,6 +163,12 @@ if(BUILD_CUML_MG_TESTS AND NOT SINGLEGPU)
set(BUILD_CUML_MPI_COMMS ON)
endif(BUILD_CUML_MG_TESTS AND NOT SINGLEGPU)

if(USE_CCACHE)
set(CMAKE_C_COMPILER_LAUNCHER ccache)
set(CMAKE_CXX_COMPILER_LAUNCHER ccache)
set(CMAKE_CUDA_COMPILER_LAUNCHER ccache)
endif(USE_CCACHE)

##############################################################################
# - Requirements -------------------------------------------------------------

Expand Down