Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[REVIEW] ENH Allow arbitrary CMake config options in build.sh #8996

Merged
merged 15 commits into from
Aug 13, 2021
Merged
Changes from 10 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 48 additions & 24 deletions build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -18,26 +18,27 @@ ARGS=$*
REPODIR=$(cd $(dirname $0); pwd)

VALIDARGS="clean libcudf cudf dask_cudf benchmarks tests libcudf_kafka cudf_kafka custreamz -v -g -n -l --allgpuarch --disable_nvtx --show_depr_warn --ptds -h"
HELP="$0 [clean] [libcudf] [cudf] [dask_cudf] [benchmarks] [tests] [libcudf_kafka] [cudf_kafka] [custreamz] [-v] [-g] [-n] [-h] [-l]
clean - remove all existing build artifacts and configuration (start
over)
libcudf - build the cudf C++ code only
cudf - build the cudf Python package
dask_cudf - build the dask_cudf Python package
benchmarks - build benchmarks
tests - build tests
libcudf_kafka - build the libcudf_kafka C++ code only
cudf_kafka - build the cudf_kafka Python package
custreamz - build the custreamz Python package
-v - verbose build mode
-g - build for debug
-n - no install step
-l - build legacy tests
--allgpuarch - build for all supported GPU architectures
--disable_nvtx - disable inserting NVTX profiling ranges
--show_depr_warn - show cmake deprecation warnings
--ptds - enable per-thread default stream
-h | --h[elp] - print this text
HELP="$0 [clean] [libcudf] [cudf] [dask_cudf] [benchmarks] [tests] [libcudf_kafka] [cudf_kafka] [custreamz] [-v] [-g] [-n] [-h] [-l] [--cmake-args=\"<args>\"]
clean - remove all existing build artifacts and configuration (start
over)
libcudf - build the cudf C++ code only
cudf - build the cudf Python package
dask_cudf - build the dask_cudf Python package
benchmarks - build benchmarks
tests - build tests
libcudf_kafka - build the libcudf_kafka C++ code only
cudf_kafka - build the cudf_kafka Python package
custreamz - build the custreamz Python package
-v - verbose build mode
-g - build for debug
-n - no install step
-l - build legacy tests
--allgpuarch - build for all supported GPU architectures
--disable_nvtx - disable inserting NVTX profiling ranges
--show_depr_warn - show cmake deprecation warnings
--ptds - enable per-thread default stream
--cmake-args=\\\"<args>\\\" - pass arbitrary list of CMake configuration options (escape all quotes in argument)
-h | --h[elp] - print this text

default action (no args) is to build and install 'libcudf' then 'cudf'
then 'dask_cudf' targets
Expand Down Expand Up @@ -71,6 +72,26 @@ function hasArg {
(( ${NUMARGS} != 0 )) && (echo " ${ARGS} " | grep -q " $1 ")
}

function cmakeArgs {
dillon-cullinan marked this conversation as resolved.
Show resolved Hide resolved
# Check for correctly formatted cmake args option
if [[ -n $(echo $ARGS | grep -E "\-\-cmake\-args=\"") ]]; then
# There are possible weird edge cases that may cause this regex filter to output nothing and fail silently
# the true pipe will catch any weird edge cases that may happen and will cause the program to fall back
# on the invalid option error
CMAKE_ARGS=$(echo $ARGS | { grep -Eo "\-\-cmake\-args=\".+\"" || true; })
if [[ -n ${CMAKE_ARGS} ]]; then
# Remove the full CMAKE_ARGS argument from list of args so that it passes validArgs function
ARGS=${ARGS//$CMAKE_ARGS/}
# Filter the full argument down to just the extra string that will be added to cmake call
CMAKE_ARGS=$(echo $CMAKE_ARGS | grep -Eo "\".+\"" | sed -e 's/^"//' -e 's/"$//')
dillon-cullinan marked this conversation as resolved.
Show resolved Hide resolved
fi
elif [[ -z $(echo $ARGS | grep -E "\-\-cmake\-args=") ]]; then
CMAKE_ARGS="$(echo $ARGS | grep -Eo "\-\-cmake\-args=(.+)? ")"
echo "Invalid formatting for --cmake-args, see --help: $CMAKE_ARGS"
exit 1
fi
}

function buildAll {
((${NUMARGS} == 0 )) || !(echo " ${ARGS} " | grep -q " [^-]\+ ")
}
Expand All @@ -82,9 +103,11 @@ fi

# Check for valid usage
if (( ${NUMARGS} != 0 )); then
# Check for cmake args
cmakeArgs
for a in ${ARGS}; do
if ! (echo " ${VALIDARGS} " | grep -q " ${a} "); then
echo "Invalid option: ${a}"
echo "Invalid option, check --help: ${a}"
exit 1
fi
done
Expand Down Expand Up @@ -139,7 +162,6 @@ fi
# Configure, build, and install libcudf

if buildAll || hasArg libcudf; then

if (( ${BUILD_ALL_GPU_ARCH} == 0 )); then
CUDF_CMAKE_CUDA_ARCHITECTURES="-DCMAKE_CUDA_ARCHITECTURES="
echo "Building for the architecture of the GPU in the system..."
Expand All @@ -156,7 +178,8 @@ if buildAll || hasArg libcudf; then
-DBUILD_BENCHMARKS=${BUILD_BENCHMARKS} \
-DDISABLE_DEPRECATION_WARNING=${BUILD_DISABLE_DEPRECATION_WARNING} \
-DPER_THREAD_DEFAULT_STREAM=${BUILD_PER_THREAD_DEFAULT_STREAM} \
-DCMAKE_BUILD_TYPE=${BUILD_TYPE}
-DCMAKE_BUILD_TYPE=${BUILD_TYPE} \
${CMAKE_ARGS}
dillon-cullinan marked this conversation as resolved.
Show resolved Hide resolved

cd ${LIB_BUILD_DIR}

Expand Down Expand Up @@ -196,7 +219,8 @@ if hasArg libcudf_kafka; then
cmake -S $REPODIR/cpp/libcudf_kafka -B ${KAFKA_LIB_BUILD_DIR} \
-DCMAKE_INSTALL_PREFIX=${INSTALL_PREFIX} \
-DBUILD_TESTS=${BUILD_TESTS} \
-DCMAKE_BUILD_TYPE=${BUILD_TYPE}
-DCMAKE_BUILD_TYPE=${BUILD_TYPE} \
${CMAKE_ARGS}
robertmaynard marked this conversation as resolved.
Show resolved Hide resolved


cd ${KAFKA_LIB_BUILD_DIR}
Expand Down