Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check for __CUDA_ARCH_LIST__ for setting GT_CUDA_ARCH #1776

Closed
wants to merge 1 commit into from

Conversation

iomaganaris
Copy link
Contributor

@iomaganaris iomaganaris commented Apr 15, 2024

For some reason __CUDA_ARCH__ isn't set for the compilation of all intermediate files generated by nvcc. Meanwhile __CUDA_ARCH_LIST__ is.
This generates compilation errors for projects with GridTools (and more specifically ones that include storage/gpu.hpp) since code that needs to be generated for the GPU that is under a GT_CUDA_ARCH #ifdef isn't compiled.
Behavior was tested using CUDA 12.3 in various systems. Below is an example of the nvcc compilation command with the --verbose flag.

#$ gcc -std=c++20 -D__CUDA_ARCH__=600 -D__CUDA_ARCH_LIST__=600 -E -x c++  -DCUDA_DOUBLE_MATH_FUNCTIONS -D__CUDACC__ -D__NVCC__  -I"/scratch/snx3000/ioannmag/cycle21/icon_structured_benchmark/include" -I"/scratch/snx3000/ioannmag/cycle21/icon_structured_benchmark/GridTools" -I"/scratch/snx3000/ioannmag/cycle21/icon_structured_benchmark/build_gpu/_deps/gridtools-src/include" "-I/scratch/snx3000/ioannmag/spack/opt/spack/cray-cnl7-haswell/intel-2022.1.0/cuda-12.3.0-wagarpvzjr6mefjj42ym5343h3dketrg/bin/../targets/x86_64-linux/include"   -isystem "/scratch/snx3000/ioannmag/spack/opt/spack/cray-cnl7-haswell/intel-2022.1.0/boost-1.84.0-uw2w34r3fsrqlahxynev3rmv4oi55lq7/include" -isystem "/scratch/snx3000/ioannmag/spack/opt/spack/cray-cnl7-haswell/intel-2022.1.0/cuda-12.3.0-wagarpvzjr6mefjj42ym5343h3dketrg/targets/x86_64-linux/include"  -D "BOOST_PP_VARIADICS=1" -D__CUDACC_VER_MAJOR__=12 -D__CUDACC_VER_MINOR__=3 -D__CUDACC_VER_BUILD__=52 -D__CUDA_API_VER_MAJOR__=12 -D__CUDA_API_VER_MINOR__=3 -D__NVCC_DIAG_PRAGMA_SUPPORT__=1 -include "cuda_runtime.h" -m64 "/scratch/snx3000/ioannmag/cycle21/icon_structured_benchmark/src/wrapper.cpp" -o "wrapper.cpp1.ii"
#$ gcc -std=c++20 -D__CUDA_ARCH_LIST__=600 -E -x c++ -D__CUDACC__ -D__NVCC__  -I"/scratch/snx3000/ioannmag/cycle21/icon_structured_benchmark/include" -I"/scratch/snx3000/ioannmag/cycle21/icon_structured_benchmark/GridTools" -I"/scratch/snx3000/ioannmag/cycle21/icon_structured_benchmark/build_gpu/_deps/gridtools-src/include" "-I/scratch/snx3000/ioannmag/spack/opt/spack/cray-cnl7-haswell/intel-2022.1.0/cuda-12.3.0-wagarpvzjr6mefjj42ym5343h3dketrg/bin/../targets/x86_64-linux/include"   -isystem "/scratch/snx3000/ioannmag/spack/opt/spack/cray-cnl7-haswell/intel-2022.1.0/boost-1.84.0-uw2w34r3fsrqlahxynev3rmv4oi55lq7/include" -isystem "/scratch/snx3000/ioannmag/spack/opt/spack/cray-cnl7-haswell/intel-2022.1.0/cuda-12.3.0-wagarpvzjr6mefjj42ym5343h3dketrg/targets/x86_64-linux/include"  -D "BOOST_PP_VARIADICS=1" -D__CUDACC_VER_MAJOR__=12 -D__CUDACC_VER_MINOR__=3 -D__CUDACC_VER_BUILD__=52 -D__CUDA_API_VER_MAJOR__=12 -D__CUDA_API_VER_MINOR__=3 -D__NVCC_DIAG_PRAGMA_SUPPORT__=1 -include "cuda_runtime.h" -m64 "/scratch/snx3000/ioannmag/cycle21/icon_structured_benchmark/src/wrapper.cpp" -o "wrapper.cpp4.ii"
#$ -- Filter Dependencies -- > CMakeFiles/benchmark.dir/src/wrapper.cpp.o.d

For the use of gridtools::storage::gpu in the above example see https://github.com/GridTools/icon_structured_benchmark/blob/ioannmag/gpu/CMakeLists.txt, https://github.com/GridTools/icon_structured_benchmark/blob/ioannmag/gpu/include/nabla4_gridtools.hpp and https://github.com/GridTools/icon_structured_benchmark/blob/ioannmag/gpu/include/nabla4_structured_torus_gridtools_halo.hpp

@gridtoolsjenkins
Copy link
Collaborator

Hi there, this is jenkins continuous integration...
Do you want me to verify this patch?

@iomaganaris iomaganaris changed the title Check for __CUDA_ARCH_LIST__ for setting GT_CUDA_ARCH as well Check for __CUDA_ARCH_LIST__ for setting GT_CUDA_ARCH Apr 16, 2024
@havogt
Copy link
Contributor

havogt commented May 30, 2024

replaced by #1778

@havogt havogt closed this May 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants