compile time issues with nvcc and GCC 10.3

[psychocoderHPC]

The CI is showing compile time issues with nvcc and GCC 10.3

/usr/include/c++/10/chrono: In substitution of 'template<class _Rep, class _Period> template<class _Period2> using __is_harmonic = std::__bool_constant<(std::ratio<((_Period2::num / std::chrono::duration<_Rep, _Period>::_S_gcd(_Period2::num, _Period::num)) * (_Period::den / std::chrono::duration<_Rep, _Period>::_S_gcd(_Period2::den, _Period::den))), ((_Period2::den / std::chrono::duration<_Rep, _Period>::_S_gcd(_Period2::den, _Period::den)) * (_Period::num / std::chrono::duration<_Rep, _Period>::_S_gcd(_Period2::num, _Period::num)))>::den == 1)> [with _Period2 = _Period2; _Rep = _Rep; _Period = _Period]':
/usr/include/c++/10/chrono:473:154:   required from here
/usr/include/c++/10/chrono:428:27: internal compiler error: Segmentation fault
  428 |  _S_gcd(intmax_t __m, intmax_t __n) noexcept
      |                           ^~~~~~
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-10/README.Bugs> for instructions.

see:

• https://github.com/alpaka-group/alpaka/pull/1294/checks?check_run_id=2384249915#step:5:3859
• https://github.com/alpaka-group/alpaka/pull/1294/checks?check_run_id=2384249448#step:5:3876

A similar issue was reported:

• nccl errors after moving to gcc 10.3.0 NVIDIA/nccl#494
• https://www.mail-archive.com/[email protected]&q=subject:%22ICE%22&o=newest&f=1

GCC bug reports:

#1297 (comment)

I am not sure but the error could be fixed with https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100101

ATTENTION Currently two tests in the CI are disabled: #1302

[SimeonEhrig]

CUDA 11.3 does only supports up to GCC 9.x: https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html
I had also some problems with GCC 10 and CUDA.

[psychocoderHPC]

CUDA 11.3 does only supports up to GCC 9.x: https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html
I had also some problems with GCC 10 and CUDA.

I checked CUDA 11.1 include file crt/host_config.h, GCC 10 is supported

#if __GNUC__ > 10

#error -- unsupported GNU version! gcc versions later than 10 are not supported! The nvcc flag '-allow-unsupported-compiler' can be used to override this version check; however, using an unsupported host compiler may cause compilation failure or incorrect run time execution. Use at your own risk.

#endif /* __GNUC__ > 10 */

[psychocoderHPC]

@alpaka-group/alpaka-maintainers
I was not able to work around the GCC bug.

I disabled for now the requirement to pass the CI tests for:

• linux_nvcc-11.1_gcc-10_debug
• linux_nvcc-11.2_gcc-10_debug

This avoids that other developments get blocked due to a compiler bug.

[psychocoderHPC]

Unluckily it is not enough to mark the tests as not required (you need still admin rights to merge the PR), therefore the combination nvcc+GCC10.3 is currently disabled. #1302 :-(

[edrozenberg]

CUDA 11.3 does only supports up to GCC 9.x: https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html
I had also some problems with GCC 10 and CUDA.

Nvidia's install guide is not updated properly and who knows when it will be.

GCC 10 support was mentioned in a few earlier CUDA 11 release notes https://developer.nvidia.com/cuda-toolkit-archive such as
https://docs.nvidia.com/cuda/archive/11.1.1/cuda-toolkit-release-notes/index.html

The following compilers are supported as host compilers in nvcc:
GCC 10.0
Clang 10.0

I've had no problems with latest CUDA and GCC 10 until GCC 10.3.0.

[psychocoderHPC]

CUDA 11.3 does only supports up to GCC 9.x: https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html
I had also some problems with GCC 10 and CUDA.

Nvidia's install guide is not updated properly and who knows when it will be.

GCC 10 support was mentioned in a few earlier CUDA 11 release notes https://developer.nvidia.com/cuda-toolkit-archive such as
https://docs.nvidia.com/cuda/archive/11.1.1/cuda-toolkit-release-notes/index.html

The following compilers are supported as host compilers in nvcc:
GCC 10.0
Clang 10.0

I've had no problems with latest CUDA and GCC 10 until GCC 10.3.0.

Yes GCC 10 works fine, except for the mentioned issue with 10.3. Do not trust documentation, only trust code^^

[psychocoderHPC]

I reopened this issue, the issue is still not solved, only handled by a workaround.

[bernhardmgruber]

The same error occurs with GCC 11.0.1, libstdc++ 10.3 and CUDA 11.2, which is weird because nvcc causes GCC 11 to not use libstdc++ 11, but version 10. If I compile a non-CUDA file, the right libstdc++ is selected.

[psychocoderHPC]

The same error occurs with GCC 11.0.1, libstdc++ 10.3 and CUDA 11.2, which is weird because nvcc causes GCC 11 to not use libstdc++ 11, but version 10. If I compile a non-CUDA file, the right libstdc++ is selected.

https://gitlab.kitware.com/cmake/cmake/-/issues/19458 sounds a little bit like your issue. Do you have the issue with alpaka or with a tiny reporducer and command line nvcc? In the linked issue the mentioned giving the host compiler explicit to nvcc solves the issue.

[edrozenberg]

@psychocoderHPC I'm not an Alpaka user - that Gitlab issue didn't solve my problems with the Nvidia stack, for ex. trying to build https://github.com/NVIDIA/nccl-tests

Relevant gcc bug reports. Don't know how quickly gcc issues get fixed, so for users on gcc 10.3.0 the Nvidia CUDA stack is essentially unusable for now for some indefinite amount of time.

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100102
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100240

[psychocoderHPC]

@edrozenberg Thanks for the clarification 👍 and the links to the corresponding gcc bug reports.

[edrozenberg]

The GCC project has committed a patch:

https://gcc.gnu.org/git/gitweb.cgi?p=gcc.git;h=5357ab75dedef403b0eebf9277d61d1cbeb5898f
(in response to the problem report https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100102)

The patch removes ~ 40 lines from gcc/cp/pt.c and adds a couple of test cases.

The Nvidia stack works again for me after I built and installed patched gcc 10.3.0 packages.

[j-stephan]

Almost a year later there is still no sight of GCC 10.4 :-(

[j-stephan]

gcc 10.4 has been released. It includes the above-mentioned bugfix.

[fwyzard]

👍