Deprecation of CentOS 7 support

[jakirkham]

As of CUDA 12.4.0, CentOS 7 support is deprecated as noted in the CUDA 12.4.0 release notes. Have copied the relevant snippet below:

1.4.2. Deprecated Operating Systems

CUDA Toolkit 12.4 deprecates support for Red Hat Enterprise Linux 7 and CentOS 7. Support for these operating systems will be removed in an upcoming release.

[kmittman]

Officially CUDA 12.5.0 and newer drop support for EL7; the NVIDIA 555 driver branch does not support the EL7 3.10 kernel, the minimum version changed to 4.15 kernel.

[jakirkham]

Thanks Kevin! 🙏

As we don't ship the driver in Conda, users are on their own to get it, which means they will encounter any errors installing or running the driver via the usual mechanisms

Think what is of most interest to conda-forge is GLIBC compatibility of binaries in these packages. Currently that has been GLIBC 2.17 (or older), which is consistent with EL7 compatibility. Having some insight on when this will change would be helpful for planning

Happy to discuss this more offline

[jakirkham]

To prepare for AlmaLinux 8 images, there are a couple CDTs we are using that are not available there. We have the following PRs to clean these up. They may need a bit of help to get across the line

• Use rdma-core package (instead of CDT) & add to linux_aarch64 libcufile-feedstock#14
• Replace CDTs with Conda packages & cleanup yum_requirements.txt nsight-compute-feedstock#32

Not seeing any other CDT usage in the CTK packages, but if I missed anything please add them to this issue so we can track them

[jakirkham]

To upgrade to Alma 8, we will need to update feedstocks' conda-forge.ymls to include os_version for the linux_* versions supported with their current OS version (like cos7 in many cases). This can later be updated to alma8

The CTK feedstocks had their conda-forge.yml's updated to specify os_version as part of issue: #26

However did find a few non-CTK feedstocks that NVIDIA maintains that came in after {{ stdlib("c") }} / os_version work, which currently don't specify conda-forge.yml (or underspecify it). Have filed issues for those feedstocks below. If more come up, we can update this list as needed

• Extending conda-forge.yml's os_version for other linux_*s cudnn-feedstock#93
• Adding os_version to conda-forge.yml libcal-feedstock#6
• Adding os_version to conda-forge.yml libcublasmp-feedstock#5
• Adding os_version to conda-forge.yml libnvpl-fft-feedstock#2
• Adding os_version to conda-forge.yml libnvpl-lapack-feedstock#2
• Adding os_version to conda-forge.yml libnvpl-rand-feedstock#2
• Adding os_version to conda-forge.yml libnvpl-scalapack-feedstock#2
• Adding os_version to conda-forge.yml libnvpl-sparse-feedstock#2
• Adding os_version to conda-forge.yml libnvpl-tensor-feedstock#2

[h-vetinari]

Much appreciated John! Please note that in the case of CUDA 11.8 images, any specification for linux_aarch64 / linux_ppc64le is a no-op (and will remain so), because we do not have a choice there in terms of images -- only ubi8-based images exist for aarch/ppc. In other words, just

os_version:
  linux_64: cos7
  # this space intentionally left blank

would be enough for 11.8 feedstocks; However, since most feedstocks build CUDA 12.x as well (or even only that), it would be consistent to use

os_version:
  linux_64: cos7
  linux_aarch64: cos7
  linux_ppc64le: cos7

That however will run into issues as some pieces of the CUDA infrastructure on aarch already rely on newer glibc symbols. If you use this approach, you will probably see CI breakage once conda-forge/conda-forge-pinning-feedstock#6626 lands; at that point, you can change to

os_version:
  linux_64: cos7
  linux_aarch64: alma8
  linux_ppc64le: alma8

and things should work fine again.

[carterbox]

All of the nvpl feedstocks are already using alma8 through direct specification of the docker image, so they cannot be transitioned until after the alma8 option for os_version is available. Example below:

https://github.com/conda-forge/libnvpl-tensor-feedstock/pull/3/files

[h-vetinari]

so they cannot be transitioned until after the alma8 option for os_version is available. Example below:

They cannot be transitioned (in the sense of removing docker_image from CBC), but it's possible already to add os_version: ... to conda-forge.yml, as that's currently a no-op.

[jakirkham]

Next step here is to add testing across CUDA packages for minimum GLIBC version used

This will help us verify when packages are compatible and help us correct any mismatches between metadata and the binaries themselves

@carterbox and @mtjrider are working on a testing script

[h-vetinari]

Next step here is to add testing across CUDA packages for minimum GLIBC version used

Here's some inspiration for a script that does just that: conda-forge/libcufile-feedstock#24

[jakirkham]

Thanks Axel! 🙏

IIUC Daniel and Matt's work is in PR: conda-forge/cudnn-feedstock#97

If you have a moment to look that over, it would be great to incorporate your feedback into that script