[Backport release-23.11] cuda-modules #272784

ConnorBaker · 2023-12-07T22:51:15Z

Important

This PR must be rebased to target staging-23.11 after the following PRs are merged:

Backport 271078 to staging 23.11 #273040

Important

This PR is an amalgamation of changes and subsequent which landed in master as part of the work surrounding #256324. This PR includes:

PRs to be included once merged:

Description of changes

Things done

Backport of #256324.

Add a 👍 reaction to pull requests you find important.

(cherry picked from commit 4a25023)

(cherry picked from commit 397d95d)

cudaPackages.cuda_compat: ignore missing libs provided at runtime cudaPackages.gpus: Jetson should never build by default cudaPackages.flags: don't build Jetson capabilities by default cudaPackages: re-introduce filter for pre-existing CUDA redist packages in overrides cudaPackages: only recurseIntoAttrs for the latest of each major version cudaPackages.nvccCompatabilities: use GCC 10 through CUDA 11.5 to avoid a GLIBC incompatability cudaPackages.cutensor: acquire libcublas through cudatoolkit prior to 11.4 cudaPackages.cuda_compat: mark as broken on aarch64-linux if not targeting Jetson cudaPackages.cutensor_1_4: fix build cudaPackages: adjust use of autoPatchelfIgnoreMissingDeps cudaPackages.cuda_nvprof: remove unecessary override to add addOpenGLRunpath cudaPackages: use getExe' to avoid patchelf warning about missing meta.mainProgram cudaPackages: fix evaluation with Nix 2.3 cudaPackages: fix platform detection for Jetson/non-Jetson aarch64-linux python3Packages.tensorrt: mark as broken if required packages are missing Note: evaluating the name of the derivation will fail if tensorrt is not present, which is why we wrap the value in `lib.optionalString`. cudaPackages.flags.getNixSystem: add guard based on jetsonTargets cudaPackages.cudnn: use explicit path to patchelf cudaPackages.tensorrt: use explicit path to patchelf (cherry picked from commit 8e800ce)

(cherry picked from commit bfaefd0)

(cherry picked from commit 0a7dacf)

(cherry picked from commit aaf735e)

…helf (cherry picked from commit 6179d88)

After testing on a Jetson device, it turns out `cuda_compat` requires libnvdla_runtime.so which can't be satisfied by autoPatchElf, as it is provided by the runtime driver. This commit simply adds this library to the list of dependency to be ignored by autoPatchElf. (cherry picked from commit a3ac436)

RaitoBezarius · 2023-12-12T15:09:37Z

I'm not sure I understand whether this fits the backport policy, what is the rationale here?

SomeoneSerge · 2023-12-12T15:17:42Z

Before this gets into a release, the following need to be addressed:

cudaPackages.cuda_cuobjdump: $out is empty #273321
libcutensor build failure on x86_64-linux as of 3df29656 #273674 (libcutensor build failure on x86_64-linux as of 402ae9be #272797)
Possibly nsight_systems, not sure about the source yet

(cherry picked from commit a39043f)

(cherry picked from commit 26dd975)

Some nvidia devices, such as the Jetson family, support the Nvidia compatibility package (nvidia_compat) which allows to run executables built against a higher CUDA major version on a system with an older CUDA driver. On such platforms, the consensus among CUDA maintainers is that there is no downside in always enabling it by default. This commit links to the relevant cuda_compat shared libraries by patching the CUDA core packages' runpaths when cuda_compat is available, in the same way as we do for OpenGL drivers currently. (cherry picked from commit d6c198a)

jonringer · 2023-12-12T20:28:24Z

I'm not sure I understand whether this fits the backport policy, what is the rationale here?

It does if there's no regressions. A lot of this work is about improving Nixpkgs + Cuda support, which can be viewed as an additive feature. It should be tested well to ensure that it doesn't introduce any new failures.

jonringer · 2023-12-12T20:29:15Z

cc @NixOS/cuda-maintainers , who would have a good idea of potential regressions

(cherry picked from commit f13c797)

...to prevent the evaluation of cuda_compat (cherry picked from commit 6256391)

(cherry picked from commit 076fc3a)

(cherry picked from commit 7ce9921)

(cherry picked from commit 51c3e87)

ConnorBaker · 2023-12-13T04:16:34Z

@jonringer excellent point about regression tests -- doubly so for runtime behavior. We don't have anything in the way of tracking performance over time... we're still working on making sure we have enough builders to keep our cache somewhat populated, let alone automated testing or regression detection.

The only runtime tests I have are manual: cudaPackages.saxpy, which is really just meant to check that the driver works, and nix-cuda-test (https://github.com/ConnorBaker/nix-cuda-test), which has a dinky little ViT model.

@graham33 would you happen to have anything you'd be able to use for regression testing on the 23.11 branch? You're one of the handful of people I know using Nix+CUDA and I'd like to make sure this doesn't cause fires or headaches :)

gbpdt · 2023-12-13T09:56:43Z

@graham33 would you happen to have anything you'd be able to use for regression testing on the 23.11 branch? You're one of the handful of people I know using Nix+CUDA and I'd like to make sure this doesn't cause fires or headaches :)

(Switching to my other Github ID)

We've started our internal upgrade process to adopt 23.11, which would at least allow us to do some basic testing. We could definitely try to build against your branch and make sure we can get some basic CUDA validation working.

This resolves crashes in nsys-ui (cherry picked from commit a5b8caa)

SomeoneSerge · 2023-12-13T15:53:02Z

I mentioned this in the matrix chat, but I'll reiterate here: I don't see a particular rush for backporting these changes to 23.11, unless there's a specific user/customer interested in the new functionality (meaning mostly jetsons) but otherwise limited to using the release branches. I haven't any objections either

(cherry picked from commit 85bcd8c)

IlyaNiklyaev · 2024-02-26T14:24:25Z

Hi 👋
Our team is interested in this backport to land stable channel 🙂 Yes, we target both jetsons and x86 and we're not ready to switch to unstable.
At the moment we have this PR as a patch over nixos-23.11, and it works just fine as we can see (for both x86 and jetson). The only issue is cross-build (buildPlatform x86_64, hostPlatform - aarch64, both linux) : I guess we need some additional fixes from master, right now I'm trying to figure out the right set of commits.

to @ConnorBaker we can definitely help with testing if you have some specific tests to check before this PR can be merged.

SomeoneSerge · 2024-03-01T08:46:14Z

Hi @IlyaNiklyaev,

I think backporting new features and breaking interface changes is far from the optimal way to go here:

it's two months until the next release;
the PR had (as anticipated) caused a bit of fall out on the unstable which we've been slowly addressing, that it won't be clear what else to backport;
cross-compilation support actually requires more work anyway (I think @ConnorBaker is iterating on that rn in https://github.com/ConnorBaker/cuda-modules).

I too would like to see a model more like rolling releases with merge trains, but right now it's easier to accept that "new features" take up to 6 months to "stabilize". Wdyt?

IlyaNiklyaev · 2024-03-07T16:38:25Z

Hi @SomeoneSerge,
I think we can't wait until the next release, but I totally understand that there is a lot of moving around this functionality in the unstable which prevents us from just backporting all of it. So I'll continue to fix cross-build in our current state (stable 23.11 + the patch from this PR). I think a couple of our fixes would be useful in unstable too :)

SomeoneSerge · 2024-03-07T17:46:57Z

@IlyaNiklyaev I wonder, do you actually "patch" anything, or is it enough to use overlays? E.g.

inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-23.11";
inptus.nixpkgs-unstable.url = "...";
outputs = { nixpkgs, nixpkgs-unstable, ... }: {
  packages.x86_64-linux = let pkgs = import nixpkgs { ...; overlays = [(final: prev: {
    cudaPackages = final.callPackage "${nixpkgs-unstable}/pkgs/top-level/cuda-packages.nix" { };
    cudaPackages_XX_Y = ...;
    # ...
  })];
};

...might suffice to use the current cudaPackages expressions with the older nixpkgs release

IlyaNiklyaev · 2024-03-11T12:38:41Z

@SomeoneSerge hmm I even haven't tried that. I guess this is a tree-wide change, so it would be much more difficult to apply it as an overlay?
For example, I just tried what you suggested, and got this

evaluation aborted with the following error message: 'Function called without required argument "addDriverRunpath" at /nix/store/zafmv50n5r12j0v3cpazsmj2s7v8f5wc-source/pkgs/development/cuda-modules/cuda/overrides.nix:1'

So nixos-23.11 is not compatible with this cfa0701. Ofc I could bisect and try to find some point in unstable's history that will work but it's just much simpler to take this PR as a patch for now.

SomeoneSerge · 2024-03-11T15:27:43Z

I believe that most of the changes were actually contained to the cudaPackages* attributes and to the respective file paths... OOC, I tested cudaPackages.saxpy here and it worked: https://github.com/SomeoneSerge/nixos-2311-unstable-cuda/blob/master/mk-overlay.nix

IlyaNiklyaev · 2024-03-12T08:56:37Z

@SomeoneSerge wow that really works :) Thanks, I'll try to follow this approach. It doesn't help much with cross-build though but it's not directly related to this backport I suppose.

wegank · 2024-06-30T20:42:06Z

As NixOS 23.11 reaches end-of-life today, this PR is closed as part of the EOL cleanup.

ConnorBaker added the 6.topic: cuda Parallel computing platform and API label Dec 7, 2023

ConnorBaker self-assigned this Dec 7, 2023

github-actions bot added 6.topic: python 8.has: documentation This PR adds or changes documentation labels Dec 7, 2023

ConnorBaker mentioned this pull request Dec 7, 2023

cudaPackages: support multiple platforms #256324

Merged

13 tasks

ofborg bot added 8.has: clean-up 8.has: package (new) This PR adds a new package 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 101-500 labels Dec 8, 2023

SomeoneSerge mentioned this pull request Dec 11, 2023

Backport 271078 to staging 23.11 #273040

Merged

13 tasks

ConnorBaker changed the title ~~Backport 256324 to release 23.11~~ [Backport release-23.11] #256324 Dec 12, 2023

ConnorBaker marked this pull request as draft December 12, 2023 14:27

Connor Baker added 7 commits December 12, 2023 14:35

cudaPackages: regen & move manifests to cuda-modules

2b3b947

(cherry picked from commit 4a25023)

cudaPackages: move config expressions to cuda-modules

f3695c6

(cherry picked from commit 397d95d)

cudaPackages: add docs

70209ff

(cherry picked from commit bfaefd0)

cudaPackages_12_3: init at 12.3.0

0f376a9

(cherry picked from commit 0a7dacf)

cudaPackages.saxpy: only available from CUDA 11.4 and on

65458e5

(cherry picked from commit aaf735e)

cuda-modules: tidy generic-builder/manifest installPhase and postPatc…

6c0c6a0

…helf (cherry picked from commit 6179d88)

ConnorBaker changed the title ~~[Backport release-23.11] #256324~~ [Backport release-23.11] cuda-modules Dec 12, 2023

ConnorBaker force-pushed the backport-256324-to-release-23.11 branch from c5a131e to d883056 Compare December 12, 2023 14:40

ConnorBaker mentioned this pull request Dec 12, 2023

[Backport release-23.11] #273748 #273766

Closed

yannham added 3 commits December 12, 2023 15:34

Apply markdown lints to CUDA READMEs

4c91b2c

(cherry picked from commit a39043f)

Add basic documentation on cuda_compat

7e6a091

(cherry picked from commit 26dd975)

Connor Baker and others added 5 commits December 13, 2023 01:12

cudaPackages: prefix libPath with lib in manifest builder

2cb8686

(cherry picked from commit f13c797)

cudaPackages.autoAddCudaCompatRunpathHook: broken on non-jetsons

358d328

...to prevent the evaluation of cuda_compat (cherry picked from commit 6256391)

cudaPackages.autoAddCudaCompatRunpathHook: dummy libcudaPath when broken

616a21d

(cherry picked from commit 076fc3a)

cudaPackages.cudnn: correct hash for CUDA 10.2, v7.6.5.32 release

7466c27

(cherry picked from commit 7ce9921)

cudaPackages.cudnn: 8.9.6.50 -> 8.9.7.29

8a52a1a

(cherry picked from commit 51c3e87)

cudatoolkit: Replace vendored Qt plugins with symlinks

e90f5c9

This resolves crashes in nsys-ui (cherry picked from commit a5b8caa)

ConnorBaker mentioned this pull request Dec 13, 2023

cudaPackages: add missing newline to manifest builder #273873

Merged

13 tasks

cudaPackages: add missing newline to manifest builder

0f6bf29

(cherry picked from commit 85bcd8c)

wegank added the 2.status: merge conflict This PR has merge conflicts with the target branch label Mar 20, 2024

wegank closed this Jun 30, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Backport release-23.11] cuda-modules #272784

[Backport release-23.11] cuda-modules #272784

ConnorBaker commented Dec 7, 2023 •

edited

Loading

RaitoBezarius commented Dec 12, 2023

SomeoneSerge commented Dec 12, 2023 •

edited by ConnorBaker

Loading

jonringer commented Dec 12, 2023

jonringer commented Dec 12, 2023

ConnorBaker commented Dec 13, 2023

gbpdt commented Dec 13, 2023

SomeoneSerge commented Dec 13, 2023

IlyaNiklyaev commented Feb 26, 2024 •

edited

Loading

SomeoneSerge commented Mar 1, 2024

IlyaNiklyaev commented Mar 7, 2024 •

edited

Loading

SomeoneSerge commented Mar 7, 2024

IlyaNiklyaev commented Mar 11, 2024 •

edited

Loading

SomeoneSerge commented Mar 11, 2024

IlyaNiklyaev commented Mar 12, 2024

wegank commented Jun 30, 2024

[Backport release-23.11] cuda-modules #272784

[Backport release-23.11] cuda-modules #272784

Conversation

ConnorBaker commented Dec 7, 2023 • edited Loading

Description of changes

Things done

RaitoBezarius commented Dec 12, 2023

SomeoneSerge commented Dec 12, 2023 • edited by ConnorBaker Loading

jonringer commented Dec 12, 2023

jonringer commented Dec 12, 2023

ConnorBaker commented Dec 13, 2023

gbpdt commented Dec 13, 2023

SomeoneSerge commented Dec 13, 2023

IlyaNiklyaev commented Feb 26, 2024 • edited Loading

SomeoneSerge commented Mar 1, 2024

IlyaNiklyaev commented Mar 7, 2024 • edited Loading

SomeoneSerge commented Mar 7, 2024

IlyaNiklyaev commented Mar 11, 2024 • edited Loading

SomeoneSerge commented Mar 11, 2024

IlyaNiklyaev commented Mar 12, 2024

wegank commented Jun 30, 2024

ConnorBaker commented Dec 7, 2023 •

edited

Loading

SomeoneSerge commented Dec 12, 2023 •

edited by ConnorBaker

Loading

IlyaNiklyaev commented Feb 26, 2024 •

edited

Loading

IlyaNiklyaev commented Mar 7, 2024 •

edited

Loading

IlyaNiklyaev commented Mar 11, 2024 •

edited

Loading