Backport 271078 to staging 23.11

[SomeoneSerge]

Description of changes

Supersedes #272336. Incorporates #272063, #273048, #272964 and #272812

This probably goes into staging before #272784

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
  • sandbox = relaxed
  • sandbox = true
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

---

Add a 👍 reaction to pull requests you find important.

[SomeoneSerge]

I should probably reorder the commits so that blender won't be broken in the intermediate commits either

[ConnorBaker]

Note

Template nixpkgs-review command:

PR=273040; \
SYSTEM="aarch64-linux"; \
CUDA_SUPPORT="true"; \
CUDA_CAPABILITIES='[ "7.5" ]'; \
nixpkgs-review pr "$PR" \
  --system "$SYSTEM" \
  --no-shell \
  --checkout commit \
  --allow aliases \
  --build-args "--max-jobs 1" \
  --extra-nixpkgs-config "{
    allowUnfree = true;
    allowBroken = false;
    cudaSupport = ${CUDA_SUPPORT:-false};
    cudaCapabilities = ${CUDA_CAPABILITIES:-[]};
  }"

🕐 aarch64-darwin

In progress.

🕐 x86_64-darwin

In progress.

🕐 aarch64-linux

✅ Jetson

Unsupported pending merge of #272784.

✅ Non-Jetson

Unsupported pending merge of #272784.

🕐 Non-CUDA

In progress.

❌ x86_64-linux

🕐 Non-Jetson

In progress.

❌ Non-CUDA

Failure is due to my explicitly passing an empty list for cudaCapabilities, which should not be done as it overrides the default prior to #272784, causing derivations which assume they will always be built with CUDA support (like magma-cuda) to fail with an out of bounds error when indexing into the empty list.

$ PR=273040; SYSTEM="x86_64-linux"; CUDA_SUPPORT="false"; CUDA_CAPABILITIES='[ ]'; nixpkgs-review pr "$PR"   --system "$SYSTEM"   --no-shell   --checkout commit   --allow aliases   --build-args "--max-jobs 1"   --extra-nixpkgs-config "{
    allowUnfree = true;
    allowBroken = false;
    cudaSupport = ${CUDA_SUPPORT:-false};
    cudaCapabilities = ${CUDA_CAPABILITIES:-[]};
  }"
$ git -c fetch.prune=false fetch --no-tags --force https://github.com/NixOS/nixpkgs staging-23.11:refs/nixpkgs-review/0 pull/273040/head:refs/nixpkgs-review/1
$ git worktree add /home/connorbaker/.cache/nixpkgs-review/pr-273040-2/nixpkgs d8ed2baa7c48a75470e2093e6dfdf497ffa31d4c
Preparing worktree (detached HEAD d8ed2baa7c48)
Updating files: 100% (38204/38204), done.
HEAD is now at d8ed2baa7c48 cudaPackages.cudatoolkit: propagate the hook to nativeBuildInputs correctly
error:
       … while evaluating attribute 'magma-cuda'

         at /home/connorbaker/.cache/nixpkgs-review/pr-273040-2/nixpkgs/lib/attrsets.nix:488:14:

          487|     value:
          488|     { inherit name value; };
             |              ^
          489|

       … while evaluating attribute 'broken'

         at /nix/store/3mzsbqwxz6va3f0fwsbsvkahpp15w62y-nixpkgs-review/lib/python3.11/site-packages/nixpkgs_review/nix/evalAttrs.nix:39:20:

           38|           {
           39|             inherit exists broken;
             |                    ^
           40|             path = if !broken then maybePath.value else null;

       (stack trace truncated; use '--show-trace' to show the full trace)

       error: list index 0 is out of bounds

       at «none»:0: (source not available)
https://github.com/NixOS/nixpkgs/pull/273040 failed to build: nix --extra-experimental-features nix-command no-url-literals --system x86_64-linux eval --nix-path nixpkgs=/home/connorbaker/.cache/nixpkgs-review/pr-273040-2/nixpkgs nixpkgs-overlays=/tmp/tmpbi17390e --json --impure --no-allow-import-from-derivation --expr (import /nix/store/3mzsbqwxz6va3f0fwsbsvkahpp15w62y-nixpkgs-review/lib/python3.11/site-packages/nixpkgs_review/nix/evalAttrs.nix { attr-json = /tmp/tmpbb0h7tim; }) failed to run, /tmp/tmpbb0h7tim was stored inspection

[ConnorBaker]

Just waiting for ofborg-eval to finish before merging...

[vcunat]

cctag won't build now. It did on 23.11.
https://hydra.nixos.org/build/245993514

[vcunat]

At a glance, I'd say that such large changes aren't even suitable for backporting to stable.

[SomeoneSerge]

Cctag definitely was affected by the opencvConfig issue on master at the time, as were two dozen other cudaSupport=true packages in the release branch (link tbd, writing from a mobile). The backport is huge partly because I tried to merge and reorder fixes from several correlated PRs. I'll look into the hydra failure, thx

[vcunat]

For reference, target PR #279139