Running CUDA inside Docker\WSL on Windows 11 Hyper-v VM with GPU Patritioning

[PolyglotBuckeye]

Is your feature request related to a problem? Please describe.

The problem is that WSL under a Hyper-V GPU Partitioned machine can't use the GPU (Nvidia), as stated in #8179 2 YEARS ago.

Describe the solution you'd like

Expected behavior on vm:
https://www.youtube.com/watch?v=JaHVsZa2jTc

For this to work there should be some configuration regarding the linkers to the drivers that are virtualized inside the Hyper-v VM.

The files from "C:\Windows\System32\lxss" when copied to the VM should do the link to the drivers. Or they should be changed in a way to accommodate the virtual GPU.

Describe alternatives you've considered

I'll try to do GPU partitioning with a Ubuntu VM under Hyper-v.

Additional context

Tech stack:
Windows 11 ✅ → Hyper-v ✅-> Windows 11 guest + Gpu Partitioning + Nested Virtualization ✅-> Docker Desktop + WSL ✅ → GPU acceleration (nvidia-smi) ❌

This is what I get in WSL when I copy the linkers("C:\Windows\System32\lxss") to VM:

    Ubuntu:$ nvidia-smi

    Failed to initialize NVML: GPU access blocked by the operating system

    Failed to properly shut down NVML: GPU access blocked by the operating system

More details and my hardware are here:
https://forums.developer.nvidia.com/t/run-cuda-inside-docker-wsl-on-windows-11-hyper-v-vm-with-gpu-patritioning/308212

#Cuda #WSL #GPU-P #Paravirtualization #GPU Partitioning #NVIDIA #Docker #Nested Virtualization #nvidia-smi #Hyper-v

[github-actions]

Logs are required for review from WSL team

If this a feature request, please reply with '/feature'. If this is a question, reply with '/question'.
Otherwise please attach logs by following the instructions below, your issue will not be reviewed unless they are added. These logs will help us understand what is going on in your machine.

How to collect WSL logs

Download and execute collect-wsl-logs.ps1 in an administrative powershell prompt:

Invoke-WebRequest -UseBasicParsing "https://raw.githubusercontent.com/microsoft/WSL/master/diagnostics/collect-wsl-logs.ps1" -OutFile collect-wsl-logs.ps1
Set-ExecutionPolicy Bypass -Scope Process -Force
.\collect-wsl-logs.ps1

The script will output the path of the log file once done.

If this is a networking issue, please use collect-networking-logs.ps1, following the instructions here

Once completed please upload the output files to this Github issue.

Click here for more info on logging
If you choose to email these logs instead of attaching to the bug, please send them to [email protected] with the number of the github issue in the subject, and in the message a link to your comment in the github issue and reply with '/emailed-logs'.

View similar issues

Please view the issues below to see if they solve your problem, and if the issue describes your problem please consider closing this one and thumbs upping the other issue to help us prioritize it!

Open similar issues:

• Cannot use GPU support in virtualized Windows 11 21H2 (22000.556) (#8179), similarity score: 0.81
• dxgkrnl under GPU-P (nested virtualization) (#10357), similarity score: 0.81
• PCI Passthrough/Discrete Device Assignment for WSL2. (#5492), similarity score: 0.74

Note: You can give me feedback by thumbs upping or thumbs downing this comment.

[PolyglotBuckeye]

/feature

[github-actions]

Diagnostic information

Found '/feature', adding tag 'feature'

[sirredbeard]

I'll try to do GPU partitioning with a Ubuntu VM under Hyper-v.

I am curious what would happen in this scenario.

In my experience working with partitioned GPUs on Azure (but admittedly never on WSL) requires the NVIDIA GRID drivers.