could not select device driver “” with capabilities: [[gpu]]

[eddydw]

I am using WSL2 and docker and GPU aware docker.

I have windows 10 version 2004 (build 20161.1000) I have installed WSL 2 and have Docker Desktop 2.3.0.3 on my Windows System running. I have Ubuntu 18.04 LTS installed in WSL 2 too. I have installed the NVIDIA driver enter image description here

The linux version is 4.19.121-microsoft-standard. The NVIDIA driver version is 455.41 for my Laptop GPU QUADRO M2000M. Actually I followed all the steps described in https://ubuntu.com/blog/getting-started-with-cuda-on-ubuntu-on-wsl-2 until the step where I have to run "sudo service docker stop" in an Ubuntu terminal.

This results in a message docker: unrecognized service.

I have to restart docker desktop in Windows 10 in order to get the daemon running.

I test then in the Ubuntu terminal: docker run hello-world ==> this runs fine Also the command docker run -it ubuntu bash ==> runs the file in the Ubuntu terminal os WSL 2.

BUT when I run : docker run -u (id−u):(id -g) -it --gpus all -p 8888:8888 tensorflow/tensorflow:latest-gpu-py3-jupyter

then I get the error : docker: Error response from daemon: could not select device driver "" with capabilities: [[gpu]]

This involves Microsoft, Ubuntu, NVIDIA. I have searched the support sites but could not find anything that solves my problem.

Can anyone help me here?

[wicca1984]

I was running into the same problem.
I followed this article https://docs.nvidia.com/cuda/wsl-user-guide/index.html#installing-docker

I think there's a problem with the Docker installation part
"Install Docker
Use the Docker installation script to install Docker for your choice of WSL 2 Linux distribution. Note that NVIDIA Container Toolkit does not yet support Docker Desktop WSL 2 backend."

[stephen-turner]

Thanks for the reports. As you know, Microsoft have only just released GPU support in WSL 2 to the Insiders builds, so this is all very new. We are aware that the Nvidia drivers don't yet run on Docker Desktop, and we're working on it.

[stephen-turner]

Also, we have an item about this on our public roadmap -- docker/roadmap#96 -- feel free to give it a thumbs up if you haven't already.

[eddydw]

Hi Stephen Thanks for the update. Best regards Eddy Op wo 8 jul. 2020 om 17:36 schreef Stephen Turner <[email protected]

…

: Thanks for the reports. As you know, Microsoft have only just released GPU support in WSL 2 to the Insiders builds, so this is all very new. We are aware that the Nvidia drivers don't yet run on Docker Desktop, and we're working on it. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#7581 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABFRBCZ5XNDAOCVFZV4DHT3R2SG7ZANCNFSM4OS6S7ZQ> .

[eddydw]

Hi I did follow the instruction without any change tough Kind regards Eddy

…

---------- Forwarded message --------- Van: wicca <[email protected]> Date: wo 8 jul. 2020 om 11:18 Subject: Re: [docker/for-win] could not select device driver “” with capabilities: [[gpu]] (#7581) To: docker/for-win <[email protected]> Cc: eddydw <[email protected]>, Author <[email protected]> I was running into the same problem. I followed this article https://docs.nvidia.com/cuda/wsl-user-guide/index.html#installing-docker I think there's a problem with the Docker installation part "Install Docker Use the Docker installation script to install Docker for your choice of WSL 2 Linux distribution. Note that NVIDIA Container Toolkit does not yet support Docker Desktop WSL 2 backend." — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#7581 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABFRBC6VXIKCTRQWYLGQKLLR2Q2WLANCNFSM4OS6S7ZQ> .

[jnoelvictorino]

I also recently encountered this roadblock. What I did was to keep Docker Desktop installed in Windows 10.
But I also installed Docker inside WSL2 distro (as mentioned in the instructions).

This setup is working for now, but it is not ideal.
When I want to use Docker to run with GPUs, I have to turn off Docker Desktop and turn on Docker inside WSL2 distro.
Some goes if I want to use Docker Desktop only.

[dshep]

I finally got it to work.

Installed a new instance of ubuntu 18.04 that was not my default WSL2 instance, and not linked to docker for windows.

Installed docker into this instance following the instructions here: https://dev.to/bartr/install-docker-on-windows-subsystem-for-linux-v2-ubuntu-5dl7

I still got an error with mounts when i ran the hello world container. This was fixed with the following 2 commands (from this issue: microsoft/WSL#4189)
sudo mkdir /sys/fs/cgroup/systemd
sudo mount -t cgroup -o none,name=systemd cgroup /sys/fs/cgroup/systemd

Once i had docker running successfully, I followed the instruction these instructions to get docker instances running using my RTX GPU: https://docs.nvidia.com/cuda/wsl-user-guide/index.html#setting-containers. I skipped the Running Cuda applications section because the demo's have changed. There are also build issues with the RTX cards, because not all the demos support the version of CUDA that card does, so you have to manually set compile flags to reduce the version.

Hope this helps

[Agrover112]

Yes you are right even I skipped them

[jnoelvictorino]

Thanks for the tip @dshep!
Docker Desktop and Docker inside WSL2 distro now work at the same time.

I made docker-desktop the default distro for WSL2

wsl -S docker-desktop

Then in the Docker Desktop settings, I did not Enable integration with additional distros: Ubuntu

[tadam98]

This makes it work.
Basically install docker on wsl2.
https://dev.to/bartr/install-docker-on-windows-subsystem-for-linux-v2-ubuntu-5dl7

After restart of wsl2 do the following procedure:

$ docker ps
$ docker ps -aq
$ docker stop $(docker ps -aq)
$ docker stop # dockers reported by ps
$ rm ~/.docker/config.json
$ sudo service docker stop
$ sudo service docker start
$ sudo mkdir /sys/fs/cgroup/systemd
$ sudo mount -t cgroup -o none,name=systemd cgroup /sys/fs/cgroup/systemd
$ docker ps
$ docker run hello-world
$ docker run --gpus all nvcr.io/nvidia/k8s/cuda-sample:nbody nbody -gpu -benchmark
GPU Device 0: "GeForce RTX 2080 Ti" with compute capability 7.5

> Compute 7.5 CUDA device: [GeForce RTX 2080 Ti]
69632 bodies, total time for 10 iterations: 117.583 ms
= 412.357 billion interactions per second
= 8247.147 single-precision GFLOP/s at 20 flops per interaction

One more thing you need to know. The usual docker-compose (last one is version 1.27) that gets installed with the docker does not support starting dockers with gpu. The only way to manage docker containers with gpu is using an experimental version of docker-compose that does support management of docker containers that use a gpu.

$ which docker-compose
$ sudo rm -rf <folder of docker compose> 
$ pip install git+https://github.com/yoanisgil/compose.git@device-requests
$ docker-composer --version
docker-compose version 1.26.0dev, build unknown
# to use it you have to make sure that API version is 1.40. This can be done on the same command-line:
$ COMPOSE_API_VERSION=1.40 docker-compose up

In the docker-compose.yml for the docker container <docker_name_needing_gpu> that needs a gpu define:

    NVIDIA_DRIVER_PRESENT: "True"
    container_name: docker_name_needing_gpu
    device_requests:
        - capabilities:
            - "gpu"

[jnoelvictorino]

Yes, exactly! 💯
Whenever I restart my machine, I had to repeat these steps:

$ sudo service docker start
$ sudo mkdir /sys/fs/cgroup/systemd # sometimes needed, sometimes not needed
$ sudo mount -t cgroup -o none,name=systemd cgroup /sys/fs/cgroup/systemd # sometimes needed, sometimes not needed

So, I added these commands in my ~/.bash_profile or ~/.bashrc or ~/.profile

[tadam98]

Windows insider updated to 20262.1
With the update, the regular nvidia driver was re-installed. Not good for wsl2.
Also, nvidia issued a new driver that you should install in windows:
https://developer.nvidia.com/cuda/wsl
The new drive is 465.12
Download and install it.
You must reboot after the install or wsl2 will not see the gpu as of yet.
Start wsl2

$ python
> import tensorflow as tf
>  tf.test.is_gpu_available()
2020-11-19 17:15:13.395676: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): GeForce RTX 2080 Ti, Compute Capability 7.5
True

Then do the above docker procedures to check that the docker aslo works.
In my case, it works fine.

[stephen-turner]

We released a preview of GPU support in Docker Desktop on WSL 2 to the users on our development preview program yesterday. If you're interested in joining the program, please see https://www.docker.com/blog/expanding-dockers-developer-preview-program/.

[tadam98]

I am interested. As I noted, there are two issues. One is docker gpu support which I solved by simply installing docker in wsl2. Second is that docker-compose does not support gpu dockers (regardless of wsl2) which is solved by using the dev version I mentioned. Both issues need to be addressed. Best, Mickey

…

________________________________ From: Stephen Turner <[email protected]> Sent: Friday, December 11, 2020 6:10:04 PM To: docker/for-win <[email protected]> Cc: tadam98 <[email protected]>; Comment <[email protected]> Subject: Re: [docker/for-win] could not select device driver “” with capabilities: [[gpu]] (#7581) We released a preview of GPU support in Docker Desktop on WSL 2 to the users on our development preview program yesterday. If you're interested in joining the program, please see https://www.docker.com/blog/expanding-dockers-developer-preview-program/. — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#7581 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AFBMCTYST2EYM7H5BHFPX2TSUI75ZANCNFSM4OS6S7ZQ>.

[Agrover112]

Yes, exactly! 💯
Whenever I restart my machine, I had to repeat these steps:

$ sudo service docker start
$ sudo mkdir /sys/fs/cgroup/systemd # sometimes needed, sometimes not needed
$ sudo mount -t cgroup -o none,name=systemd cgroup /sys/fs/cgroup/systemd # sometimes needed, sometimes not needed

So, I added these commands in my ~/.bash_profile or ~/.bashrc or ~/.profile

Does this solve the problem? @tadam98 I am still facing the same error? So GPU containers in Docker-Desktop are not yet supported for WSL2 is it?

[mattkenefick]

Are we still in workaround territory or do we have a proper solution RE: @stephen-turner 's preview

[docker-robott]

Issues go stale after 90 days of inactivity.
Mark the issue as fresh with /remove-lifecycle stale comment.
Stale issues will be closed after an additional 30 days of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle stale

[stephen-turner]

This was released in Docker Desktop back in January. Note that it still requires an Insiders build of Windows until Microsoft include the necessary functionality in a GA build. For that reason it's still officially experimental, but it's complete from the Docker side.

[docker-robott]

Closed issues are locked after 30 days of inactivity.
This helps our team focus on active issues.

If you have found a problem that seems similar to this, please open a new issue.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle locked