Unable to get working OpenVINO/ONNX GPU accelleration on Ubuntu 24.04, Intel i7 1355U #24797
Comments
|
Hi @ilya-lavrenov , Installation of ONNX Runtime with OpenVINO For installing ONNX Runtime, rather than using pip install openvino, please follow these steps:
Running Docker Container Please use the following command to run the Docker container:
This command ensures that the container has the same group permissions as the host for the Direct Rendering Manager (DRM) devices, allowing you to perform inference using GPU devices. We have tested this setup on Ubuntu 20.04, and i believe it should work on Ubuntu 24.04 as well. |
|
tried running the more elaborate docker command but does not seem to work: |
|
tried to add the group in after the container has been created:
But still does not work: |
|
Hi, can you share the current GPU driver version that is installed in your ubuntu 24.04 OS and the link for installing the driver |
|
I'm having the same issue, |
Sure. This is "stock Ubuntu 24.04 LTS", have not installed anything additional. So I don't think I have more details than the kernel version and how the driver informs us in dmesg. It did not look like you have to install a out-of-tree driver to make it work? As OpenVino does find the GPU on the host. |
|
@khhsenti I just setup my own docker from base 24.04 image, added their installs and got gpu acceleration successfully. |
|
@khhsenti has your issue here been addressed? According to the instructions there are no additional dependencies (i.e. graphics driver) to install within Docker. You can refer to Using OpenVINO™ Toolkit containers with GPU accelerators for additional guidance on accessing GPU. Please note the latest release of OpenVINO 2024.3.0 just added preview support for Ubuntu 24.04 OS, you may want to have a try and see if the issue you encountered has been addressed in this latest drop. Check out the release notes for additional details. Hope this helps. |
|
@khhsenti also based on the System Requirements for environment it lists Python 3.8 to Python 3.11 as a requirement. I see that on the details you are using Python 3.12.3 as it is the default version on Ubuntu 24.04. If not already done so, is it possible to try using Python 3.11 if you are building ONNX with OpenVINO? Note I am able to access the GPU within Docker on a Meteor Lake system (Core Ultra 7 155H running Ubuntu 24.04 natively on host) using the openvino/ubuntu22_runtime image. This Docker image has driver 23.13.26032.30, whereas openvino/onnxruntime_ep_ubuntu20 has 22.43.24595 which is a bit old (released almost 2 years ago on Dec 2022). So the issue may also be due the fact this image contains an outdated graphics driver. FYI I've tried the steps below and this working for me, hope it does for you as well. |
|
@avitial have been busy with something else lately, will take another look at this during next week. Thank you for the pointers perhaps this is solved by the new release |
|
I have some problem, and i found some models can run on GPU(like hello_classification sample), some not. openvino version: 2024.3.0 |
|
@avitial here is the output from running what you are suggesting: driver version (on host): inside your suggested docker container: But, the Docker image suggested here, does not provide ONNX runtime, which is a requirement from our side. On the newest release of the ONNX docker image (openvino/onnxruntime_ep_ubuntu20:latest) i get: |
|
i just tried building my own docker image based on: https://github.com/microsoft/onnxruntime/blob/main/dockerfiles/Dockerfile.openvino it still does not work: will try to update it to use the latest release of the openvino/ubuntu22_runtime |
|
happy to report that it works now with the updated docker images: when testing above, i forgot the additional --group-add which makes it work on the onnx docker image release a couple of months ago. |
|
that is |
OpenVINO Version
onnxruntime-openvino 1.17.1
Operating System
Other (Please specify in description)
Hardware Architecture
x86 (64 bits)
Target Platform
DT Research tablet DT302-RP with Intel i7 1355U , running Ubuntu 24.04 LTS
Build issue description
We are trying to evaluate the possible performance improvement when running AI inference through the OpenVINO framework. Our AI models are ONNX based, and as such the integration between OpenVINO and ONNX need to work.
We have tried using the provided demonstration Docker container, but it is unable to find the GPU.
Installing the OpenVINO framework on the host OS directly, does provide the GPU as an option, but the Python version on Ubuntu 24.04 is too new that there is a PIP package available for the OpenVINO ONNX integration. We have then continued to build the ONNX integration manually, which worked, but the OpenVINO execution provider does not load.
Build script or step-by-step to reproduce
Run the provided docker container:
Install OpenVINO manually on host OS:
Follow: https://docs.openvino.ai/2022.3/openvino_docs_install_guides_configurations_for_intel_gpu.html
pip install openvino newest version
root@sentispec-DT302RP-dsv-1:~# source openvino_env/bin/activate (openvino_env) root@sentispec-DT302RP-dsv-1:~# python3 -c "from openvino import Core; print(Core().available_devices)" ['CPU', 'GPU'] (openvino_env) root@sentispec-DT302RP-dsv-1:~#Build ONNX with OpenVino wheel to manual installation:
https://onnxruntime.ai/docs/build/eps.html#linux-4
pip install /home/sentispec-admin/onnxruntime_openvino-1.19.0-cp312-cp312-linux_x86_64.whl
Try to use it:
Happy to provide any details, decided to keep it as short as possible for now. I am aware ubuntu 24.04 is not currently officially supported but was thinking that it is planned e.g. also regarding this: #23905
Relevant log output
No response
Issue submission checklist
The text was updated successfully, but these errors were encountered: