Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revert to use out-of-tree GPU driver #11761

Merged
merged 4 commits into from
Aug 12, 2024
Merged

Revert to use out-of-tree GPU driver #11761

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
4f3c2bf
Revert to use out-of-tree GPU driver since the performance with out-o…
liu-shaojun Aug 12, 2024
87b0768
add spaces
liu-shaojun Aug 12, 2024
20dd546
add troubleshooting case
liu-shaojun Aug 12, 2024
376427d
update Troubleshooting
liu-shaojun Aug 12, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/mddocs/DockerGuides/docker_pytorch_inference_gpu.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ root@arda-arc12:/# sycl-ls
> export USE_XETLA=OFF
>
> # Enable immediate command lists mode for the Level Zero plugin. Improves performance on Intel Arc™ A-Series Graphics and Intel Data Center GPU Max Series; however, it depends on the Linux Kernel, and some Linux kernels may not necessarily provide acceleration.
> # Recommended for use on Intel Arc™ A-Series Graphics and Intel Data Center GPU Max Series, but it depends on the Linux kernel, Non-i915 kernel drivers may cause performance regressions.
> # Recommended for use on Intel Arc™ A-Series Graphics and Intel Data Center GPU Max Series, but it depends on the Linux kernel, Upstream i915 kernel drivers may cause performance regressions.
> export SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=1
>
> # Controls persistent device compiled code cache. Set to '1' to turn on and '0' to turn off.
Expand Down
10 changes: 8 additions & 2 deletions docs/mddocs/Overview/install_gpu.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -171,7 +171,7 @@ IPEX-LLM GPU support on Linux has been verified on:

> **Tip**:
>
> Please refer to our [driver installation](https://dgpu-docs.intel.com/driver/installation.html) for general purpose GPU capabilities.
> For client GPUs, such as the Intel® Arc™ A-series, please refer to [Client GPU Installation Guide](https://dgpu-docs.intel.com/driver/client/overview.html). For data center GPUs, including Intel® Data Center GPU Max Series and Intel® Data Center GPU Flex Series, please refer to our [Installation for Data Center GPU](https://dgpu-docs.intel.com/driver/installation.html) for general purpose GPU capabilities.
>
> See [release page](https://dgpu-docs.intel.com/releases/index.html) for latest version.

Expand Down Expand Up @@ -311,7 +311,7 @@ IPEX-LLM GPU support on Linux has been verified on:

> **Tip**:
>
> Please refer to our [driver installation](https://dgpu-docs.intel.com/driver/installation.html) for general purpose GPU capabilities.
> For client GPUs, such as the Intel® Arc™ A-series, please refer to [Client GPU Installation Guide](https://dgpu-docs.intel.com/driver/client/overview.html). For data center GPUs, including Intel® Data Center GPU Max Series and Intel® Data Center GPU Flex Series, please refer to our [Installation for Data Center GPU](https://dgpu-docs.intel.com/driver/installation.html) for general purpose GPU capabilities.
>
> See [release page](https://dgpu-docs.intel.com/releases/index.html) for latest version.

Expand Down Expand Up @@ -623,3 +623,9 @@ The reason for such errors is that oneAPI has not been initialized properly befo
* For oneAPI installed using APT or Offline Installer, make sure you execute `setvars.sh` of oneAPI Base Toolkit before running IPEX-LLM.
* For PIP-installed oneAPI, activate your working environment and run ``echo $LD_LIBRARY_PATH`` to check if the installation path is properly configured for the environment. If the output does not contain oneAPI path (e.g. ``~/intel/oneapi/lib``), check [Prerequisites](#prerequisites-1) to re-install oneAPI with PIP installer.
* Make sure you install matching versions of ipex-llm/pytorch/IPEX and oneAPI Base Toolkit. IPEX-LLM with PyTorch 2.1 should be used with oneAPI Base Toolkit version 2024.0. IPEX-LLM with PyTorch 2.0 should be used with oneAPI Base Toolkit version 2023.2.

#### 2. `core dump` when running with GPU
If encountered random `core dump` when running with GPU, please remove out of tree driver.
```
sudo apt purge -y intel-i915-dkms intel-fw-gpu
```
12 changes: 10 additions & 2 deletions docs/mddocs/Quickstart/install_linux_gpu.md
View file
Open in desktop
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add trouble shooting in case out of tree driver is conflicted with kernel driver.

If encountered random core dump when running with GPU, please remove out of tree driver.

Original file line number Diff line number Diff line change
Expand Up @@ -34,13 +34,17 @@ IPEX-LLM currently supports the Ubuntu 20.04 operating system and later, and sup

```bash
sudo apt-get update

# Install out-of-tree driver
sudo apt-get -y install \
gawk \
dkms \
linux-headers-$(uname -r) \
libc6-dev
sudo apt install intel-i915-dkms intel-fw-gpu

sudo apt-get install -y gawk libc6-dev udev\
# Install Compute Runtime
sudo apt-get install -y udev \
intel-opencl-icd intel-level-zero-gpu level-zero \
intel-media-va-driver-non-free libmfx1 libmfxgen1 libvpl2 \
libegl-mesa0 libegl1-mesa libegl1-mesa-dev libgbm1 libgl1-mesa-dev libgl1-mesa-dri \
Expand Down Expand Up @@ -82,13 +86,17 @@ IPEX-LLM currently supports the Ubuntu 20.04 operating system and later, and sup

```bash
sudo apt-get update

# Install out-of-tree driver
sudo apt-get -y install \
gawk \
dkms \
linux-headers-$(uname -r) \
libc6-dev
sudo apt install -y intel-i915-dkms intel-fw-gpu

sudo apt-get install -y gawk libc6-dev udev\
# Install Compute Runtime
sudo apt-get install -y udev \
intel-opencl-icd intel-level-zero-gpu level-zero \
intel-media-va-driver-non-free libmfx1 libmfxgen1 libvpl2 \
libegl-mesa0 libegl1-mesa libegl1-mesa-dev libgbm1 libgl1-mesa-dev libgl1-mesa-dri \
Expand Down
12 changes: 10 additions & 2 deletions docs/readthedocs/source/doc/LLM/Quickstart/install_linux_gpu.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,13 +25,17 @@ IPEX-LLM currently supports the Ubuntu 20.04 operating system and later, and sup

```bash
sudo apt-get update

# Install out-of-tree driver
sudo apt-get -y install \
gawk \
dkms \
linux-headers-$(uname -r) \
libc6-dev
sudo apt install intel-i915-dkms intel-fw-gpu

sudo apt-get install -y gawk libc6-dev udev\
# Install Compute Runtime
sudo apt-get install -y udev \
intel-opencl-icd intel-level-zero-gpu level-zero \
intel-media-va-driver-non-free libmfx1 libmfxgen1 libvpl2 \
libegl-mesa0 libegl1-mesa libegl1-mesa-dev libgbm1 libgl1-mesa-dev libgl1-mesa-dri \
Expand Down Expand Up @@ -73,13 +77,17 @@ IPEX-LLM currently supports the Ubuntu 20.04 operating system and later, and sup

```bash
sudo apt-get update

# Install out-of-tree driver
sudo apt-get -y install \
gawk \
dkms \
linux-headers-$(uname -r) \
libc6-dev
sudo apt install intel-i915-dkms intel-fw-gpu

sudo apt-get install -y gawk libc6-dev udev\
# Install Compute Runtime
sudo apt-get install -y udev \
intel-opencl-icd intel-level-zero-gpu level-zero \
intel-media-va-driver-non-free libmfx1 libmfxgen1 libvpl2 \
libegl-mesa0 libegl1-mesa libegl1-mesa-dev libgbm1 libgl1-mesa-dev libgl1-mesa-dri \
Expand Down