Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Doc] xpu backend requires running setvars.sh #6393

Merged
merged 1 commit into from
Jul 15, 2024

Conversation

rscohn2
Copy link
Contributor

@rscohn2 rscohn2 commented Jul 12, 2024

IPEX depends on MKL, and you need to do setvars.sh to setup the LD_LIBRARY_PATH to find it. It works fine with with setvars.sh and 2024.2 oneapi.

@jikunshang: Thanks for adding XPU support!

Here is the error message:

(.venv) rscohn1@gkdse-dnp-23:~/projects/cutlass/vllm$ python examples/offline_inference.py 
WARNING 07-12 23:25:17 _custom_ops.py:14] Failed to import from vllm._C with ModuleNotFoundError("No module named 'vllm._C'")
WARNING 07-12 23:25:17 utils.py:218] Import Error for IPEX: libmkl_intel_lp64.so.2: cannot open shared object file: No such file or directory
WARNING 07-12 23:25:17 utils.py:222] not found ipex lib

Also appears to work fine with 2024.2
Copy link

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only trigger fastcheck CI to run, which consists only a small and essential subset of tests to quickly catch small errors.

Full CI run is still required to merge this PR so please make sure that you run full CI before merging or if you need more test signals.

To run full CI, you can do one of these:

  • Add ready label to the PR
  • Comment /ready on the PR
  • Enable auto-merge.

🚀

@jikunshang
Copy link
Contributor

LGTM! Thanks for this fix!

@DarkLight1337 DarkLight1337 requested a review from WoosukKwon July 14, 2024 08:12
Copy link
Collaborator

@WoosukKwon WoosukKwon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks for the fix!

@WoosukKwon WoosukKwon merged commit 61e85db into vllm-project:main Jul 15, 2024
30 checks passed
@rscohn2 rscohn2 deleted the patch-1 branch July 15, 2024 00:29
dtrifiro pushed a commit to opendatahub-io/vllm that referenced this pull request Jul 17, 2024
@zhouyuan
Copy link
Contributor

Note: it's recommended to picked the "paired" version of oneAPI and IPEX in the XPU docker file as it's more tested
https://github.com/vllm-project/vllm/blob/main/Dockerfile.xpu

thanks, -yuan

xjpang pushed a commit to xjpang/vllm that referenced this pull request Jul 24, 2024
Alvant pushed a commit to compressa-ai/vllm that referenced this pull request Oct 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants