Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[NPU] Add initial support for minicpm-llama-v2.5 #11962

Merged
merged 6 commits into from
Aug 30, 2024

Conversation

sgwhat
Copy link
Contributor

@sgwhat sgwhat commented Aug 29, 2024

Description

MiniCPMV is divided into 3 components: LLM, VPM, and Resampler. We run the LLM ccomponet on NPU and others on CPU.

1. Why the change?

Add npu optimization for minicpm-llama-v-2.5.

2. User API changes

-------------------- Input --------------------
http://farm6.staticflickr.com/5268/5602445367_3504763978_z.jpg
-------------------- Prompt --------------------
What is in the image?
-------------------- Output --------------------
The image depicts a young girl holding a small white teddy bear. She is wearing a pink and white striped dress. The background shows a stone wall with red flowers in the foreground

3. Summary of the change

  • npu multi-processing optimization
  • npu example and readme
  • vpm attention optimization (another pr)

4. How to test?

We test this model with images input.

  • MTL
  • LNL

@sgwhat sgwhat changed the title [WIP] Add initial support for minicpm-llama-v2.5 [NPU] Add initial support for minicpm-llama-v2.5 Aug 30, 2024
@sgwhat sgwhat requested review from plusbang and rnwang04 August 30, 2024 07:56
@sgwhat
Copy link
Contributor Author

sgwhat commented Aug 30, 2024

Impl in npu_model.py will be replace by #11966 then.

@sgwhat sgwhat merged commit 158289d into intel-analytics:main Aug 30, 2024
1 check passed
cranechu0131 pushed a commit to cranechu0131/ipex-llm that referenced this pull request Sep 9, 2024
* add initial support for minicpm-llama-v2.5

* update impl

* add minicpm-llama3-v2.5 example
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants