-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support vpm and resampler module of minicpm-v on NPU #12375
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
others LGTM
padded_weight = F.pad(module.lm_head.weight, | ||
(0, 0, 0, 152064-151666)) # 152064 is qwen2-7b vocab_size |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will this impact accuracy for channel-wise?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will this impact accuracy for channel-wise?
The original weight shape is [151666, 3584] and pad it to [152064, 3584], each row has no change. I think this don't influence CW : )
Description
Update minicpm-v usage on NPU.
2. User API changes
torch_dtype=torch.float32
andmodules_to_not_convert=['vpm', 'resampler']
3. Summary of the change
MinicpmVPatchEmbedding
andMinicpmVLayerNorm
4. How to test?
https://github.com/analytics-zoo/nano/issues/1724#issuecomment-2467282958