Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CPU] Disable ConvertGatherToGatherCompressed optimization for quantized models #25478

Merged

Conversation

xipingyan
Copy link
Contributor

@xipingyan xipingyan commented Jul 10, 2024

Details:

  • Disable ConvertGatherToGatherCompressed pass in case useLPT is false

Tickets:

  • 138337

Reasons:
1: Small size weights have little impact on compile_model performance;
2: There is a high probability that constfolding will not be performed when compile_model;

Signed-off-by: xipingya <[email protected]>
@xipingyan xipingyan requested review from a team as code owners July 10, 2024 02:52
@xipingyan xipingyan requested review from itikhono and removed request for a team July 10, 2024 02:52
@github-actions github-actions bot added category: CPU OpenVINO CPU plugin category: transformations OpenVINO Runtime library - Transformations labels Jul 10, 2024
@dmitry-gorokhov dmitry-gorokhov self-assigned this Jul 10, 2024
… model versa.

disable it when useLpt == false.

Signed-off-by: xipingya <[email protected]>
@xipingyan xipingyan force-pushed the xp/fix_versa_acc_degradation branch from c39a9bd to 1420794 Compare July 10, 2024 06:40
@github-actions github-actions bot removed the category: transformations OpenVINO Runtime library - Transformations label Jul 10, 2024
@dmitry-gorokhov dmitry-gorokhov changed the title [CPU]Filter out small weights for GatherCompressed. [CPU] Disable ConvertGatherToGatherCompressed optimization for quantized models Jul 11, 2024
1: Add test
2: When u8/i8 + useLpt, disable "ConvertGatherToGatherCompressed".

Signed-off-by: xipingya <[email protected]>
@github-actions github-actions bot added the category: transformations OpenVINO Runtime library - Transformations label Jul 12, 2024
Signed-off-by: xipingya <[email protected]>
@dmitry-gorokhov dmitry-gorokhov added this to the 2024.4 milestone Jul 17, 2024
@dmitry-gorokhov
Copy link
Contributor

@v-Golubev could you please also take a look?

2: u8/i8: check if exist Gather
3: i8: check MatMul's runtime precision is u8;

Signed-off-by: xipingya <[email protected]>
@xipingyan xipingyan force-pushed the xp/fix_versa_acc_degradation branch from e73b9e1 to 6b30374 Compare July 19, 2024 06:52
@dmitry-gorokhov dmitry-gorokhov added this pull request to the merge queue Jul 23, 2024
Merged via the queue into openvinotoolkit:master with commit bb7f0e7 Jul 23, 2024
123 checks passed
@xipingyan xipingyan deleted the xp/fix_versa_acc_degradation branch July 23, 2024 06:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category: CPU OpenVINO CPU plugin category: transformations OpenVINO Runtime library - Transformations
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants