Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support the scale estimation inside GPTQ #2784

Merged
merged 2 commits into from
Sep 4, 2024
Merged

Support the scale estimation inside GPTQ #2784

merged 2 commits into from
Sep 4, 2024

Conversation

alexsu52
Copy link
Contributor

@alexsu52 alexsu52 commented Jul 4, 2024 •
edited
Loading

Changes

  • The scale estimation algorithm is used to calculate quantization parameters inside GPTQ.
  • Combination GPTQ and AWQ algorithm is supported.
  • The layer-wise scheduler is deterministic.

Reason for changes

Improve model compression accuracy for low bits.
image

Related tickets

ref: 143637

Tests

job/manual/job/post_training_weight_compression/152/

@github-actions github-actions bot added NNCF OpenVINO Pull requests that updates NNCF OpenVINO NNCF PTQ Pull requests that updates NNCF PTQ labels Jul 4, 2024
@alexsu52 alexsu52 force-pushed the as/gptq_scale_estimation branch from d6fd572 to 70381cc Compare September 2, 2024 07:27
@github-actions github-actions bot removed the NNCF OpenVINO Pull requests that updates NNCF OpenVINO label Sep 2, 2024
@alexsu52 alexsu52 force-pushed the as/gptq_scale_estimation branch 2 times, most recently from c4c7db4 to 32dcf13 Compare September 2, 2024 08:14
@github-actions github-actions bot added the NNCF OpenVINO Pull requests that updates NNCF OpenVINO label Sep 2, 2024
@alexsu52 alexsu52 force-pushed the as/gptq_scale_estimation branch from 32dcf13 to 488cacc Compare September 2, 2024 09:22
@alexsu52 alexsu52 marked this pull request as ready for review September 2, 2024 14:10
@alexsu52 alexsu52 requested a review from a team as a code owner September 2, 2024 14:10
@alexsu52 alexsu52 requested a review from ljaljushkin September 3, 2024 12:48
andreyanufr
andreyanufr approved these changes Sep 4, 2024
View reviewed changes
Copy link
Collaborator

@andreyanufr andreyanufr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dev SE and PR SE has the same ppl numbers.

@alexsu52 alexsu52 merged commit 03efd62 into openvinotoolkit:develop Sep 4, 2024
13 checks passed
@alexsu52 alexsu52 mentioned this pull request Sep 4, 2024
Merged
KodiaqQ pushed a commit that referenced this pull request Sep 5, 2024
@alexsu52
Support the scale estimation inside GPTQ (#2948)
5cee8b6 
PR: #2784
@alexsu52 alexsu52 mentioned this pull request Sep 5, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@andreyanufr andreyanufr andreyanufr approved these changes

@ljaljushkin ljaljushkin ljaljushkin approved these changes

Assignees
No one assigned
Labels
NNCF OpenVINO Pull requests that updates NNCF OpenVINO NNCF PTQ Pull requests that updates NNCF PTQ
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@alexsu52 @ljaljushkin @andreyanufr