Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extend weight compression with INT8 symmetric scheme #2288

Merged
merged 6 commits into from
Dec 7, 2023

Conversation

l-bat
Copy link
Collaborator

@l-bat l-bat commented Nov 23, 2023

Changes

Added INT8_SYM compression mode

Reason for changes

INT8_SYM mode can provide a better performance and required for dynamic quantization

Related tickets

124823

Tests

Updated tests/openvino/native/quantization/test_weights_compression.py

@l-bat l-bat requested a review from a team as a code owner November 23, 2023 21:20
@github-actions github-actions bot added documentation Improvements or additions to documentation NNCF PT Pull requests that updates NNCF PyTorch NNCF OpenVINO Pull requests that updates NNCF OpenVINO NNCF PTQ Pull requests that updates NNCF PTQ labels Nov 23, 2023
@openvino-nncf-ci openvino-nncf-ci added the API Public API-impacting changes label Nov 23, 2023
Copy link

codecov bot commented Nov 23, 2023

Codecov Report

Merging #2288 (479d4f4) into develop (08cd0f7) will decrease coverage by 4.80%.
Report is 8 commits behind head on develop.
The diff coverage is 100.00%.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #2288      +/-   ##
===========================================
- Coverage    90.76%   85.96%   -4.80%     
===========================================
  Files          486      487       +1     
  Lines        43764    43805      +41     
===========================================
- Hits         39723    37659    -2064     
- Misses        4041     6146    +2105     
Flag Coverage Δ
COMMON 15.86% <10.81%> (+0.05%) ⬆️
ONNX ?
OPENVINO 38.71% <89.18%> (+0.02%) ⬆️
TENSORFLOW 30.01% <10.81%> (-0.01%) ⬇️
TORCH 62.70% <24.32%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
nncf/parameters.py 100.00% <100.00%> (ø)
...ization/algorithms/weight_compression/algorithm.py 95.91% <ø> (ø)
...ntization/algorithms/weight_compression/backend.py 100.00% <ø> (ø)
.../algorithms/weight_compression/openvino_backend.py 98.23% <100.00%> (-0.03%) ⬇️
nncf/quantization/quantize_model.py 82.89% <100.00%> (-1.83%) ⬇️
nncf/torch/quantization/quantize_model.py 91.42% <100.00%> (ø)

... and 44 files with indirect coverage changes

@l-bat l-bat requested a review from ljaljushkin November 29, 2023 15:44
Copy link
Contributor

@ljaljushkin ljaljushkin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Special thanks for refactoring and fixing issue with printing statistics

INT4_SYM = "int4_sym"
INT4_ASYM = "int4_asym"
NF4 = "nf4"
INT8 = "int8" # Deprecated mode
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think about leaving INT8 as an alias for INT8_ASYM?

@alexsu52 alexsu52 merged commit 6d08f52 into openvinotoolkit:develop Dec 7, 2023
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
API Public API-impacting changes documentation Improvements or additions to documentation NNCF OpenVINO Pull requests that updates NNCF OpenVINO NNCF PT Pull requests that updates NNCF PyTorch NNCF PTQ Pull requests that updates NNCF PTQ
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants