[PTQ][Torch][KQV self attention] Align FQ placement between OV and Torch backend #2166
+345
−40
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## develop #2166 +/- ##
===========================================
+ Coverage 36.62% 36.67% +0.05%
===========================================
Files 486 486
Lines 43373 43417 +44
===========================================
+ Hits 15885 15923 +38
- Misses 27488 27494 +6
|
daniil-lyakhov
force-pushed
the
dl/torch/patterns_alignment
branch
from
7abfdaf to
9698e6c
Compare
daniil-lyakhov
changed the title
|
Please run post_training build and update metrics if needed. |
github-actions
bot
added
NNCF Common
daniil-lyakhov
force-pushed
the
dl/torch/patterns_alignment
branch
2 times, most recently
from
b7816df to
c6dcc08
Compare
daniil-lyakhov
commented
Oct 6, 2023
|
PTQ conformance test finished successfully (build 175), metrics are updated accordingly |
daniil-lyakhov
changed the title
daniil-lyakhov
changed the title
daniil-lyakhov
force-pushed
the
dl/torch/patterns_alignment
branch
from
9f6ebe0 to
9837d2b
Compare
AlexanderDokuchaev
approved these changes
Oct 12, 2023
alexsu52
reviewed
Oct 13, 2023
KodiaqQ
requested changes
Oct 13, 2023
tests/common/data/reference_graphs/passes/dropout_synthetic_model.dot
Outdated
Show resolved
Hide resolved
KodiaqQ
approved these changes
Oct 18, 2023
daniil-lyakhov
force-pushed
the
dl/torch/patterns_alignment
branch
from
4dc9572 to
03c2b5e
Compare
alexsu52
approved these changes
Oct 19, 2023
daniil-lyakhov
force-pushed
the
dl/torch/patterns_alignment
branch
2 times, most recently
from
6db6937 to
ce14175
Compare
ksilligan
added a commit
to ksilligan/nncf
that referenced
this pull request
Nov 6, 2023
* Bump `numpy` version upper bound (openvinotoolkit#2202) ### Changes Bump `numpy` version. ### Reason for changes Current version causes issues in OpenVINO CI: https://github.com/intel-innersource/frameworks.ai.openvino.ci.product-configs/pull/2803 ### Related tickets 122802 * Add links to OneFormer and InstructPix2Pix notebooks (openvinotoolkit#2201) ### Changes Added links to `OneFormer` and `InstructPix2Pix` OpenVINO notebooks * Added documentation for NF4 mode (openvinotoolkit#2194) Co-authored-by: Alexander Kozlov <[email protected]> * [Torch] SE block HW fusing pattern is presented (openvinotoolkit#2177) ### Changes * SE block ignored pattern is presented for Torch backend  * SE block ignored pattern for OV backend is adjusted * NON_PATTERN_NODE_WITH_TYPE is introduced in the graph matcher ### Reason for changes To align FQ layout between TORCH and OV backend for efficientnet_b0 model To fix FQ placement for SE block: Before:  After:  ### Related tickets 121647 ### Tests * test_non_pattern_node_with_type for the graph matcher Post training quantization manual build 180 | Model | Backend | Metric name | Metric value | Metric diff | Num FQ | RAM MiB | Quant. time | Total time | Status | |-------|----------------------|-------------|--------------|-------------|---------|---------|-------------|------------|---------| | 1 | timm/efficientnet_b0 | OV | Acc@1 | 0.7688 | -0.0042 | 92 | 1033 | 0:00:25 | 1:03:58 | | | 2 | timm/efficientnet_b0 | TORCH | Acc@1 | 0.7680 | -0.0049 | 92 | 892 | 0:00:48 | 1:04:27 | * pattern manager tests * [PTQ][Torch][KQV self attention] Align FQ placement between OV and Torch backend (openvinotoolkit#2166) ### Changes * "unbing" and "__matmul__" ops are added to torch patterns * Dropout removing pass is added to function `transform_to_inference_graph` * LayerNorm and GroupNorm metatypes are added to ignored metatypes in MinMax algorithm ### Reason for changes To align quantization placement between OV and Torch backends for the following models: * timm/crossvit_9_240 * timm/deit3_small_patch16_224 * timm/swin_base_patch4_window7_224 ### Related tickets 121647 ### Tests * test_metatypes_to_ignore for quantization propagation solver * Added ov_version parameter to the env tests (openvinotoolkit#2209) ### Changes - Added `ov_version` variable for the pytest in `test_examples`; - Added `ov_version` variable for the pytest in the `test_install_*`; ### Reason for changes - More flexible tests ### Related tickets - 122860 ### Tests - Updated * Do not test common code in backend-specific test targets (openvinotoolkit#2212) ### Changes As stated in the title ### Reason for changes Common code has an own target now which is run using GH actions, no reason to run these tests additionally in the backend-specific targets. ### Related tickets TBA ### Tests Existing precommit scope * Enable SmoothQuant only for OV backend (openvinotoolkit#2208) ### Changes Run the SmoothQuant and ChannelAlignment algorithms only for the OpenVINO backend. ### Reason for changes Only the OpenVINO backend supports SmoothQuant and ChannelAlignment algorithms. ### Related tickets N/A ### Tests N/A * [OV] Add caching of OMZ models (openvinotoolkit#2214) ### Changes Added cache_dir for OMZ models ### Reason for changes * Сhanges made according to the requirements of the validation team ### Related tickets <!--- Post the numerical ID of the ticket, if available --> ### Tests <!--- How was the correctness of changes tested and whether new tests were added --> * QuantizationPreset.MIXED for ModelType.TRANSFORMER by default (openvinotoolkit#2216) ### Changes Made the QuantizationPreset.MIXED preset for ModelType.TRANSFORMER by default. ### Reason for changes Quantization of transformer-based models with QuantizationPreset.MIXED preset shows the best accuracy w/o performance degradation. ### Related tickets ref: 123235 ### Tests test_create_nncf_config test_quantization_preset * Updated reference for 2023.2.0-12979-04c766e9f17 (openvinotoolkit#2219) - [ ] pass nncf + openvino-nightly tests - build#7 ### Changes <!--- What was changed (briefly), how to reproduce (if applicable), what the reviewers should focus on --> ### Reason for changes <!--- Why should the change be applied --> ### Related tickets <!--- Post the numerical ID of the ticket, if available --> ### Tests <!--- How was the correctness of changes tested and whether new tests were added --> * Using ov.save_model(...) in PTQ examples (openvinotoolkit#2156) ### Changes Using ov.save_model(...) in PTQ examples ### Reason for changes ov.save_model(...) is introduced in OpenVINO==2023.1.0 ### Related tickets N/A ### Tests examples * [PTQ] Remove insert_null_biases pass (openvinotoolkit#2217) ### Changes - Removed `insert_bull_biases` method ### Reason for changes - Inserting is not necessary since correcting the null biases is limited by magnitude. - Inserting of the new increases binary file size. ### Related tickets - 120843 * Fix index out of range (openvinotoolkit#2224) ### Changes Fix index out of range ### Reason for changes File "nncf/nncf/quantization/algorithms/pipeline.py", line 119, in run_step current_model = pipeline_step[-1].apply(current_model, current_graph, step_statistics) IndexError: list index out of range ### Related tickets N/A ### Tests N/A * Added `keepdims` argument for min,max functions (openvinotoolkit#2225) ### Changes as stated in the title ### Reason for changes Follow-up from comment openvinotoolkit#2218 (comment) ### Related tickets n/a ### Tests test_fn_max test_fn_min test_max test_min * [PTQ] Avg pool is added as ignored metatype for TRANSFORMER model_type (openvinotoolkit#2228) ### Changes [PTQ] Avg pool is added as ignored metatype for TRANSFORMER model_type in TORCH/OV/ONNX backends ### Reason for changes * To align PTQ backends as for example TORCH avg_pool is mapping to reduce_mean in OV, for timm/visformer_small in particular ### Related tickets 110985 119910 ### Tests * Remove pylint (openvinotoolkit#2205) ### Changes Pylint was removed from the make targets and from code. Added `.pylintrc` to `.gitignore` so that developers could have their own `.pylintrc` if they still want to use pylint in informational capacity locally. ### Reason for changes Following through with the decision to use ruff as the main linter. Most of the pylint rules not covered by ruff will be covered by mypy in the future. ### Related tickets N/A ### Tests Existing test scope * Use torch 2.1 and remove torch 1.13 (openvinotoolkit#2211) ### Changes As stated in the title ### Reason for changes Keeping up with the latest torch version ### Related tickets N/A ### Tests Existing torch test scope * [Graph visualization] Reserved dot symbols are replaced for each graph visualization (openvinotoolkit#2231) ### Changes Visualization of debug graph is fixed by replacing : to ^ ### Reason for changes To fix graph visualization for OV backend ### Related tickets ### Tests * [ONNX][OV] Update SE block pattern (openvinotoolkit#2229) ### Changes Add SE block ignored pattern for ONNX;  Add hsigmoid to OV SE block pattern OV new quantization scheme for mobilentv3  ONNX efficientnet-b0:  ONNX mobilentv3:  ### Reason for changes Improve accuracy metrics for modes with SE block ### Related tickets 123692 123691 ### Tests Models from conformance test are tested manually * Fix github actions codecov uploader for ONNX * Fix github actions codecov uploader for ONNX and COMMON * Update upload_coverage_for_develop.yml --------- Co-authored-by: Przemyslaw Wysocki <[email protected]> Co-authored-by: Liubov Talamanova <[email protected]> Co-authored-by: Lyalyushkin Nikolay <[email protected]> Co-authored-by: Alexander Kozlov <[email protected]> Co-authored-by: Daniil Lyakhov <[email protected]> Co-authored-by: Nikita Malinin <[email protected]> Co-authored-by: Vasily Shamporov <[email protected]> Co-authored-by: Andrey Churkin <[email protected]> Co-authored-by: Alexander Suslov <[email protected]> Co-authored-by: Aleksei Kashapov <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes
transform_to_inference_graphReason for changes
To align quantization placement between OV and Torch backends for the following models:
Related tickets
121647
Tests