[Torch] Serialize and load NNCF transformations #2531
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
documentation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## develop #2531 +/- ##
============================================
- Coverage 91.20% 29.88% -61.32%
============================================
Files 493 495 +2
Lines 45519 45961 +442
============================================
- Hits 41514 13734 -27780
- Misses 4005 32227 +28222
... and 323 files with indirect coverage changes
Flags with carried forward coverage won't be shown. Click here to find out more.
|
daniil-lyakhov
force-pushed
the
dl/ptq/load_state
branch
from
922170d to
953f395
Compare
daniil-lyakhov
force-pushed
the
dl/ptq/load_state
branch
2 times, most recently
from
86d1356 to
e8f7f15
Compare
daniil-lyakhov
force-pushed
the
dl/ptq/load_state
branch
6 times, most recently
from
d616ca0 to
ec9e451
Compare
github-actions
bot
removed
the
documentation
daniil-lyakhov
force-pushed
the
dl/ptq/load_state
branch
4 times, most recently
from
42d0b67 to
c95fcf6
Compare
daniil-lyakhov
changed the title
daniil-lyakhov
changed the title
daniil-lyakhov
commented
Mar 28, 2024
daniil-lyakhov
force-pushed
the
dl/ptq/load_state
branch
from
5f834bb to
59ab905
Compare
AlexanderDokuchaev
pushed a commit
that referenced
this pull request
Apr 2, 2024
Preparation for #2531 ### Changes 1) `PTQuantizerInsertionCommand` is removed and replaced with create_quantizer_insertion_command function 2) `SharedFNInsertionCommand` updates with one new attribute: compression_module_type 3) `ExtraOpCallHook` doesn't require context in constructor anymore 4) Multidevice support is moved from `apply_quantizers_insertion_commands_transformation` to `apply_insertion_transformation` ### Reason for changes 1) To make it easier to store and restore commands: less commands - less amount of adapters are needed 2) To make it possible to express `PTQuantizerInsertionCommand` by `SharedFNInsertionCommand` 3) To make it possible to create `ExtraOpCallHook` outside of the `PTModelTransformer` 4) To unify multidevice support for all insertion operations ### Related tickets 2531 ### Tests 1)`test_quantizer_insertion_transformation` is updated 2) - 3) `test_shared_fn_insertion_point` is updated 4) `test_pt_insertion_command` is introduced
daniil-lyakhov
force-pushed
the
dl/ptq/load_state
branch
from
59ab905 to
8ec693f
Compare
daniil-lyakhov
force-pushed
the
dl/ptq/load_state
branch
2 times, most recently
from
808d068 to
f14aa92
Compare
API code moved to a separate PR
daniil-lyakhov
force-pushed
the
dl/ptq/load_state
branch
4 times, most recently
from
846fe06 to
fa077fa
Compare
alexsu52
reviewed
May 2, 2024
nncf/torch/layer_utils.py
Outdated
| """ | ||
|
|
||
| @abstractmethod | ||
| def get_state(self) -> Dict[str, Any]: |
There was a problem hiding this comment.
What do you think about get_config, from_config to align with high-level API?
alexsu52
approved these changes
May 6, 2024
alexsu52
pushed a commit
that referenced
this pull request
May 8, 2024
### Changes `_compression_lr_multiplier` attribute introduced in #2531 is removed from the `CompressionParameter` ### Reason for changes `_compression_lr_multiplier` makes the `CompressionParameter` a stateful parameter which for some reason does not work properly in distributed/dataparallel mode ### Tests torch_nightly/213/ - finished successfully
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
On top of #2584 #2595
Changes
serialize_transformationsandload_transformationsfunctions are introduced:serialize_transformationscould serializePTTransformationLayoutto a dict which could be serialized by json; serialized transformation could be recovered by theload_transformationsfunction.StatefullTorchModuleInterfaceis introduced to make it possible to serialize all compression modules for quantization, sparisification, weights compression and pruning algorithmsReason for changes
nncf.quantizeand quantizes initializationRelated tickets
129586
Tests