-
Notifications
You must be signed in to change notification settings - Fork 3.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Relay][Quantization] Fix add_rewrite and UnifyDTypeScale #3534
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for the patch! LGTM
@ZihengJiang can you review this PR before I merge it? |
LOG(FATAL) << "should not touch here."; | ||
dtype = cfg->dtype_activation; | ||
} | ||
for (size_t i = 0; i < ret.size(); ++i) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hey @vinx13 , I guess we can remove this part with rewrite_for_vta
, which will insert cast
and stop_fusion
in the end of residual block
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ZihengJiang This part is needed when rewrite_for_vta
is not enabled (store_lowbit_output=False
), or we can remove this part in rewrite_for_vta
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@vinx13 Current Realize
pass has been used to achieve multiple goals: insert stop_fusion
; decide datatype; unify scales for some operator, etc. It's better to decouple things for passes, I would also like to make rewrite_for_vta
and store_lowbit_output
enabled by default.
* [Relay][Quantization] Fix issue introduced in apache#3135 * Recover StopFusion * Fix fmultiref * Fix lint
* [Relay][Quantization] Fix issue introduced in apache#3135 * Recover StopFusion * Fix fmultiref * Fix lint
This PR fixed a few broken things due to previous changes:
UnifyDTypeScale
ForwardRewrite
argumentsrelay._quantize.annotate
(we moved to mod api)See https://discuss.tvm.ai/t/quantization-broken-due-to-pr-3135/3237/6
cc @ZihengJiang @tmoreau89