Refactor Pass for fused_conv #48848
Refactor Pass for fused_conv #48848
Conversation
|
你的PR提交成功，感谢你对开源项目的贡献! |
|
@zyfncg Hi, We just started our automatic testing process on this PR (should take 1-2 days). We will let you know if all is fine or on problems found |
| QuantizeConv(graph, "fused_conv2d", true /*with_residual_data*/); | ||
| QuantizeConv(graph, "fused_conv2d", false /*with_residual_data*/); | ||
| QuantizeConv(graph, "conv2d", true /*with_residual_data*/); | ||
| QuantizeConv(graph, "conv2d", false /*with_residual_data*/); |
There was a problem hiding this comment.
This pass works on convolution ops that had scales calculated during int8_scales_calculation_pass because it needs attributes Scale_weights to quantize weights and Bias_scales to quantize bias if it exists. You have added the code linked below that changes conv2d to fused_conv2d there: https://github.com/PaddlePaddle/Paddle/pull/48848/files?show-viewed-files=true&file-filters%5B%5D=#diff-202dc497cf83197c83edced692ade07f943a14605f0b2cefc9f10a36a244212bR126-R127.
So we should be able to safely assume that this pass - params_quantization_mkldnn_pass - can work only on fused_conv2d.
There was a problem hiding this comment.
Thank you, I think you are right. The conv2d in params_quantization_mkldnn_pass has been removed.
| attrs { | ||
| name: "Scale_weights" | ||
| type: FLOATS | ||
| } |
There was a problem hiding this comment.
There can also be "Bias_scales" optionally.
| if ((prev_op_nodes[0]->Op()->Type() != "conv2d" && | ||
| prev_op_nodes[0]->Op()->Type() != "fused_conv2d") || | ||
| is_not_conv_mkldnn) { | ||
| LOG(WARNING) << "This fuse pass supports only conv2d(mkldnn) | " |
There was a problem hiding this comment.
We try to gradually replace MKLDNN with oneDNN
| @@ -39,14 +39,44 @@ def { | |||
| name: "data_format" | |||
| type: STRING | |||
| } | |||
| } | |||
| extra { | |||
There was a problem hiding this comment.
Shouldn't conv2d.pbtxt and conv_op.cc be also adjusted? Or this will be done in next PR?
There was a problem hiding this comment.
Yes, this will be done in next PR.
|
@zyfncg I have just got results from analysis of this PR. And we noticed no perf regression and perf improvement on bf16 convolutional models . |
PR types
Others
PR changes
Others
Describe
PR47579将conv2d算子拆分为基础conv2d和fused_conv2d两个算子，拆分后需要对相关的Pass逻辑进行修改适配。
本PR在PR47579的基础上对剩余的conv算子相关Pass进行调整，修改的Pass包括：
conv_activation_mkldnn_fuse_passconv_affine_channel_mkldnn_fuse_passconv_elementwise_add_mkldnn_fuse_passconv_bn_fuse_passdepthwise_conv_bn_fuse_passint8_scale_calculation_mkldnn_passparams_quantization_mkldnn_passPass调整的策略为：
conv2d算子的替换为fused_conv2d算子，避免conv2d算子继续使用fuse相关的参数和功能。conv2d作为输入算子的基础上增加对fused_conv2d作为输入的支持。避免出现算子被Pass处理为fused_conv2d后无法被其他Pass所处理的情况。In PR47579
conv2doperator was split into two operatorsfused_conv2dand baseconv2d. We need to update the related Passes, so the remaining related Passes are modified in this PR:conv_activation_mkldnn_fuse_passconv_affine_channel_mkldnn_fuse_passconv_elementwise_add_mkldnn_fuse_passconv_bn_fuse_passdepthwise_conv_bn_fuse_passint8_scale_calculation_mkldnn_passparams_quantization_mkldnn_passThe strategy of modifying Pass:
conv2dafter fusion withfused_conv2d. By doing this, the fuse-related parameters in conv2d operator will not be used, so we could delete them in the future.fused_conv2das input operator for some related Passes. Theconv2dcan be processed recursively in the fuse Passes, so this can avoid the situation that thefused_conv2dcannot be processed by other Passes after being fused asfused_conv2dby current Pass.