[Relay][PASS] CombineParallelConv2D generating new kernel and causing performance issue

[kevinthesun]

Since some developers have the similar issue, I open this issue to track the status of discuss topic https://discuss.tvm.ai/t/relay-alter-op-layout-pass-regression/1870/7. CombineParallelConv2D pass generates new workloads and in some cases cause performance regression. Another issue is the combination of this pass with AutoTVM. This pass might need to be done before AutoTVM is launched.

[yzhliu]

I think this pass can also be considered as a part of graph tuner.

[kevinthesun]

@yzhliu Yes. Maybe we can add a call of this pass in autotvm before any other workload extraction/graph analysis is executed.

[tqchen]

fixed by #2961

[merrymercy]

Is this issue solved? We still don't call CombineParallelConv2d before workload extraction.
Reopen it.

[vinx13]

CombineParallelConv2d doesn't always give better performance. We should extract workload with and without this pass and then decide whether to apply this pass.

[kevinthesun]

Is it possible to move this pass to higher opt level? Looks like it's not quite suitable to put it together with other mandatory optimization passes in opt 3.

[vinx13]

@kevinthesun Yes we can move it to opt 4 for now

[vinx13]

Let me summarize the actions here:

• Move CombineParallelConv2D to opt level 4
• Extract autotvm tuning tasks: either require users to manually apply this pass before task extraction, or automatically extract tasks with & without this pass
• Apply this pass if the performance is better, this can possibly be integrated with graph tuner

Would like to hear your comments @kevinthesun @merrymercy @yzhliu @tqchen

[kevinthesun]

@vinx13 Sounds good to me. For autotvm and graph tuning parts, we can update the tutorial to add one more parameter to indicate whether to apply this pass before tuning.

[kevinthesun]

CombineParallelConv2D moved to opt level 4. We can have another thread to deal with integration with autotvm.