[OpFusion] Make the max number of fused ops configurable

[masahi]

This adds support for configuring the number of ops in one fused function. The motivation is for a use case like
hummingbird where there can be unbounded number of operations fused into one function, depending on the dataset. This leads to huge compilation time.

More details explained in
https://discuss.tvm.ai/t/aggressive-operator-fusion-and-its-consequence-of-huge-inlined-tir-expression/7687

There is already a hard coded constant kMaxFusedOps and the condition to check the number of fused ops:

https://github.com/apache/incubator-tvm/blob/a8e44710c6472a2ee5cb66283c7f5e77f4e4ca0d/src/relay/transforms/fuse_ops.cc#L82

https://github.com/apache/incubator-tvm/blob/a8e44710c6472a2ee5cb66283c7f5e77f4e4ca0d/src/relay/transforms/fuse_ops.cc#L678

But this condition is simply not correct, since for a diamond structure there are multiple paths connecting a child node (group_node above) and its dom parent. Correctly calculating the number of fused nodes requires more care, and I believe my implementation is correct.

Please review @tqchen @zhiics

[junrushao]

LGTM! Could you also share the compilation time impact of hummingbird by tuning this option? Thanks!

[masahi]

LGTM! Could you also share the compilation time impact of hummingbird by tuning this option? Thanks!

It's hard to say. I've only tested on my laptop, but without this change (unbounded fuse depth) I cannot compile for even a tiny dataset of size (20, 10) in 1 hour. Compiling for a real world dataset is infeasible.

With configurable depth, the time it takes to run relay.build on the dataset of size (20, 10) is roughly:

• max fuse depth = 15: 1 sec
• max fuse depth = 50: 10 sec

Note that with this change, fusion is no longer a problem for compiling hummingbird models, no matter what the dataset size is. I can now compile for the dataset of size (500, 10), but (1000, 10) gives a segfault for a different reason not related to fusion.

[junrushao]

Thanks for the information @masahi! It is definitely helpful!

[tqchen]

Thanks @masahi @zhiics @junrushao1994 !