[FoldConstant] Create Interpreter for each constant subgraph

[anijain2305]

This is related to https://discuss.tvm.ai/t/vm-slow-compilation-of-tf-object-detection-models/7479

For TF object detection models, a module has many functions (TF SSD mobilenet has 48), where a couple of functions are pretty huge (more than 10k call nodes). FoldConstant, because it is a function pass, is called for each function in the module. However, FoldConstant also creates an Interpreter on every invocation, which currently is based on the full mod and therefore expensive.

This PR creates an interpreter for each constant subgraph. I am not sure if this is the right way. The purpose of this PR is to start a discussion and identify if there is some other higher-level design issue that needs to be resolved.

With this PR, compilation time of

• SSD reduced from 955 to 453 seconds
• Faster RCNN reduced from 4630 to 1227 seconds

@zhiics @masahi @kevinthesun @icemelon9

[MarisaKirisame]

This look right.

[zhiics]

Thanks for the fix. lgtm

[zhiics]

Thanks @anijain2305 @MarisaKirisame

[junrushao]

It is consistent with your findings
CC: @were

[tmoreau89]

Interestingly this PR seems to have introduced a bug in the VTA image classification example: https://github.com/apache/incubator-tvm/commits/master/vta/tutorials/frontend/deploy_classification.py

To reproduce you can go back to d892881c4cc8c9a29bc03233aeac2b1532a9c6891 and the test passes.

Edit the config.cmake lines to run the VTA simulator:

echo 'set(USE_VTA_FSIM ON)' >> config.cmake
echo 'set(USE_LLVM llvm-config-10)' >> config.cmake

Go to 3684a51728e929576c93ff9887ac23e1b51582de (this PR), and the test fails with the following stack trace:

Stack trace:
  [bt] (0) 1   libmxnet.so                         0x000000011a0e92b0 mxnet::Storage::Get() + 4880
  [bt] (1) 2   libsystem_platform.dylib            0x00007fff690495fd _sigtramp + 29
  [bt] (2) 3   libvta_fsim.dylib                   0x0000000136ad14d0 _ZZN4dmlc16ThreadLocalStoreIN3tvm7runtime16VTAWorkspacePoolEE3GetEvE4inst$tlv$init + 0
  [bt] (3) 4   ???                                 0x0000000137dd1494 0x0 + 5232202900
  [bt] (4) 5   libtvm.dylib                        0x0000000131d09f3c tvm::relay::Interpreter::InvokePrimitiveOp(tvm::relay::Function const&, tvm::runtime::Array<tvm::runtime::ObjectRef, void> const&) + 3516
  [bt] (5) 6   libtvm.dylib                        0x0000000131d0824b tvm::relay::Interpreter::Invoke(tvm::relay::InterpreterClosure const&, tvm::runtime::Array<tvm::runtime::ObjectRef, void> const&, tvm::relay::Var const&) + 171
  [bt] (6) 7   libtvm.dylib                        0x0000000131d03481 tvm::relay::Interpreter::VisitExpr_(tvm::relay::CallNode const*) + 961
  [bt] (7) 8   libtvm.dylib                        0x0000000131d070f8 tvm::relay::ExprFunctor<tvm::runtime::ObjectRef (tvm::RelayExpr const&)>::InitVTable()::'lambda4'(tvm::runtime::ObjectRef const&, tvm::relay::ExprFunctor<tvm::runtime::ObjectRef (tvm::RelayExpr const&)>*)::__invoke(tvm::runtime::ObjectRef const&, tvm::relay::ExprFunctor<tvm::runtime::ObjectRef (tvm::RelayExpr const&)>*) + 24
  [bt] (8) 9   libtvm.dylib                        0x0000000131d05aaf tvm::NodeFunctor<tvm::runtime::ObjectRef (tvm::runtime::ObjectRef const&, tvm::relay::ExprFunctor<tvm::runtime::ObjectRef (tvm::RelayExpr const&)>*)>::operator()(tvm::runtime::ObjectRef const&, tvm::relay::ExprFunctor<tvm::runtime::ObjectRef (tvm::RelayExpr const&)>*) const + 255
Stack trace:
  [bt] (0) 1   libmxnet.so                         0x000000011a0e92b0 mxnet::Storage::Get() + 4880
  [bt] (1) 2   libsystem_platform.dylib            0x00007fff690495fd _sigtramp + 29
  [bt] (2) 3   ???                                 0x0000000000000012 0x0 + 18
  [bt] (3) 4   libtvm.dylib                        0x0000000131e652ee void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, tvm::runtime::threading::ThreadGroup::Impl::Impl(int, std::__1::function<void (int)>, bool)::'lambda'()> >(void*) + 62
  [bt] (4) 5   libsystem_pthread.dylib             0x00007fff69055109 _pthread_start + 148
  [bt] (5) 6   libsystem_pthread.dylib             0x00007fff69050b8b thread_start + 15
Stack trace:
  [bt] (0) 1   libmxnet.so                         0x000000011a0e92b0 mxnet::Storage::Get() + 4880
  [bt] (1) 2   libsystem_platform.dylib            0x00007fff690495fd _sigtramp + 29
  [bt] (2) 3   ???                                 0x0000000000000012 0x0 + 18
  [bt] (3) 4   libtvm.dylib                        0x0000000131e652ee void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, tvm::runtime::threading::ThreadGroup::Impl::Impl(int, std::__1::function<void (int)>, bool)::'lambda'()> >(void*) + 62
  [bt] (4) 5   libsystem_pthread.dylib             0x00007fff69055109 _pthread_start + 148
  [bt] (5) 6   libsystem_pthread.dylib             0x00007fff69050b8b thread_start + 15
Stack trace:
  [bt] (0) 1   libmxnet.so                         0x000000011a0e92b0 mxnet::Storage::Get() + 4880
  [bt] (1) 2   libsystem_platform.dylib            0x00007fff690495fd _sigtramp + 29
  [bt] (2) 3   ???                                 0x0000000000000012 0x0 + 18
  [bt] (3) 4   libtvm.dylib                        0x0000000131e652ee void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, tvm::runtime::threading::ThreadGroup::Impl::Impl(int, std::__1::function<void (int)>, bool)::'lambda'()> >(void*) + 62
  [bt] (4) 5   libsystem_pthread.dylib             0x00007fff69055109 _pthread_start + 148
  [bt] (5) 6   libsystem_pthread.dylib             0x00007fff69050b8b thread_start + 15