[relay][external codegen] outline and inline lifted functions for external codegen #4996
Conversation
zhiics
force-pushed
the
inline_partitioned_functions
branch
from
58e5b5e to
b97bbc6
Compare
zhiics
force-pushed
the
inline_partitioned_functions
branch
from
b97bbc6 to
6d6d332
Compare
|
How about adding batch norm to the test to make sure it is still there after opt passes? |
zhiics
force-pushed
the
inline_partitioned_functions
branch
from
4dcf21d to
8554928
Compare
|
@masahi Good point. Done |
|
Will wait for @mbaret's approval. |
|
Also need to wait for the github bug to be resolved |
|
@zhiics It looks like this makes it so we can't run any passes on external functions. Is this the intended behaviour, and if so are we intending to add a mechanism at some point to allow for passes on external functions? |
|
Yes, this is intended. We plan to do it. For example, we may want to have a build pipeline that executes target independent passes first and then target dependent passes. We can partition after platform independent passes and rely on external compilers to take care of them. You can do this right now manually. For example, you can run constant folding, CSE, etc, before partitioning. |
|
Beforehand, we had been running some passes after partitioning (for instance a layout transform for ACL functions). It looks like we can't even force these to run now though as no function passes are allowed to touch external functions. We can't run the passes before partitioning because it will transform the entire graph, not just the bits for external partitioning. |
|
I personally don't think TVM should be aware of accelerator specific transformations because they are very hardware specific. For example, we don't want this on TRT, dnnl, and many other accelerators. However, you can still do this even now. You can directly invoke layout transformation without using the pass infra. |
|
It's possible we're doing something wrong, but the function LowerExternalFunctions in compile_engine produces modules just containing external functions. We want to call passes on these modules from our ACL-specific codegen (so they'll only affect our codegen, not all TVM functions). However, it looks like the passes are skipping all the functions in these modules because they don't use the default compiler. Is there another way we can manually run the passes? |
|
Can you post the questions on discussion forum? |
|
Yep, I'll try and give a better explanation there in the morning. |
…ernal codegen (apache#4996) * outline and inline lifted functions for external codegen * add batch_norm test * test batch_norm inline
…ernal codegen (apache#4996) * outline and inline lifted functions for external codegen * add batch_norm test * test batch_norm inline
This PR outlines the functions that will be handled by external codegen and then inline them back to avoid the optimizations from Relay. All function level passes are handled directly from the pass manager. A few module level passes needs to detect if a function uses default compiler or not. This is because these passes take the functions internally and they are invisible to the pass manager.