[microTVM] Add support for host-driven AoT Executor #11044
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ape encoded in metadata appears to be incorrect;
…); test_aot_executor() is now passing;
…ot to include metadata.h c++ code
…der to avoid c/c++ issues and remove the need for the TVM_IS_CPP_RUNTIME define
areusch
approved these changes
May 10, 2022
mehrdadh
pushed a commit
to mehrdadh/tvm
that referenced
this pull request
May 16, 2022
* Generate AOT Metadata when targeting C runtime and packed API. * Also copy metadata.h and metadata_base.h to standalone_crt. * add support for get_input_index as well as setting up get_input_info as unsupported * add support for tvm.aot_executor.create in C runtime * changes in-progress to unit tests * Include get_c_metadata in emitted function list * make CRT error codes generic for graph or AoT executor, fix AoT lib link order * add AoT executor creation and initializaion, as well as support for get_input_index() * add allocation of inputs, outputs, and pools; add get_input(), but shape encoded in metadata appears to be incorrect; * add support to test_aot_executor for get_input() * fix numpy array shape so that get_input() works properly * implement run(), get_output(), get_num_inputs(), and get_num_outputs(); test_aot_executor() is now passing; * fix up some issues from rebase with main * clean up logging and test_graph_executor() * lint clean-up * more lint clean-up * fix i386 build errors * first set of changes addressing PR feedback * more PR feedback: device pass-by-value, docstring entries, return variable name * add mangling of get_c_metadata() name to avoid function name collisions * only mangle get_c_metadata() when using C runtime * add static specifier to all kTvmgenMetadata variables to avoid namespace collisions * use TVM_IS_CPP_RUNTIME preprocessor define to deteremine whether or not to include metadata.h c++ code * add TVM_IS_CPP_RUNTIME define for cpptest * add TVM_IS_CPP_RUNTIME to apps/bundle_deploy * add TVM_IS_CPP_RUNTIME web/Makefile * update number of expected generated C files for AoT source files * break out metadata data structures into separate metadata_types.h header to avoid c/c++ issues and remove the need for the TVM_IS_CPP_RUNTIME define * remove TVM_IS_CPP_RUNTIME from web makefile * fix metadata.h include-order lint issue * correct error mask bits * address PR feedback * trigger build * trigger build * trigger build * trigger build * add alternate name for test_graph_executor() too see if it runs in CI * fix lint * revert alternate test code Co-authored-by: Andrew Reusch <[email protected]>
shtinsa
pushed a commit
to Deelvin/tvm
that referenced
this pull request
May 17, 2022
* Generate AOT Metadata when targeting C runtime and packed API. * Also copy metadata.h and metadata_base.h to standalone_crt. * add support for get_input_index as well as setting up get_input_info as unsupported * add support for tvm.aot_executor.create in C runtime * changes in-progress to unit tests * Include get_c_metadata in emitted function list * make CRT error codes generic for graph or AoT executor, fix AoT lib link order * add AoT executor creation and initializaion, as well as support for get_input_index() * add allocation of inputs, outputs, and pools; add get_input(), but shape encoded in metadata appears to be incorrect; * add support to test_aot_executor for get_input() * fix numpy array shape so that get_input() works properly * implement run(), get_output(), get_num_inputs(), and get_num_outputs(); test_aot_executor() is now passing; * fix up some issues from rebase with main * clean up logging and test_graph_executor() * lint clean-up * more lint clean-up * fix i386 build errors * first set of changes addressing PR feedback * more PR feedback: device pass-by-value, docstring entries, return variable name * add mangling of get_c_metadata() name to avoid function name collisions * only mangle get_c_metadata() when using C runtime * add static specifier to all kTvmgenMetadata variables to avoid namespace collisions * use TVM_IS_CPP_RUNTIME preprocessor define to deteremine whether or not to include metadata.h c++ code * add TVM_IS_CPP_RUNTIME define for cpptest * add TVM_IS_CPP_RUNTIME to apps/bundle_deploy * add TVM_IS_CPP_RUNTIME web/Makefile * update number of expected generated C files for AoT source files * break out metadata data structures into separate metadata_types.h header to avoid c/c++ issues and remove the need for the TVM_IS_CPP_RUNTIME define * remove TVM_IS_CPP_RUNTIME from web makefile * fix metadata.h include-order lint issue * correct error mask bits * address PR feedback * trigger build * trigger build * trigger build * trigger build * add alternate name for test_graph_executor() too see if it runs in CI * fix lint * revert alternate test code Co-authored-by: Andrew Reusch <[email protected]>
shingjan
pushed a commit
to shingjan/tvm
that referenced
this pull request
May 17, 2022
* Generate AOT Metadata when targeting C runtime and packed API. * Also copy metadata.h and metadata_base.h to standalone_crt. * add support for get_input_index as well as setting up get_input_info as unsupported * add support for tvm.aot_executor.create in C runtime * changes in-progress to unit tests * Include get_c_metadata in emitted function list * make CRT error codes generic for graph or AoT executor, fix AoT lib link order * add AoT executor creation and initializaion, as well as support for get_input_index() * add allocation of inputs, outputs, and pools; add get_input(), but shape encoded in metadata appears to be incorrect; * add support to test_aot_executor for get_input() * fix numpy array shape so that get_input() works properly * implement run(), get_output(), get_num_inputs(), and get_num_outputs(); test_aot_executor() is now passing; * fix up some issues from rebase with main * clean up logging and test_graph_executor() * lint clean-up * more lint clean-up * fix i386 build errors * first set of changes addressing PR feedback * more PR feedback: device pass-by-value, docstring entries, return variable name * add mangling of get_c_metadata() name to avoid function name collisions * only mangle get_c_metadata() when using C runtime * add static specifier to all kTvmgenMetadata variables to avoid namespace collisions * use TVM_IS_CPP_RUNTIME preprocessor define to deteremine whether or not to include metadata.h c++ code * add TVM_IS_CPP_RUNTIME define for cpptest * add TVM_IS_CPP_RUNTIME to apps/bundle_deploy * add TVM_IS_CPP_RUNTIME web/Makefile * update number of expected generated C files for AoT source files * break out metadata data structures into separate metadata_types.h header to avoid c/c++ issues and remove the need for the TVM_IS_CPP_RUNTIME define * remove TVM_IS_CPP_RUNTIME from web makefile * fix metadata.h include-order lint issue * correct error mask bits * address PR feedback * trigger build * trigger build * trigger build * trigger build * add alternate name for test_graph_executor() too see if it runs in CI * fix lint * revert alternate test code Co-authored-by: Andrew Reusch <[email protected]>
SebastianBoblest
pushed a commit
to SebastianBoblest/tvm
that referenced
this pull request
May 27, 2022
* Generate AOT Metadata when targeting C runtime and packed API. * Also copy metadata.h and metadata_base.h to standalone_crt. * add support for get_input_index as well as setting up get_input_info as unsupported * add support for tvm.aot_executor.create in C runtime * changes in-progress to unit tests * Include get_c_metadata in emitted function list * make CRT error codes generic for graph or AoT executor, fix AoT lib link order * add AoT executor creation and initializaion, as well as support for get_input_index() * add allocation of inputs, outputs, and pools; add get_input(), but shape encoded in metadata appears to be incorrect; * add support to test_aot_executor for get_input() * fix numpy array shape so that get_input() works properly * implement run(), get_output(), get_num_inputs(), and get_num_outputs(); test_aot_executor() is now passing; * fix up some issues from rebase with main * clean up logging and test_graph_executor() * lint clean-up * more lint clean-up * fix i386 build errors * first set of changes addressing PR feedback * more PR feedback: device pass-by-value, docstring entries, return variable name * add mangling of get_c_metadata() name to avoid function name collisions * only mangle get_c_metadata() when using C runtime * add static specifier to all kTvmgenMetadata variables to avoid namespace collisions * use TVM_IS_CPP_RUNTIME preprocessor define to deteremine whether or not to include metadata.h c++ code * add TVM_IS_CPP_RUNTIME define for cpptest * add TVM_IS_CPP_RUNTIME to apps/bundle_deploy * add TVM_IS_CPP_RUNTIME web/Makefile * update number of expected generated C files for AoT source files * break out metadata data structures into separate metadata_types.h header to avoid c/c++ issues and remove the need for the TVM_IS_CPP_RUNTIME define * remove TVM_IS_CPP_RUNTIME from web makefile * fix metadata.h include-order lint issue * correct error mask bits * address PR feedback * trigger build * trigger build * trigger build * trigger build * add alternate name for test_graph_executor() too see if it runs in CI * fix lint * revert alternate test code Co-authored-by: Andrew Reusch <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
It is currently not possible to use the AoT executor with microTVM over the RPC channel. Add support for host-driven AoT executor using the generated AoT metadata to allocate working buffer memory for operations such as get_input(). Add test_aot_executor() test function to test_crt.py to exercise this new functionality.
This PR addresses #10076
cc @gromero @mehrdadh @areusch @mkatanbaf