From ef7a760c87aabf83e6e246738e0d58badffed341 Mon Sep 17 00:00:00 2001 From: Mehrdad Hessar Date: Fri, 27 May 2022 15:44:32 -0700 Subject: [PATCH] MLF with Multiple Module RFC --- rfcs/0075_mlf_with_multiple_modules.md | 162 +++++++++++++++++++++++++ 1 file changed, 162 insertions(+) create mode 100644 rfcs/0075_mlf_with_multiple_modules.md diff --git a/rfcs/0075_mlf_with_multiple_modules.md b/rfcs/0075_mlf_with_multiple_modules.md new file mode 100644 index 00000000..7b763b9d --- /dev/null +++ b/rfcs/0075_mlf_with_multiple_modules.md @@ -0,0 +1,162 @@ +- Feature Name: mlf_with_multiple_module_support +- Start Date: 2022-05-26 +- RFC PR: [apache/tvm-rfcs#0075](https://github.com/apache/tvm-rfcs/pull/0075) +- GitHub Issue: [apache/tvm#0000](https://github.com/apache/tvm/issues/0000) + +# Summary +[summary]: #summary + +This RFC describes a new version of model library format where we can support adding multiple Relay module builds into a single MLF file. + +# Motivation +[motivation]: #motivation + +MLF artifact is an important piece of building a microTVM project. In the first version of the MLF artifact, +MLF only supports a single "Relay module"/"PackedFunc" build. However, there are cases where adding +multiple Relay modules is required. For example, corstone300 tests using multiple Relay modules was introduced by +contributors from ARM. The goal of this RFC is to create a new version of MLF which has standard support for these cases. + +# Guide-level explanation +[guide-level-explanation]: #guide-level-explanation + +Users call `export_model_library_format` function with multiple Relay builds (output from `tvm.relay.build`) or +single a Relay build module. We change the `export_model_library_format` API to support multiple Relay Build, however +it is backward compatible and, we handle required changes internally. + +Since we are including multiple modules in a single MLF file, we need to make some changes. Let's assume we +are want to export MLF file for two Relay build modules called `mod1` and `mod2`. + +1. We change format of `metadata.json` file. We will add `modules` as a key to JSON file which represents + multiple Relay build module. `modules` is a dictionary and each Relay build module is differentiated by + its module name. For example: + ```javascript + { + 'modules': { + 'mod1': { + + }, + 'mod2': { + + }, + ... + } + } + ``` + 2. There are multiple Relay text file generated per each Relay build. We propose to structure these files by using + module name and `.relay` extension to differentiate its format(E.g. {`mod1.relay`, `mod2.relay`, ...}). Similarly, for + `graph.json` file we propose to restructure to graph JSON file to {`mod1.graph`, `mod2.graph`, ...}. This is + compatible with existing structure for parameters where we use {`mod1.params`, `mod2.params`, ...}. + 3. Finally, we keep higher level information which are not specific to a module at higher level in the metadata file. + Currently, we only have `version` which shows the MLF version, but this could grow in the future. + +# Reference-level explanation +[reference-level-explanation]: #reference-level-explanation + +Building on the same example in previous section, here we explain what are the API calls and expected output after this change. + +First, we build two modules: +```python +mod1 = ... +mod2 = ... + +executor = Executor("graph") +runtime = Runtime("crt") +target = tvm.target.target.micro("host") + +with tvm.transform.PassContext(opt_level=3): + factory1 = tvm.relay.build(mod1, target, runtime=runtime, executor=executor, mod_name="mod1") + factory2 = tvm.relay.build(mod2, target, runtime=runtime, executor=executor, mod_name="mod2") +``` + +Then, we pass both results of `tvm.relay.build` to `export_model_library_format` function and path to generated model +library format: +```python +micro.export_model_library_format([factory1, factory2], mlf_tar_path) +``` + +Now, if we extract MLF file here is the file structure: +```bash +# codegen source files +codegen/ + host/ + src/ + mod1_lib0.c + mod1_lib1.c + mod1_lib2.c + mod2_lib0.c + mod2_lib1.c + mod2_lib2.c + +# graph +executor-config/ + graph/ + mod1.graph + mod2.graph + +# metadata file +metadata.json + +# parameters +parameters/ + mod1.params + mod2.params + +# relay text output +./src/mod1.relay +./src/mod2.relay +``` + +# Drawbacks +[drawbacks]: #drawbacks + +The drawback here is that we are changing generated metadata and file structure in the MLF file which means external +tools which are dependent to this need to update their tool. Hopefully this RFC makes it clear on what steps they need +to take. + +# **Rationale and alternatives** +An alternative way to implement this feature is to break down each Relay build module to a subdirectory and keep the +previous format inside each sub Relay build module. Using the example of `mod1` and `mod2`, in this approach we have +an MLF file format with structure bellow if we extract: +```bash +# mod1 +mod1/ + codegen/ + host/ + src/ + mod_lib0.c + mod_lib1.c + mod_lib2.c + +#mod2 +mod1/ + codegen/ + host/ + src/ + mod2_lib0.c + mod2_lib1.c + mod2_lib2.c + +# graph +executor-config/ + graph/ + mod1.graph + mod2.graph + +# metadata file +metadata.json + +# parameters +parameters/ + mod1.params + mod2.params + +# relay text output +./src/mod1.relay +./src/mod2.relay +``` + +# Prior art +[prior-art]: #prior-art + +Prior art is the [RFC](https://discuss.tvm.apache.org/t/rfc-tvm-model-library-format/9121) for the first version of the MLF design. +