MLF with Multiple Module RFC

rfcs/0075_mlf_with_multiple_modules.md

@@ -0,0 +1,162 @@
+- Feature Name: mlf_with_multiple_module_support
+- Start Date: 2022-05-26
+- RFC PR: [apache/tvm-rfcs#0075](https://github.com/apache/tvm-rfcs/pull/0075)
+- GitHub Issue: [apache/tvm#0000](https://github.com/apache/tvm/issues/0000)
+
+# Summary
+[summary]: #summary
+
+This RFC describes a new version of model library format where we can support adding multiple Relay module builds into a single MLF file.
+
+# Motivation
+[motivation]: #motivation
+
+MLF artifact is an important piece of building a microTVM project. In the first version of the MLF artifact, 
+MLF only supports a single "Relay module"/"PackedFunc" build. However, there are cases where adding
+multiple Relay modules is required. For example, corstone300 tests using multiple Relay modules was introduced by
+contributors from ARM. The goal of this RFC is to create a new version of MLF which has standard support for these cases. 
+
+# Guide-level explanation
+[guide-level-explanation]: #guide-level-explanation
+
+Users call `export_model_library_format` function with multiple Relay builds (output from `tvm.relay.build`) or
+single a Relay build module. We change the `export_model_library_format` API to support multiple Relay Build, however 
+it is backward compatible and, we handle required changes internally.
+
+Since we are including multiple modules in a single MLF file, we need to make some changes. Let's assume we
+are want to export MLF file for two Relay build modules called `mod1` and `mod2`.
+
+1. We change format of `metadata.json` file. We will add `modules` as a key to JSON file which represents
+   multiple Relay build module. `modules` is a dictionary and each Relay build module is differentiated by 
+   its module name. For example:
+    ```javascript
+    {
+      'modules': {
+        'mod1': {
+
+        },
+        'mod2': {
+
+        },
+        ...
+      }
+    }
+    ```
+  2. There are multiple Relay text file generated per each Relay build. We propose to structure these files by using
+     module name and `.relay` extension to differentiate its format(E.g. {`mod1.relay`, `mod2.relay`, ...}). Similarly, for 
+     `graph.json` file we propose to restructure to graph JSON file to {`mod1.graph`, `mod2.graph`, ...}. This is 
+     compatible with existing structure for parameters where we use {`mod1.params`, `mod2.params`, ...}.
+  3. Finally, we keep higher level information which are not specific to a module at higher level in the metadata file. 
+     Currently, we only have `version` which shows the MLF version, but this could grow in the future.
+
+# Reference-level explanation
+[reference-level-explanation]: #reference-level-explanation
+
+Building on the same example in previous section, here we explain what are the API calls and expected output after this change.
+
+First, we build two modules:
+```python
+mod1 = ...
+mod2 = ...
+
+executor = Executor("graph")
+runtime = Runtime("crt")
+target = tvm.target.target.micro("host")
+
+with tvm.transform.PassContext(opt_level=3):
+  factory1 = tvm.relay.build(mod1, target, runtime=runtime, executor=executor, mod_name="mod1")
+  factory2 = tvm.relay.build(mod2, target, runtime=runtime, executor=executor, mod_name="mod2")
+```
+
+Then, we pass both results of `tvm.relay.build` to `export_model_library_format` function and path to generated model
+library format:
+```python
+micro.export_model_library_format([factory1, factory2], mlf_tar_path)
+```
+
+Now, if we extract MLF file here is the file structure:
+```bash
+# codegen source files
+codegen/
+  host/
+    src/
+      mod1_lib0.c
+      mod1_lib1.c
+      mod1_lib2.c
+      mod2_lib0.c
+      mod2_lib1.c
+      mod2_lib2.c
+
+# graph 
+executor-config/
+  graph/
+    mod1.graph
+    mod2.graph
+
+# metadata file
+metadata.json
+
+# parameters
+parameters/
+  mod1.params
+  mod2.params
+
+# relay text output
+./src/mod1.relay
+./src/mod2.relay
+```
+
+# Drawbacks
+[drawbacks]: #drawbacks
+
+The drawback here is that we are changing generated metadata and file structure in the MLF file which means external 
+tools which are dependent to this need to update their tool. Hopefully this RFC makes it clear on what steps they need
+to take.
+
+# **Rationale and alternatives**
+An alternative way to implement this feature is to break down each Relay build module to a subdirectory and keep the
+previous format inside each sub Relay build module. Using the example of `mod1` and `mod2`, in this approach we have
+an MLF file format with structure bellow if we extract:
+```bash
+# mod1
+mod1/
+  codegen/
+    host/
+      src/
+      mod_lib0.c
+      mod_lib1.c
+      mod_lib2.c
+
+#mod2
+mod1/
+  codegen/
+    host/
+      src/ 
+      mod2_lib0.c
+      mod2_lib1.c
+      mod2_lib2.c
+
+# graph 
+executor-config/
+  graph/
+    mod1.graph
+    mod2.graph
+
+# metadata file
+metadata.json
+
+# parameters
+parameters/
+  mod1.params
+  mod2.params
+
+# relay text output
+./src/mod1.relay
+./src/mod2.relay
+```
+
+# Prior art
+[prior-art]: #prior-art
+
+Prior art is the [RFC](https://discuss.tvm.apache.org/t/rfc-tvm-model-library-format/9121) for the first version of the MLF design.
+