Add TVM application extension with WASM runtime

[leonwanghui]

Background

As demonstrated in TVM runtime tutorials, TVM already supports WASM as the optional hardware backend, so we can leverage the features of WebAssembly (portability, security) and TVM runtime (domain-specific, optimization) to build a flexible and auto-optimized operator backend for all deep learning frameworks.

Proposal Summary

This PR is mainly proposed to add new TVM application extension that provides WASM operator backend for deep learning frameworks with TVM runtime. We believe it would bring some new ideas by combining WASM runtime with TVM compiler stack.

Thanks for reviewing this proposal, considering it's at the experimental stage to incubation, so any suggestions or questions are welcomed. And we wish more contributors can join in to make it grow up.

@tqchen @jroesch @kazum PTAL, thanks!

[tqchen]

Thanks @leonwanghui Given that TVM is a compilation stack, directly generating generic ops may not give the best performance for deployment, as the compilation approach takes a lot of shape specialization and fusions.

A better approach for integration might be creating a relay or TIR-level integration from the framework to the TVM's unified IR, then let the compilation produces a standalone module.

[leonwanghui]

@tqchen Thanks for your suggestion, if we target relay or TIR-level integration, does that mean we need to create Rust FFI for them at first?

[tqchen]

@jroesch is already working on some of the RUST FFI access. We can also do that through python or other language binding and only use the rust runtime to execute in wasm.

[leonwanghui]

Thanks @tqchen, it seems promising to me. I will finish the PoC and update this PR later.

[leonwanghui]

@tqchen @jroesch @kazum Now this PR is updated, please take a review, thanks!

[leonwanghui]

cc @tqchen @jroesch

[tqchen]

please remove the csv file as we cannot checkin binary to the codebase

[leonwanghui]

please remove the csv file as we cannot checkin binary to the codebase

@tqchen Done

[tqchen]

Thanks @leonwanghui some high level comments, please also check the CI error and add the license header to the related files

[leonwanghui]

cc @tqchen Please review it again, thanks!

[leonwanghui]

@tqchen It seems that the ./cargo/config file is not allowed to be included into the repo, the ci error is as follows:

+ docker/bash.sh tvmai/ci-lint:v0.61 ./tests/scripts/task_lint.sh

WORKSPACE: /scratch/jenkins-tvm/cudabuild/workspace/exec_3/tvm/sanity

DOCKER CONTAINER NAME: tvmai/ci-lint:v0.61



Running './tests/scripts/task_lint.sh' inside tvmai/ci-lint:v0.61...

mesg: ttyname failed: Inappropriate ioctl for device

Adding group `tvm' (GID 1000) ...

Done.

Check file types...

------File type check report----

apps/wasm-standalone/wasm-graph/.cargo/config

Found 1 files that are now allowed

We do not check in binary files into the repo.

If necessary, please discuss with committers andmodify tests/lint/check_file_type.py to enable the file you need.

script returned exit code 255

[tqchen]

@leonwanghui it seems that we might want to include .cargo/config, can you modify https://github.com/apache/incubator-tvm/blob/master/tests/lint/check_file_type.py#L110 to enable that?

[tqchen]

cc @kazum

[leonwanghui]

cc @kazum I have resolved your comments, PTAL

[kazum]

Looks good to me.

[leonwanghui]

cc @tqchen

[nhynes]

Thanks for making the style fixes. Please remember to also run cargo clippy!

[leonwanghui]

cc @tqchen @nhynes @jroesch, I think this PR is ready to go, please notice that some of clippy warnings in tvm-graph-rt module would not be addressed in this PR.

[tqchen]

@leonwanghui please send another commit to trigger the CI, @jroesch @nhynes please followup and take another look

[nhynes]

@tqchen This PR looks fine. I have a few style nits, but I don't think they'd be well received.

Thanks @leonwanghui for the contribution!

[leonwanghui]

cc @tqchen Your comments have been addressed, PTAL

[tqchen]

Thanks @leonwanghui @jroesch @nhynes @kazum ! this PR is now merged