feat: add tvm wasm export #582

WenheLI · 2020-09-09T09:58:46Z

This PR adds a wasm export solution for tensorflow.
More models' support will come in the future.

To export the model into wasm, you need to export WASM=1 in the command line to set the env variable.

The above env variable is just a temporary solution. In the final release, we should be able to config the output format per job. Therefore, this support will be added after #565 is landed.

yorkie · 2020-09-10T04:52:33Z

To export the model into wasm, you need to export WASM=1 in the command line to set the env variable.

I think adding an option from CLI is better than the environment:

$ pipcook run foobar.json --type wasm|node

yorkie · 2020-09-10T04:56:29Z

And I found some .bc files in our source, it might be unstable because they expect to run on specific v8.

yorkie · 2020-09-10T04:57:41Z

docs/INSTALL.md

@@ -10,6 +10,8 @@ Before starting the installation, please make sure the following environments ar
 - macOS, Linux
 - Node.js 12

+**Note:** To use `wasm` output format, you need to manually install [`emsdk`](https://emscripten.org/docs/introducing_emscripten/index.html) and export the `emcc` and `emsdk` to the environmental variable.


Shall we include the emcc/emsdk inside Pipcook?

yorkie · 2020-09-10T04:59:38Z

packages/daemon/templates/wasm/predict.js

+  return output.toArray();
+}
+
+module.exports = predict;


yorkie · 2020-09-10T05:02:37Z

packages/daemon/package.json

@@ -61,7 +62,8 @@
    "test": "midway-bin test --ts --full-trace",
    "migration": "sequelize-cli db:migrate",
    "migration:undo": "sequelize-cli db:migrate:undo",
-    "benchmark": "node benchmark/bootstrap.js"
+    "benchmark": "node benchmark/bootstrap.js",
+    "postinstall": "python3 -m pip install https://files.pythonhosted.org/packages/67/ff/011f588d54153c5d8ee3841d137acf752933f78eb1e3d1c5ffc6c1cb5a32/pipcook_tvm-0.7.dev1-cp37-cp37m-macosx_10_15_x86_64.whl"


Shall we have a PyPi package for this? Installing from this whl causes the following 2 issues:

Python upgrade/downgrade not working.

mirror index not working.

yorkie · 2020-09-10T05:03:56Z

packages/daemon/binary/preload.js

@@ -0,0 +1,41 @@
+/*


This is created by us? We don't specify the license in this way :(

yorkie · 2020-09-10T05:05:17Z

packages/daemon/src/service/pipeline.ts

+    const {dict, open} = boa.builtins();
+
+    // download tvm runtime from oss
+    const tvmjsPromise = execAsync(`wget http://ai-sample.oss-cn-hangzhou.aliyuncs.com/tvmjs/dist/tvmjs.bundle.js`, {cwd: dist});


No need to use string template.

We need a cache for this bundle.

yorkie · 2020-09-10T05:09:09Z

packages/daemon/src/service/pipeline.ts

+    const tvmjsPromise = execAsync(`wget http://ai-sample.oss-cn-hangzhou.aliyuncs.com/tvmjs/dist/tvmjs.bundle.js`, {cwd: dist});
+    fileQueue.push(tvmjsPromise);
+
+    const model = keras.models.load_model(path.join(opts.modelPath, "model.h5"));


How about splitting the framework-specific generator to another file?

yorkie

And the CI should be fixed :)

WenheLI · 2020-09-22T07:48:16Z

Await TVM releasement.

yorkie · 2020-09-22T07:50:34Z

Await TVM releasement.

@WenheLI May I ask you to share which release is to be waited, and what problem(s) to be resolved by this release?

WenheLI · 2020-09-22T08:07:16Z

We are awaiting apache/tvm#6486 (v0.7).
They will provide binary distribution after this release, which will help us reduce the redundant work during installation.

…into feat/tvm-support

yorkie · 2020-10-09T15:53:37Z

See the following notes from CPython Documentation:

if the Python executable is found in /usr/local/bin/python, it will assume that the libraries are in /usr/local/lib/pythonX.Y. (In fact, this particular path is also the “fallback” location, used when no executable file named python is found along PATH.) The user can override this behavior by setting the environment variable PYTHONHOME, or insert additional directories in front of the standard path by setting PYTHONPATH.

Thus the PYTHONHOME is an unnecessary set, because if the python executable is /path/to/boa/.miniconda/bin/python, the libraries are at /path/to/boa/.miniconda/lib/pythonX.Y.

/cc @FeelyChau @WenheLI @rickycao-qy

yorkie · 2020-10-09T15:57:29Z

I will remove it after #602 gets merged because this PR might require it as well :)

yorkie · 2020-10-10T15:14:47Z

packages/daemon/src/service/pipeline.ts

@@ -317,6 +317,66 @@ export class PipelineService {
    return path.join(CoreConstants.PIPCOOK_RUN, id, 'output.tar.gz');
  }

+  // private _generateWASMOutput(dist: string, opts: GenerateOptions, fileQueue: Array<Promise<void | string>>): void {


@WenheLI @FeelyChau Boa is working with worker_threads at #602, this function could be reimplemented with worker_threads to avoid blocking the server.

WenheLI · 2020-12-11T03:02:19Z

Too many conflicts, open another PR for this feature

WenheLI added 6 commits September 8, 2020 19:06

add boa

a32c280

add codebase

cda9a77

doc: add docs

aa6eb93

add ci

f9a2b71

add pre

824ae18

add url

eace176

yorkie reviewed Sep 10, 2020

View reviewed changes

packages/daemon/templates/wasm/predict.js

return output.toArray();

}

module.exports = predict;

Copy link

Member

yorkie Sep 10, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

EOF

yorkie reviewed Sep 10, 2020

View reviewed changes

packages/daemon/binary/preload.js Outdated

@@ -0,0 +1,41 @@

/*

Copy link

Member

yorkie Sep 10, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is created by us? We don't specify the license in this way :(

yorkie reviewed Sep 10, 2020

View reviewed changes

yorkie suggested changes Sep 10, 2020

View reviewed changes

rm

ff80ea1

yorkie mentioned this pull request Sep 26, 2020

plugins for structured data and relative models #566

Open

WenheLI and others added 10 commits October 9, 2020 11:48

add tlcpack distributation

7627d10

rm

36cb1ae

Merge branch 'master' into feat/tvm-support

1cd0241

change

0c968f6

Merge branch 'feat/tvm-support' of https://github.com/alibaba/pipcook …

b07993c

…into feat/tvm-support

add setuptools

1265a49

add

9696f42

add key

f11ae7a

fix conflict

b79d1cd

fix lint

14ed9ee

WenheLI and others added 11 commits October 9, 2020 16:34

change template str

2d12f51

rm

9330636

add test script

28a0142

add test

6d54394

add

ea686c7

test

cb81a8e

Update runtime.ts

7139b23

debug

cc004ea

add

def08c4

Merge branch 'feat/tvm-support' of https://github.com/alibaba/pipcook …

187d110

…into feat/tvm-support

add

1edf04e

yorkie mentioned this pull request Oct 10, 2020

boa: don't set PYTHONHOME #610

Closed

yorkie reviewed Oct 10, 2020

View reviewed changes

WenheLI closed this Dec 11, 2020

WenheLI deleted the feat/tvm-support branch December 11, 2020 03:14

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add tvm wasm export #582

feat: add tvm wasm export #582

WenheLI commented Sep 9, 2020

yorkie commented Sep 10, 2020

yorkie commented Sep 10, 2020

yorkie Sep 10, 2020

yorkie Sep 10, 2020

yorkie Sep 10, 2020

yorkie Sep 10, 2020

yorkie Sep 10, 2020

yorkie Sep 10, 2020

yorkie Sep 10, 2020

yorkie left a comment

WenheLI commented Sep 22, 2020

yorkie commented Sep 22, 2020

WenheLI commented Sep 22, 2020

yorkie commented Oct 9, 2020

yorkie commented Oct 9, 2020

yorkie Oct 10, 2020

WenheLI commented Dec 11, 2020

feat: add tvm wasm export #582

feat: add tvm wasm export #582

Conversation

WenheLI commented Sep 9, 2020

yorkie commented Sep 10, 2020

yorkie commented Sep 10, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yorkie left a comment

Choose a reason for hiding this comment

WenheLI commented Sep 22, 2020

yorkie commented Sep 22, 2020

WenheLI commented Sep 22, 2020

yorkie commented Oct 9, 2020

yorkie commented Oct 9, 2020

Choose a reason for hiding this comment

WenheLI commented Dec 11, 2020