[JS] Add GenAI Node.js bindings #1193

vishniakov-nikolai · 2024-11-11T16:58:40Z

Adding Node.js bindings for GenAI pipelines.

155187
158132

Limitations

Current version it's primary backbone of future development. Supports bindings of LLMPipeline only.

TODO

Test build configuration
Integrate unit tests run into GHA
Add script to download runtime binaries (after binaries publication)

Wovchena · 2024-12-12T15:08:56Z

src/js/README.md

+   ```
+1. Configure CMake with OpenVINO extra modules:
+   ```sh
+   cmake -DOPENVINO_EXTRA_MODULES=*relative (from openvino folder) path to genai repository* -DCPACK_ARCHIVE_COMPONENT_INSTALL=OFF \


relative (from openvino folder) path to genai repository*

Any path representation will do. Absolute path should also wrok

Changed to *absolute path*

The point is that you shouldn't care if it's absolute or relative. Just ask for a path

src/js/README.md

samples/js/chat_sample/chat_sample.js

src/js/CMakeLists.txt

Wovchena · 2024-12-20T09:12:21Z

src/js/README.md

+1. Configure CMake with OpenVINO extra modules:
+   ```sh
+   cmake -DOPENVINO_EXTRA_MODULES=*absolute path to genai repository directory* -DCPACK_ARCHIVE_COMPONENT_INSTALL=OFF \
+         -DCPACK_GENERATOR=NPM -DENABLE_JS=ON -UTBB* -DENABLE_SYSTEM_TBB=OFF \


Why do you care about tbb?

What does -UTBB* do?

samples/js/chat_sample/chat_sample.js

Wovchena · 2024-12-20T10:41:48Z

src/js/README.md

+1. In `build` folder you will find `genai_nodejs_bindings.tar.gz`.
+   Create `bin` directory by path `src/js/` and unpack archive content to it.
+   ```sh
+   tar -xzf genai_nodejs_bindings.tar.gz


Suggested change

1. In `build` folder you will find `genai_nodejs_bindings.tar.gz`.

Create `bin` directory by path `src/js/` and unpack archive content to it.

```sh

tar -xzf genai_nodejs_bindings.tar.gz

1. Install Node.js bindings

```sh

mkdir ./src/js/bin/

tar -xvf ./build/genai_nodejs_bindings.tar.gz --directory ./src/js/bin/

cd ./src/js/

npm install

Wovchena · 2024-12-20T10:43:48Z

src/js/README.md

+### Perform Test Run
+
+- To run sample you should have prepared model.
+  Use this instruction [to download model](https://github.com/openvinotoolkit/openvino.genai/blob/master/samples/js/chat_sample/README.md#download-and-convert-the-model-and-tokenizers)
+- Go to [samples/js/chat_sample/](../../samples/js/chat_sample/)
+- Read [README.md](../../samples/js/chat_sample/README.md) and follow steps there
+  to run **chat sample**.


I think you should drop this section because this is about the sample. You have a dedicated readme for that.

Wovchena · 2024-12-20T10:44:56Z

src/js/README.md

+
+### Using as npm Dependency
+
+To use this package locally use `npm link` in this directory


Suggested change

To use this package locally use `npm link` in this directory

To use this package locally use `npm link` in `src/js/` directory

Wovchena · 2024-12-20T10:45:27Z

src/js/README.md

+### Using as npm Dependency
+
+To use this package locally use `npm link` in this directory
+and `npm link genai-node` in the folder where you want add this package as dependency


Suggested change

and `npm link genai-node` in the folder where you want add this package as dependency

and `npm link genai-node` in the folder where you want to add this package as a dependency.

Wovchena · 2024-12-20T10:48:57Z

src/js/tests/bindings.test.js

+  before((_, done) => {
+    pipeline = new addon.LLMPipeline();
+
+    pipeline.init(MODEL_PATH, 'AUTO', (err) => {


Suggested change

pipeline.init(MODEL_PATH, 'AUTO', (err) => {

pipeline.init(MODEL_PATH, 'CPU', (err) => {

Setting that to CPU allows to avoid compiling all other plugins saving some CI time.

Wovchena · 2024-12-20T10:51:18Z

samples/js/chat_sample/README.md

+## Run:
+
+Create `bin` folder in [../../../src/js](../../../src/js).
+Compile GenAI JavaScript bindings archive first. Put its content into `bin` directory.


The sample should assume that bingings are already prepared. You should reference the corresponding readme instead

Wovchena · 2024-12-20T10:51:37Z

samples/js/chat_sample/README.md

+Create `bin` folder in [../../../src/js](../../../src/js).
+Compile GenAI JavaScript bindings archive first. Put its content into `bin` directory.
+
+Run `npm install` in current folder and then run a sample:


Suggested change

Run `npm install` in current folder and then run a sample:

Run `npm install` in current folder and then run the sample:

Wovchena · 2024-12-20T10:52:45Z

samples/js/chat_sample/README.md

+Create `bin` folder in [../../../src/js](../../../src/js).
+Compile GenAI JavaScript bindings archive first. Put its content into `bin` directory.
+
+Run `npm install` in current folder and then run a sample:


That's not what CI does. It runs npm link genai-node. Why are they different?

Aligned, good remark

Co-authored-by: Vladimir Zlobin <[email protected]>

Add genai nodejs bindings

0d73757

vishniakov-nikolai added the do_not_merge label Nov 11, 2024

github-actions bot added category: sampling Sampling / Decoding algorithms category: cmake / build Cmake scripts category: samples GenAI samples labels Nov 11, 2024

vishniakov-nikolai added 5 commits November 13, 2024 18:36

Include js bindings into package build

2d71b6b

Put binaries at the top level of NPM package

0c0b41c

Skip samples for NPM package

7e80306

Add model downloading into js unit tests

67d5a40

Merge branch 'master' into feature/nodejs-bindings

689e352

ilya-lavrenov removed the category: sampling Sampling / Decoding algorithms label Nov 20, 2024

Merge branch 'master' into feature/nodejs-bindings

4e3405a

github-actions bot added the no-match-files label Nov 20, 2024

Set openvino_tokenizers on latest master

6853446

github-actions bot added the category: tokenizers Tokenizer class or submodule update label Nov 20, 2024

Merge branch 'master' into feature/nodejs-bindings

e433a19

vishniakov-nikolai marked this pull request as ready for review November 25, 2024 10:58

vishniakov-nikolai added 2 commits November 26, 2024 15:58

Temporary disable most of gha wf to debug

ae806da

Minify linux wf

8632882

github-actions bot added category: llm_bench Label for tool/llm_bench folder category: GHA CI based on Github actions labels Nov 26, 2024

vishniakov-nikolai added 3 commits November 26, 2024 16:02

Fix wf file

9ceb3b5

Fix wf file

599713b

Set cpack generator as NPM

222940d

vishniakov-nikolai force-pushed the feature/nodejs-bindings branch from 7784bba to 222940d Compare November 26, 2024 16:21

vishniakov-nikolai added 5 commits November 26, 2024 17:31

Set ENABLE_JS on

901dd90

Add genai_node_addon component if ENABLE_JS is ON

09356ef

Add wf to check genai js bindings compilation

96d53b2

Remove setupvars.sh run

785bf8d

Fix ov + genai workflow

e8c53f2

vishniakov-nikolai removed the do_not_merge label Dec 12, 2024

Update build instructions

a365ef8

vishniakov-nikolai force-pushed the feature/nodejs-bindings branch from 153a7ea to a365ef8 Compare December 12, 2024 14:55

vishniakov-nikolai requested a review from as-suvorov December 13, 2024 10:40

Wovchena requested changes Dec 13, 2024

View reviewed changes

vishniakov-nikolai added 3 commits December 16, 2024 15:16

Fix review comments

6d0b501

Add run js api test in linux workflow

f241cda

Merge branch 'master' into feature/nodejs-bindings

1e97e70

vishniakov-nikolai requested a review from Wovchena December 16, 2024 15:31

vishniakov-nikolai added 10 commits December 16, 2024 16:52

Fix workflow

1f2324b

Add test for js sample

dda494d

Extend linux workflow to run js samples test

394d425

Change runner for js samples tests

3186142

Fix repo path

2fbd888

Merge branch 'master' into feature/nodejs-bindings

2daa0f3

Fix path

0a0c399

Add debug output

35c1c61

Fix js samples test

b2cb5dd

Merge branch 'master' into feature/nodejs-bindings

13c0558

Wovchena reviewed Dec 20, 2024

View reviewed changes

ilya-lavrenov added the do_not_merge label Dec 20, 2024

vishniakov-nikolai and others added 6 commits December 23, 2024 11:07

Merge branch 'master' into feature/nodejs-bindings

432e3de

Fix run sample cmd

30803b4

Co-authored-by: Vladimir Zlobin <[email protected]>

Fix js README.md

b04ee6c

Replace AUTO device to CPU

d7e0d2b

Fix js samples README

d1ce555

Align linux workflow with js samples README

ea0ff4a

vishniakov-nikolai requested a review from Wovchena December 23, 2024 11:14

vishniakov-nikolai added 2 commits December 23, 2024 16:42

Remove tbb specification from linux js workflow

7f8ee36

Merge branch 'master' into feature/nodejs-bindings

6434daf

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[JS] Add GenAI Node.js bindings #1193

[JS] Add GenAI Node.js bindings #1193

vishniakov-nikolai commented Nov 11, 2024 •

edited

Loading

Wovchena Dec 12, 2024

vishniakov-nikolai Dec 16, 2024

Wovchena Dec 20, 2024

Wovchena Dec 20, 2024

Wovchena Dec 20, 2024

Wovchena Dec 20, 2024

Wovchena Dec 20, 2024

Wovchena Dec 20, 2024

Wovchena Dec 20, 2024

Wovchena Dec 20, 2024

vishniakov-nikolai Dec 23, 2024

Wovchena Dec 20, 2024

Wovchena Dec 20, 2024

vishniakov-nikolai Dec 23, 2024


		### Using as npm Dependency

		To use this package locally use `npm link` in this directory

	and `npm link genai-node` in the folder where you want add this package as dependency
	and `npm link genai-node` in the folder where you want to add this package as a dependency.

	pipeline.init(MODEL_PATH, 'AUTO', (err) => {
	pipeline.init(MODEL_PATH, 'CPU', (err) => {

	Run `npm install` in current folder and then run a sample:
	Run `npm install` in current folder and then run the sample:

[JS] Add GenAI Node.js bindings #1193

Are you sure you want to change the base?

[JS] Add GenAI Node.js bindings #1193

Conversation

vishniakov-nikolai commented Nov 11, 2024 • edited Loading

Limitations

TODO

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vishniakov-nikolai commented Nov 11, 2024 •

edited

Loading