[Web] chrome V3 extension TypeError: URL.createObjectURL is not a function

[jonathanpeppers]

Describe the issue

We are using onnxruntime-web in a Chrome extension.

We are in the process of migrating to Chrome V3 format (worked great in V2) and hit the error:

TypeError: URL.createObjectURL is not a function
    at Object.locateFile (packed.js:formatted:18165:126)
    at O (packed.js:formatted:345:61)
    at Object.ib (packed.js:formatted:718:49)
    at Object.mb (packed.js:formatted:722:73)
    at fe (packed.js:formatted:594:48)
    at Te (packed.js:formatted:807:42)
    at ort-wasm-simd-threaded.wasm:0x772d56
    at ort-wasm-simd-threaded.wasm:0xeb64e
    at ort-wasm-simd-threaded.wasm:0x1c4048
    at ort-wasm-simd-threaded.wasm:0xe5575

URL.createObjectURL does not appear to be available to Chrome V3 extensions.

Is there a different way for us to use onnxruntime and avoid this API? Thanks!

To reproduce

You could install our extension: release.zip

Editing a multi-line text box (like GitHub) hits the error, and you can view from the extensions page:

Source code here: jonathanpeppers/inclusive-code-reviews-browser#90

Urgency

Medium. Chrome V2 extensions are being phased out this year, and this seems to be a blocker.

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.13.1

Execution Provider

WASM

[fs-eire]

thank you for the feedback. it looks like URL.createObjectURL() cannot be used in a service worker. we are trying to find a way to fix this.

[jonathanpeppers]

Thanks! It appears I can workaround with ort.env.wasm.numThreads = 1;, but I'm not sure exactly what the setting does:

jonathanpeppers/inclusive-code-reviews-browser#195

Is there a doc/readme that explains what the 4 different *.wasm files do?

[guschmue]

One thing is that you might want to use chrome.runtime.getURL("asserts/model.onnx") to fetch the model.
That has been working well for me.
I can't explain why ort.env.wasm.numThreads=1 helps, need to debug it a little.

[fs-eire]

The error does not happen in loading the model, it happens when initializing the web assembly.

The reason why ort.env.wasm.numThreads=1 works is because the function URL.createObjectURL() will only be called when initializing multi-threaded version web assembly. The function is used to create a blob for 'ort-wasm-threaded.worker.js'. So setting to load the single-threaded version is a workaround.

https://github.com/microsoft/onnxruntime/blob/main/js/web/lib/wasm/wasm-factory.ts#L124-L128

[jonathanpeppers]

@fs-eire do you know any drawbacks to the single-threaded version?

We're not trying to call our model multiple times in parallel, really just sending 1 request at a time.

[guschmue]

@fs-eire - got it.
It is this code here: https://github.com/microsoft/onnxruntime/blob/main/js/web/lib/wasm/wasm-factory.ts#L124.
Aand it sounds you cannot use URL.createObjectURL with manifest v3 (ie
https://groups.google.com/a/chromium.org/g/chromium-extensions/c/u0NH7L3v9L4).
If I understand the code correctly, we do this to avoid distributing the js files for simd.
We might want to find a different way of doing this.

@jonathanpeppers, the disadvantage for single threaded is only performance. Your model looked not very compute intensive so you might not notice.

[lizozom]

I got this error, specifically working with @xenova/transformers in a chrome extension.
I had this code throwing the same error:

import { env, pipeline } from '@xenova/transformers';
env.localModelPath = 'models/';
env.allowRemoteModels = false;
this.embedder = await pipeline("embeddings", EMBEDDING_MODEL_NAME);

I added env.backends.onnx.wasm.numThreads = 1; and it resolved the error.

[lizozom]

However, running single threaded slows everything down. I'd love to work on this if anyone points me in the right direction :-)

[xenova]

@fs-eire @guschmue any updates on this? 😇 I've been working on improving my chrome extension demo for transformers.js, and being able to get this working would be great!

[guschmue]

I can look at it - might be a little complicated to fix.

[MentalGear]

An update to allow multi-threading would be really beneficial !

[ianmarmour]

Would love to get an update on this. At the moment the the node library is pretty much unusable for me due to lacking support for onnxruntime-extensions (probably should be a separate bug report). But with this fixed we can just use rely on the web library for both web and node usage (we can now already but it would be great if threading worked properly)

[MentalGear]

Indeed this is blocking the efficient usage of the very popular transformers.js in the browser for local and private ML. @fs-eire @guschmue

[Dolidodzik]

I would love to see the update as well. I wanted to work with transformers.js to run LLMs in the browser but this issue makes it very inefficient due to lack of multithreading support.

[BrennanBarker]

@fs-eire @guschmue is there anything wrong in principle with converting the URL.createObjectURL calls to a simple Data URL?

[sebastianbenz]

Have you tried running onnxruntime-web inside an offscreen document? These are meant to supplement extension service workers in case additional APIs are required that are not available in a service worker.

[guschmue]

this PR should fix this:
#20165