Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sample: Transfer a blob from a background context to a page #766

Open
dotproto opened this issue Oct 27, 2022 · 113 comments
Open

Sample: Transfer a blob from a background context to a page #766

dotproto opened this issue Oct 27, 2022 · 113 comments

Comments

@dotproto
Copy link
Contributor

dotproto commented Oct 27, 2022

Goal of the demo

Demonstrate how an extension can create a blob in the extension's service worker and transfer it to a page's main world.

Suggested implementation

Sketch of my initial implementation plan

  1. Have a content script inject an iframe into a page
  2. Run a script in the iframe to post a message back to the service worker using navigator.serviceWorker.controller.postMessage()
  3. In the SW's message handler, generate a blob OR use structuredClone() to create a copy of a blob.
  4. Use event.source.postMessage(msg, [transferable]) to reply to the client and transfer the blob
  5. In the iframe's message handler, use window.parent.postMessage(msg, "<origin>", [transferable]) to transfer the data to the page

Related links

Notes

Initial findings suggest that it's not possible to transfer variables across origins. I'm tentatively thinking that to work around this, we can use URL.createObjectURL() in the iframe and directly reference the object URL somewhere that's easily visible to the end user (e.g. canvas, Audio/Video element, etc.).

@guest271314

This comment was marked as off-topic.

@guest271314

This comment was marked as off-topic.

@guest271314

This comment was marked as off-topic.

@tophf
Copy link

tophf commented Oct 29, 2022

Goal of the demo
transfer it to a page's main world

It's the wrong goal in the context of the linked issue. I guess its title was initially confusing as it mentioned "web page" instead of "web page's content script", so I've changed it to reflect the contents of the issue, which is about the content script. It's a more difficult task to accomplish securely, but still possible via an iframe inside a closed ShadowDOM with a random token + MessagePort as described in the comment linked in the issue. One change to the scheme would be to send the MessageChannel port from SW to the content script via the iframe, then use this port to transfer blobs transferable binary data directly and instantly by specifying the second parameter of postMessage.


The following has been proved wrong, see the clarifying comment.

Note that to transfer a huge blob instantly, which is what the related issue is all about, we need the second parameter:~

port1.postMessage(blob, [await blob.arrayBuffer()]);

~It should be mentioned explicitly that it destroys the blob in the source (in SW).

@guest271314

This comment was marked as off-topic.

@tophf
Copy link

tophf commented Oct 29, 2022

Blob is just a wrapper on the same data exposed via arrayBuffer accessor or other getters on the blob, so this code does transfer the actual data instantly.

@guest271314

This comment was marked as off-topic.

@tophf
Copy link

tophf commented Oct 29, 2022

Not the Blob, the arrayBuffer view of the same data contained in the Blob.

@guest271314

This comment was marked as off-topic.

@tophf
Copy link

tophf commented Oct 29, 2022

A Blob will be received by the listener, so it can be said that the Blob is effectively transferred.

@guest271314

This comment was marked as off-topic.

@tophf
Copy link

tophf commented Oct 29, 2022

Blob is not a Transferable but it's effectively transferable. Anyway, it's semantics.

@guest271314

This comment was marked as off-topic.

@tophf
Copy link

tophf commented Oct 29, 2022

I know.

@guest271314

This comment was marked as off-topic.

@guest271314

This comment was marked as off-topic.

@tophf
Copy link

tophf commented Oct 29, 2022

Using the code I posted (with the second parameter using blob.arrayBuffer) will transfer a Blob instantly, it's trivial to verify.

@tophf
Copy link

tophf commented Oct 29, 2022

FWIW, if you're interested what happens under the hood, here's my guess: the blob's actual data being a Transferable (blob.arrayBuffer) is sent instantly, while the Blob itself is just a wrapper over that data, which is structuredClone'able, so it's reconstructed in the receiver using the instantly transferred data.

@guest271314

This comment was marked as off-topic.

@tophf
Copy link

tophf commented Oct 29, 2022

  1. It's how Blob and other binary related wrappers (DataView, typed arrays, buffer) work. You can investigate it yourself.
  2. The blob will be copied via internal structuredClone.

@guest271314

This comment was marked as off-topic.

@tophf
Copy link

tophf commented Oct 29, 2022

It's in the specifications somewhere, which I investigated when I looked for a way to transfer blobs instantly. I found it, confirmed it worked. I don't have the links in browser history anymore, so if you're interested you can investigate and verify it yourself.

@guest271314

This comment was marked as off-topic.

@guest271314

This comment was marked as off-topic.

@tophf
Copy link

tophf commented Oct 29, 2022

Your demo prints the blob in console, just as it should.

@guest271314

This comment was marked as off-topic.

@tophf
Copy link

tophf commented Oct 29, 2022

Maybe there's no need for arrayBuffer, but try making a 1GB Blob and you will see it's transferred instantly, which is the only thing I care about in this context.

@guest271314

This comment was marked as off-topic.

@tophf
Copy link

tophf commented Oct 29, 2022

Try sending ['a'.repeat(100e6)] and new Blob(['a'.repeat(500e6)]). The string will take 1sec, the blob will be instantly sent.

@tophf
Copy link

tophf commented Nov 6, 2022

No, you don't get any data at all. The data is not a part of the function's code string.

@tophf
Copy link

tophf commented Nov 6, 2022

The data is any binary generated or downloaded stuff. It is not a part of any function.

@guest271314

This comment was marked as off-topic.

@tophf
Copy link

tophf commented Nov 6, 2022

It won't contain data from SW, it's a useless exercise in semantics.

@guest271314

This comment was marked as off-topic.

@guest271314

This comment was marked as off-topic.

@guest271314

This comment was marked as off-topic.

@guest271314

This comment was marked as off-topic.

@tophf
Copy link

tophf commented Nov 6, 2022

We're going in circles for like 100th time, but it's fun so let's do it once again.

A web page can trivially delete any element immediately once it's added as I've shown using DOMNodeInserted event. It can be done using MutationObserver too, as well as using a periodic check, it doesn't matter which one as they all are trivial to implement. This code can't be removed automatically from an unknown web page not because it specifically resists removal but because such code will be an integral part of a large minified bundle, possibly with obfuscation - you'd have to analyze it manually. Such code can be added by another extension as well, not that it's likely, but still possible and trivial to implement. It is trivial, there's no challenge in writing it. Of course there's a way for an extension to protect the iframe: it can override all DOM methods that can be used to remove an element or to rewrite/reload the page or to extract such methods from a new src-less iframe.

executeScript
fetch
import

All of these are for static data included with the extension, it's not what the linked issue is about. The issue is about data in SW context (and nowhere else within the extension). It can be generated by JS or downloaded from an external URL or captured from desktop. It's not a part of the func's code or of the extension package.

@guest271314

This comment was marked as off-topic.

@guest271314

This comment was marked as off-topic.

@tophf
Copy link

tophf commented Nov 6, 2022

Although Local Modifications is not a part of the extensions API/platform, but indeed its functionality can be replicated via chrome.debugger API, and indeed it can be used to remove the code once we identified it manually. I didn't contest this part. My point is that the analysis for such code can't be reliably automated. Also, chrome.debugger is not an API a typical user would want to allow for an extension that's not related to debugging.

Writing to a local file inside the installed extension directory is indeed a possibility but in addition to being slower than sending a Blob directly between contexts it would require either installing the extension inside the default downloads directory or to use a nativeMessaging app to write the file into the directory of an installed extension, finding which reliably and automatically is not always trivial since the browser may be launched with --user-data-dir parameter.

@guest271314

This comment was marked as off-topic.

@tophf
Copy link

tophf commented Nov 6, 2022

Ideally chrome messaging should use the standard web platform messaging mechanism under the hood. The linked issue also mentions adding a transfer parameter to enable what the web platform's "transferrable" parameter does - at least in the non-broadcast methods i.e. ports.

@guest271314

This comment was marked as off-topic.

@guest271314

This comment was marked as off-topic.

@guest271314

This comment was marked as off-topic.

@guest271314

This comment was marked as off-topic.

@guest271314

This comment was marked as off-topic.

@guest271314

This comment was marked as off-topic.

@guest271314

This comment was marked as off-topic.

@guest271314

This comment was marked as off-topic.

@guest271314

This comment was marked as off-topic.

@guest271314

This comment was marked as off-topic.

guest271314 added a commit to guest271314/chrome-extensions-samples that referenced this issue Nov 28, 2022
Fixes GoogleChrome#766

Create offscreen document, use WebRTC RTCDataChannel to send ArrayBuffer(s) from ServiceWorker to Web page.

Ideally this can be solved by programmatically setting WindowClient to arbitrary Web page, getting a reference to offscreen window, or setting opener on offscreen window for ability to use onmessage and postMessage(data, target, [data]) directly. 

RTCDataChannel send() does not transfer data. It does send ArrayBuffer(s) transferred from ServiceWorker to offscreen window to Web page window.
guest271314 added a commit to guest271314/chrome-extensions-samples that referenced this issue Nov 28, 2022
guest271314 added a commit to guest271314/chrome-extensions-samples that referenced this issue Nov 28, 2022
@guest271314

This comment was marked as off-topic.

@guest271314

This comment was marked as off-topic.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants