Investigate and document the fingerprintability of user media rendering

[tomrittervg]

From what I can see, this allows a website to view how a UA renders a or element. Canvas in particular is used quite pervasively throughout the web to fingerprint and track users by exploiting subtle rendering differences (very different from its intended purpose.)

I've never seen a study done on whether or not is rendered differently on different browsers, but as this is the first time (I think) users' rendering of media will be exposed to the web, it seems the responsible thing to do for the draft is to investigate this and document it negatively or positively - and if users rendering is in fact different, what if anything a UA can do about it. (I believe this is behind a permission prompt, so that's fortunately a large barrier against passive tracking, but the prompt might incorporate a warning, since I don't think a user would ever understand "Take a video of your gameplay" to also imply "Allow this website to uniquely identify you".)

[alvestrand]

Forgive my ignorance....
How is capturing the canvas to a MediaStream different from capturing the canvas by dumping its raw content (canvas.getImageData(), triggered by appropriate event handlers)?

[tomrittervg]

Nothing to forgive, I'm the ignorant one here, thanks for engaging with me. I'm trying to get up to speed on this draft and some web features I haven't used much myself.

It seems like you can render a video onto a canvas, and then use getImageData to expose rendered media to the web - so I was wrong when I said this was the first time that's exposed. (I'm not sure if you could do frame-by-frame playback and getImageData to accurately capture in-motion video rendering but the safe thing would be to assume yes.)

Anyway, my understanding of things is that and , with respect to allowing the server to learn how data is rendered, are overlapping but not inclusive circles. can render data not related to (like text with system fonts) and expose the result - that's a unique technique to canvas using canvas APIs. can also render video and expose the result using this new media API. And then canvas can also render video and expose the result - that's exposing the same data through canvas APIs and media APIs.

This is a new API. We learned of the risks exposed by canvas fingerprinting after canvas was exposed. We should learn from that experience and not make the same mistakes, nor should we codify even more inflexible vectors into the web.

The canvas APIs are synchronous. UAs who want to resist canvas fingerprinting have no good options to do so. Turning a synchronous API asynchronous blocks expected script execution and breaks sites. Lying about the result kind of works but breaks website functionality in unexpected ways.

Assuming that video rendering is fingerprintable, and that it can't be made normalized across systems - it would be ideal to expose these APIs as Promises. That way a UA can decide whether they want to resolve the promise immediately (and not prompt the user) or prompt the user and wait for a response before resolving the promise. That flexibility will allow UAs who want to resist fingerprinting the option of doing so.

[martinthomson]

So there are no permissions prompts here. That suggests that yes, this is a new way to fingerprint implementations based on how they render video... maybe (more below). To the extent that it allows capture from canvas, it also allows access to all the existing canvas fingerprinting capabilities.

The product of this API is a video stream (which might also include audio). To the extent that the incoming video is subject to platform-specific modification as a result of being rendered, this does expose information about the platform. Where possible, the output video will be a direct clone of the input and so won't expose platform-specific properties. That's can probably be prevented by an application though with canvas by applying a small tweak (such as a transform) that forces rendering and re-capture.

FWIW, MediaRecorder has the same problem.

Both this and MediaRecorder also expose other rendering-related things like platform-specific loss masking techniques.

[alvestrand]

Is there a writeup of the issues that were found with canvas fingerprinting somewhere easily accessible?

Is https://en.wikipedia.org/wiki/Canvas_fingerprinting an adequate writeup?
That article cites "5.7 bits of fingerprinting entropy" - I don't know how we relate fingerprints to each other, but is that a large or a small number?

[martinthomson]

The wikipedia page is reasonable. Read the paper if you want more details.

I wouldn't credit any number, especially one from a small trial like that. 5.7 out of a possible 8.2 suggests that there might be a lot more available in a larger sample set.