FF118+ audio tests #1701

Thorin-Oakenpants · 2023-08-12T11:23:50Z

test: https://arkenfox.github.io/TZP/tzp.html#audio
re: https://gitlab.torproject.org/tpo/applications/mullvad-browser/-/issues/224#note_2931592 - basically I want to confirm that all RFP users return the same results (per OS) for all audio tests on TZP
TIA .. have some 🍰 or 🥧

test 1 `offlineAudioContext` FF118+ RFP doesn't matter

was values are FF115-117

desktop/laptop
- `24fc63ce` was `a0bb5f0d`  win    thorin win11 desktop (AMD) + laptop (intel)
- `24fc63ce` was `a0bb5f0d`  win    MagicalDrizzle
- `24fc63ce` was `a0bb5f0d`  win    karlt win11 FF32-bit
- `24fc63ce` was `9b096036`  linux  mik0l 
- `24fc63ce`                 linux  pierov
- `24fc63ce` was `a0bb5f0d`  mac    fxbrit

ARM/android
- `a34c73cd` was `acbc4cf5`  thorin Galaxy A30, Android 11
- `a34c73cd` was `93e9b79c`  karlt  MacOS M2 Pro (ARM)

test 2 `CLICK` test FF118+, RFP = ON

was values are FF115-117 with RFP on

windows
- `8ce5922b` was `d7d1f70e` = thorin win11 desktop + laptop
   - ^ "audioContext_keys": "67a3eeee", "audio_hybrid": "bafe56d6", "audio_oscillator": "e9f98e24"
- `8ce5922b` was `d7d1f70e` = MagicalDrizzle
- `8ce5922b` was `912fcf6c` = karlt win11 FF32bit

linux
- `c95de417` was `9229c8e5` = mik0l
   - ^ "audioContext_keys": "9b69969b", "audio_hybrid": "bafe56d6", "audio_oscillator": "e9f98e24"
- `c95de417`                = pierov

mac (intel)
- `ccb610df` was `no change WTF?` = fxbrit
   - ^ "audioContext_keys": "debdefc0", "audio_hybrid": "bafe56d6", "audio_oscillator": "e9f98e24"

mac (ARM)
- `fe98e3e0` was `e8457c88` - karlt MacOS M2 Pro (ARM)
   - ^ "audioContext_keys": "debdefc0", "audio_hybrid": "1348e98d", "audio_oscillator": "c54b7aa9"

android
- `86efc898` = thorin Galaxy A30, Android 11
   - ^ "audioContext_keys": "2b9d44b0", "audio_hybrid": "1348e98d", "audio_oscillator": "c54b7aa9"

The text was updated successfully, but these errors were encountered:

mik0l · 2023-08-12T12:00:12Z

Release:

31258264 [1 metric]
[ i ] offlineAudioContext 1 9b096036 | 9b096036 | 35.73833402246237

Nightly:

765d70d3 [1 metric]
[ i ] offlineAudioContext 1 24fc63ce | 24fc63ce | 35.749968223273754

Release:

[ click ] hash	9229c8e5 [3 metrics]
audioContext 2	9b69969b [20 keys] [✓ RFP]
OscillatorNode 2	751b22a2
OscillatorNode/DynamicsCompressor 2	004fba37
{
  "audioContext_keys": {
    "hash": "9b69969b",
...
  "audio_hybrid": "004fba37",
  "audio_oscillator": "751b22a2"
}

Nightly:

[ click ] hash	c95de417 [3 metrics]
audioContext 2	9b69969b [20 keys] [✓ RFP]
OscillatorNode 2	e9f98e24
OscillatorNode/DynamicsCompressor 2	bafe56d6
{
  "audioContext_keys": {
    "hash": "9b69969b",
...
  "audio_hybrid": "bafe56d6",
  "audio_oscillator": "e9f98e24"
}

It's Linux.

MagicalDrizzle · 2023-08-13T03:12:32Z

Windows, RFP on.
offlineAudioContext:

116 Release:
2b5f2e29 [1 metric]
[ i ] offlineAudioContext a0bb5f0d | a0bb5f0d | 35.7383295930922

118 Nightly:
765d70d3 [1 metric]
[ i ] offlineAudioContext 24fc63ce | 24fc63ce | 35.749968223273754

Click:

116 Release: 
[ click ] hash	    d7d1f70e [3 metrics]
audioContext        67a3eeee [20 keys] [✓ RFP]
OscillatorNode	    b52d8cdb
OscillatorNode/DynamicsCompressor	c4ce2506
   - "audioContext_keys": "67a3eeee", "audio_hybrid": "c4ce2506", "audio_oscillator": "b52d8cdb"
   
118 Nightly:
[ click ] hash	     8ce5922b [3 metrics]
audioContext         67a3eeee [20 keys] [✓ RFP]
OscillatorNode       e9f98e24
OscillatorNode/DynamicsCompressor        bafe56d6
   - "audioContext_keys": "67a3eeee", "audio_hybrid": "bafe56d6", "audio_oscillator": "e9f98e24"

fxbrit · 2023-08-13T10:43:38Z

macOS Ventura 13.5

offlineAudioContext

Stable: a0bb5f0d | a0bb5f0d | 35.7383295930922
Nightly: 24fc63ce | 24fc63ce | 35.749968223273754

Stable:

hash | 388cd2a2 [3 metrics]

audioContext | debdefc0 [20 keys] [✓ RFP]
OscillatorNode | 4edcaa4d
OscillatorNode/DynamicsCompressor | e36af744

audioContext_keys: debdefc0
{
  "ac-baseLatency": 0,
  "ac-channelCount": 2,
  "ac-channelCountMode": "explicit",
  "ac-channelInterpretation": "speakers",
  "ac-maxChannelCount": 2,
  "ac-numberOfInputs": 1,
  "ac-numberOfOutputs": 0,
  "ac-outputLatency": 0.011609977324263039,
  "ac-sampleRate": 44100,
  "ac-state": "suspended",
  "an-channelCount": 2,
  "an-channelCountMode": "max",
  "an-channelInterpretation": "speakers",
  "an-fftSize": 2048,
  "an-frequencyBinCount": 1024,
  "an-maxDecibels": -30,
  "an-minDecibels": -100,
  "an-numberOfInputs": 1,
  "an-numberOfOutputs": 1,
  "an-smoothingTimeConstant": 0.8
}

Nightly:

hash | ccb610df [3 metrics]

audioContext | debdefc0 [20 keys] [✓ RFP]
OscillatorNode | e9f98e24
OscillatorNode/DynamicsCompressor | bafe56d6

keys are the same so I'm not re-posting them.

PieroV · 2023-08-16T10:33:42Z

PieroV, Linux, Firefox 118.0a1 nightly 2023-08-15:



[ re-run ] 			765d70d3 [1 metric]
--
[ i ] 			 			  offlineAudioContext 1 | 24fc63ce | 24fc63ce | 35.749968223273754
------
[ click ] 			hash | c95de417 [3 metrics]
audioContext 2 | 9b69969b [20 keys] [✓ RFP]
OscillatorNode 2 | e9f98e24
OscillatorNode/DynamicsCompressor 2 | bafe56d6

fxbrit · 2023-08-16T22:20:24Z

pants I saw you marked the comments as outdated, would you say the mac result is fine or do we need to dig deeper?

karlt · 2023-08-17T01:25:42Z

AIUI fxbrit meant audioContext_keys were the same on Nightly and Release.

ccb610df was no change WTF? = fxbrit

So this would be "ccb610df was 388cd2a2 = fxbrit".

Thorin-Oakenpants · 2023-08-17T06:52:50Z

yeah, the context keys should be identical. I wasn't super clear in my instructions, both are with RFP in the click test

fxbrit - marked it as noise because I added a WTF to OP. I was going to come back round to it (and test on my own mac)

why not test the click test again

FF118 with RFP (new value) - this looks fine, context keys is proven to work, the two oscillator hashes match windows/linux
FF116 with RFP (old value) - context keys will be the same, but the two oscillators hashes should be different

edit: it doesn't really matter, I was just trying to show that it changed. The real test is with the math patches and that looks good. I'll fire up my mac and see what I can wrangle (still grappling with how to run multiple FFs with their own profiles)

karlt · 2023-08-17T07:54:20Z

MacOS M2 Pro (ARM):

Nightly

734ed7d3 [1 metric]
[ i ] offlineAudioContext 1	a34c73cd | a34c73cd | 35.74996626004577
------
[ click ] hash	fe98e3e0 [3 metrics]
audioContext 2	debdefc0 [20 keys] [✓ RFP]
OscillatorNode 2	1348e98d
OscillatorNode/DynamicsCompressor 2	c54b7aa9

Release

c5aeae62 [1 metric]
[ i ] offlineAudioContext 1	93e9b79c | 93e9b79c | 35.73832903057337
------
[ click ] hash	e8457c88 [3 metrics]
audioContext 2	debdefc0 [20 keys] [✓ RFP]
OscillatorNode 2	319ea218
OscillatorNode/DynamicsCompressor 2	e5029720

Thorin-Oakenpants · 2023-08-17T08:42:00Z

~~interesting - your mac and my android are same for offlineAudioContext, but differ on the oscillator tests (I'm ignoring the context keys, we're different platforms)~~

scrap that, just retested. I must have copypasta'ed the wrong hashes - we're the same

karlt · 2023-08-17T09:48:26Z

32-bit Firefox on Windows 11:

Nightly 2023-08-16

765d70d3 [1 metric]
[ i ] offlineAudioContext ^1 24fc63ce | 24fc63ce | 35.749968223273754
------
[ click ] hash                       8ce5922b [3 metrics]
audioContext ^2                      67a3eeee [20 keys] [✓ RFP]
OscillatorNode ^2                    e9f98e24
OscillatorNode/DynamicsCompressor ^2 bafe56d6

Release 116.0.3

2b5f2e29 [1 metric]
[ i ] offlineAudioContext ^1 a0bb5f0d | a0bb5f0d | 35.7383295930922
------
[ click ] hash                       912fcf6c [3 metrics]
audioContext ^2                      67a3eeee [20 keys] [✓ RFP]
OscillatorNode ^2                    4edcaa4d
OscillatorNode/DynamicsCompressor ^2 645b3000

Thorin-Oakenpants · 2023-08-17T10:11:45Z

thanks karl

looks like "easy" (performant, no user gestures) fingerprinting of offlineAudioContext is two buckets: ARM vs non-ARM?

and with RFP, ignoring context keys which is per OS (and we can't hide the OS so this is equivalency), the oscillator non-performant fingerprinting is also two buckets. IDK much about ARM [1], but I guess that's at least a binary split there. We probably can't hide ARM in other tests (IDK, I lack devices)

[1] https://en.wikipedia.org/wiki/ARM_architecture_family#Operating_system_support

fxbrit · 2023-08-18T12:56:35Z

sorry for the delay, never got the e-mail for some reason.

We probably can't hide ARM in other tests (IDK, I lack devices)

isn't you mac ARM? I don't remember if we ever checked for differences in architecture or GPU (webgl) for Apple Silicon and Intel. platform is protected by RFP right? anyway we can do this elsewhere to avoid the noise.

Thorin-Oakenpants · 2023-08-18T18:07:11Z

isn't you[r] mac ARM?

what is yours?* (so I can update OP). Where do I check on mine? (I've used this mac for like a hour total since I bought it - total mac newbie)

I have no clue what I mean when I say ARM. I thought since your mac differed from karlt's and karlt specified ARM as a point of difference (and ARM is expected to cause differences), that this was a mac thing/change with newer models (like M1, M2 - I'm just throwing out words, ignore if I don't make sense)

So there is clearly some difference in macs - I just labelled it ARM because I am ignorant

Thorin-Oakenpants · 2023-08-18T19:35:59Z

excellent .. listen to the expert, not me ;)

edit: as karl points out: the other thing is the math entropy that was in audio is/was equivalency of (at least?) trigonometric math, which RFP also covers

fxbrit · 2023-08-18T22:57:58Z

what is yours?

my Mac is Intel, IIRC yours is ARM. In general, all newer Macs are ARM and they rock the M-series processor (M1, M2).

the other thing is the math entropy that was in audio is/was equivalency

which is something you've been preaching for quite a bit right? so all of this sounds like good news and the game is hardening math if I understood correctly.

Thorin-Oakenpants · 2023-08-18T23:51:24Z

which is something you've been preaching for quite a bit right

Yes. A lot of differences are because of other underlying "causes". So you fix the cause, not the symptom. Or you accept it :)

We cannot hide the OS (win, linux, mac,. android), it's impossible with JS enabled. So here for example, RFP sets a different ac-outputLatency for each of the four OSes. So the RFP context keys reveal OS = equivalency of OS (e.g. userAgent)

or userAgent RFP uses four results, because compat and we can't really hide the OS. So there are four buckets = equivalency of the actual OS. Capisce?

Another example: canvas (depending on how sophisticated your test is) can be a mix of lots of sources of entropy: math, default fonts, font anti-aliasing (e.g. clear type), unicode support, subpixels, default font sizes, and so on. If we made all users have the exact same fonts and nothing else, that would eliminate some direct font fingerprinting, and as a consequence also reduce entropy in canvas.

my Mac is Intel

well then, so it all makes sense, OP is correct :)

Thorin-Oakenpants · 2023-08-19T08:27:48Z

I've added some audio notations (health checks). They do not show unless FF118 or higher. Health check counts should be consistent, so I have to fail them (it alerts us) if they don't match, but of course we do not conclusively know all the results, and for the two oscillator tests, there will be noise/failures from some non-RFP users (but we can determine that on the back end). So technically a red X may be a false positive. I think over time we will get a full picture (and there weren't that many buckets to start with in my opinion from research and others - we may already be there!!)

So anyway, ignoring RFP audioContext keys which has been done since around FF72, each of the three have two known hashes which determines if it says ARM or x86/amd

top is my win11 FF118 with RFP enabled, bottom is a simulation when it fails our known results. Might come in handy for others, not just TZP/tor project - @karlt FYI

Thorin-Oakenpants · 2023-08-19T08:55:07Z

thanks everyone for submitting results .. here's some 🥧 and more 🍻

Thorin-Oakenpants · 2023-11-28T10:28:11Z

@abrahamjuliot re abrahamjuliot/creepjs@cdc6951

since FF118 all firefox users should report one of two sums

x86/amd: 35.749968223273754
ARM: 35.74996626004577

you only picked up on the first

Thorin-Oakenpants · 2024-02-26T02:10:20Z

OK, didn't do a moz-regression yet (not saying it was a regression, I just use that to pinpoint the bugzilla), but beta124 (and nightly) exhibit at least a new FP for x86/amd, maybe also for ARM

new results for my x86/amd is

a7c1fbb6 | a7c1fbb6 | 35.749972093850374 [✗ undocumented]

@abrahamjuliot cc: @karlt

edit: so the change was from Bug 1877221 - Update vendored ffmpeg library to current tip

this bit: https://hg.mozilla.org/integration/autoland/rev/7572eb3fac37 Clamp rolloff factor appropriately in PannerNodeEngine. r=pehrsons

edit: my ARM phone on FF124nightly and then I did an update and am now on FF25nightly, both didn't change, so I think the patch only affected x86/amd

Thorin-Oakenpants · 2024-10-27T03:49:47Z

So this came up in Tor Browser, and I should have been more vigilant - the 24fc63ce hash didn't "change", it split into two = so we went from 2 buckets to 3 buckets, but it's equivalency which is already exposed with e.g. large arrays

https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/43212#note_3099854

Thorin-Oakenpants added fingerprinting help labels Aug 12, 2023

This comment was marked as outdated.

Sign in to view

Thorin-Oakenpants removed the help label Aug 18, 2023

Thorin-Oakenpants closed this as completed Aug 19, 2023

Thorin-Oakenpants mentioned this issue Aug 19, 2023

audioLies + audio data arkenfox/TZP#67

Open

mpeter50 mentioned this issue Oct 4, 2024

Disabling webaudio breaks the Matrix Element web client allo-/ffprofile#300

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

FF118+ audio tests #1701

FF118+ audio tests #1701

Thorin-Oakenpants commented Aug 12, 2023 •

edited

Loading

mik0l commented Aug 12, 2023 •

edited

Loading

MagicalDrizzle commented Aug 13, 2023

fxbrit commented Aug 13, 2023 •

edited

Loading

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

PieroV commented Aug 16, 2023

fxbrit commented Aug 16, 2023

karlt commented Aug 17, 2023

Thorin-Oakenpants commented Aug 17, 2023 •

edited

Loading

karlt commented Aug 17, 2023

Thorin-Oakenpants commented Aug 17, 2023 •

edited

Loading

karlt commented Aug 17, 2023

Thorin-Oakenpants commented Aug 17, 2023

fxbrit commented Aug 18, 2023

Thorin-Oakenpants commented Aug 18, 2023

Thorin-Oakenpants commented Aug 18, 2023 •

edited

Loading

fxbrit commented Aug 18, 2023 •

edited

Loading

Thorin-Oakenpants commented Aug 18, 2023

Thorin-Oakenpants commented Aug 19, 2023

Thorin-Oakenpants commented Aug 19, 2023

Thorin-Oakenpants commented Nov 28, 2023

Thorin-Oakenpants commented Feb 26, 2024 •

edited

Loading

Thorin-Oakenpants commented Oct 27, 2024

FF118+ audio tests #1701

FF118+ audio tests #1701

Comments

Thorin-Oakenpants commented Aug 12, 2023 • edited Loading

test 1 offlineAudioContext FF118+ RFP doesn't matter

test 2 CLICK test FF118+, RFP = ON

mik0l commented Aug 12, 2023 • edited Loading

MagicalDrizzle commented Aug 13, 2023

fxbrit commented Aug 13, 2023 • edited Loading

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

PieroV commented Aug 16, 2023

fxbrit commented Aug 16, 2023

karlt commented Aug 17, 2023

Thorin-Oakenpants commented Aug 17, 2023 • edited Loading

karlt commented Aug 17, 2023

Thorin-Oakenpants commented Aug 17, 2023 • edited Loading

karlt commented Aug 17, 2023

Thorin-Oakenpants commented Aug 17, 2023

fxbrit commented Aug 18, 2023

Thorin-Oakenpants commented Aug 18, 2023

Thorin-Oakenpants commented Aug 18, 2023 • edited Loading

fxbrit commented Aug 18, 2023 • edited Loading

Thorin-Oakenpants commented Aug 18, 2023

Thorin-Oakenpants commented Aug 19, 2023

Thorin-Oakenpants commented Aug 19, 2023

Thorin-Oakenpants commented Nov 28, 2023

Thorin-Oakenpants commented Feb 26, 2024 • edited Loading

Thorin-Oakenpants commented Oct 27, 2024

Thorin-Oakenpants commented Aug 12, 2023 •

edited

Loading

test 1 `offlineAudioContext` FF118+ RFP doesn't matter

test 2 `CLICK` test FF118+, RFP = ON

mik0l commented Aug 12, 2023 •

edited

Loading

fxbrit commented Aug 13, 2023 •

edited

Loading

Thorin-Oakenpants commented Aug 17, 2023 •

edited

Loading

Thorin-Oakenpants commented Aug 17, 2023 •

edited

Loading

Thorin-Oakenpants commented Aug 18, 2023 •

edited

Loading

fxbrit commented Aug 18, 2023 •

edited

Loading

Thorin-Oakenpants commented Feb 26, 2024 •

edited

Loading