Should devicechange fire when the device info changes?

[chrisguttandin]

I noticed a browser inconsistency in the way the 'devicechange' event gets fired. But I think the spec isn't very clear which browser is behaving correctly.

These days all browsers reduct the information available by querying enumerateDevices() as long as the page has for example no permission to access the microphone. However this changes after a successful call to getUserMedia(). This behavior seems to be similar across all browser.

But only Safari fires a 'devicechange' event when it exposes the label of a device as a result of a permission change. This inconsistency can be easily worked around by calling enumerateDevices() again after each call to getUserMedia() but it doesn't work if the user changes the page permission in the browser settings.

The spec currently says ...

When new media input and/or output devices are made available to the User Agent, or any available input and/or output device becomes unavailable, or the system default for input and/or output devices of a MediaDeviceKind changed, ...

At least on my MacBook Chrome is the only browser which actually adds a device after the user has granted permissions to see the full list of devices. It shows the default device twice. However it doesn't fire a 'devicechange' event.

Firefox only changes the label which I would argue means the device is still the same and therefore 'devicechange' shouldn't fire at least in the way it is defined today.

Safari changes all properties including deviceId and groupId which could be interpreted as a completely new device. It consequently fires a 'devicechange' event.

As a developer building applications using this API I would love all browsers to fire a 'devicechange' event whenever anything changes in the device list including label changes.

[guidou]

I agree that the event should be fired on a page when the result of enumerateDevices() changes for that page.
This would cover not only physical additions and removal of devices, but also additions and removals due to changes in permissions, device ID changes due to browsing data deletion, changes to default devices, or any other reason that affects the view of devices for the page.

[youennf]

I agree the current spec is a bit restrictive.
Maybe we should have a more global statement that triggers the device change algorithm and a note identifying the cases we know of to help implementors.

[jan-ivar]

I'm not in favor of changing when devicechange fires. Its intent was to denote real device changes, not changes to exposure or permission.

The spec is in much better shape than current implementations, so I caution against making inferences from observations of current browsers:

1. Safari is closest to spec
2. Firefox with media.devices.enumerate.legacy.enabled set to false in about:config is shipping soon to match Safari
3. Chrome and Firefox still work the "legacy" way, which is crbug 1101860 in Chrome.

But only Safari fires a 'devicechange' event when it exposes the label of a device as a result of a permission change.

Exposure tied to permission is legacy. In the spec and Safari it's instead tied to calling getUserMedia, which is much simpler.

From testing with this demo page, Safari only fires this event if the app calls enumerateDevices first, a bug?

This inconsistency can be easily worked around by calling enumerateDevices() again after each call to getUserMedia() but it doesn't work if the user changes the page permission in the browser settings.

It sounds like Safari here is taking advantage of the following note:

Apps can use permissions.query for that.

I don't think the app is owed a devicechange event in that special case. If anything, apps are better off holding on to the lists they already have and NOT update in this case which will just erase the info.

Once implementations align, it should simplify things a lot for developers, and there should be no need to want getUserMedia to fire the devicechange event.

[jan-ivar]

To me, the primary use case for devicechange is headset insertion detection. Specifically: detecting the user act of plugging in or putting on their headphones, and reacting to that action like they had pressed a key on their keyboard.

Many VC sites rely on it for that: they either auto-switch to the headset mic upon insertion, or trigger a toast message with a button to conveniently and immediately switch to the headset mic. Firing it for other reasons may interfere with that.

For example: a user puts on their AirPods while joining a meeting or meeting lobby (not uncommon).

If getUserMedia always fires devicechange and the user has more than one device of a kind, how is the site to know whether the user just inserted that second device or not?

[guidou]

Insertion detection is possibly the primary/most common use case, but not the only one.
Applications may be interested in more general device availability, which covers other changes to the result of enumerateDevices. That doesn't mean gUM should fire devicechange every time (or even any time). We can update the text to reflect that.

[jan-ivar]

Once all browsers implement the spec correctly, it should be normal to expect:

1. Near-zero information from enumerateDevices() ahead of gUM() (basically: mic absence and/or cam absence)
2. Full access to device information of the granted kind(s) post gUM()

I see no value in ever firing devicechange from 1 to 2, even if we expand sensitivity to changes in information in 2.

Applications may be interested in more general device availability, which covers other changes to the result of enumerateDevices.

What sort of changes? The text mentions changes to the "the system default", which can be a signal that a user may wish to (auto) switch, so belongs on the list IMHO. Anything else?

[guidou]

Once all browsers implement the spec correctly, it should be normal to expect:

1. Near-zero information from enumerateDevices() ahead of gUM() (basically: mic absence and/or cam absence)
2. Full access to device information of the granted kind(s) post gUM()

I see no value in ever firing devicechange from 1 to 2, even if we expand sensitivity to changes in information in 2.

If an application is interested in tracking changes to the result to enumerateDevices(), it can just call enumerateDevices() in a loop to detect them. Firing devicechange is a better way to avoid this inefficient way of doing it.

Applications may be interested in more general device availability, which covers other changes to the result of enumerateDevices.

What sort of changes? The text mentions changes to the "the system default", which can be a signal that a user may wish to (auto) switch, so belongs on the list IMHO. Anything else?

The result of enumerateDevices() might change (depending on UA choice of implementation) due to:

• UA-level Permission revocation.
• OS-level permission revocation.
• OS-level disabling of a device without unplugging it (e.g., user closing the laptop lid and the OS suspending the camera as an active device, or the user disabling the device via an OS setting).
• User clearing browsing data, which causes device IDs to change.

All these changes are detectable by polling enumerateDevices(), and should be available via devicechange to avoid having applications inefficiently enumerating devices in a loop.

None of this implies firing devicechange every time gUM is called, which I see as an orthogonal issue. An application can call gUM multiple times while the result of enumerateDevices is visible. If the result of enumerateDevices() doesn't change as a consequence of those calls I see no reason to fire devicechange.

[jan-ivar]

I haven't heard why an application might be interested in more general device availability.

Also you haven't addressed my concern that this interferes with detection of device switching, which you agreed "is possibly the primary/most common use case".

Without a valid use case, I'd say let them poll.

[guidou]

I haven't heard why an application might be interested in more general device availability.

I have, and now so have you. I'm aware of applications that want to know when the set of devices change to decide if/when to make a new getUserMedia call or to provide a notification to the user (e.g., to request permission again).

Also you haven't addressed my concern that this interferes with detection of device switching, which you agreed "is possibly the primary/most common use case".

Why/how does it interfere with detection of device switching?

[youennf]

Firing it for other reasons may interfere with that.

AFAIK, Safari behaviour does not seem to interfere with this.
Also, the web application needs anyway logic to understand why devicechange fired (say a device that is not capturing is removed...), I do not think this application logic will be much complex w/o this additional event firing.

It seems convenient for web developpers that devicechange event is fired whenever an in-webpage device picker UI would need to be updated. This includes changes to label. I do not see any real downside to this approach.
Getting consistency across UAs seems like a good idea.

[jan-ivar]

I'm aware of applications that want to know when the set of devices change to decide if/when to make a new getUserMedia call or to provide a notification to the user (e.g., to request permission again).

I'm asking about the use case, the real-world problem that requires this change that isn't possible today.

Earlier I clarified two reasons why the observable set may change:

1. a "real" device change (use case: app wants to react to a user having just inserted a new physical device)
2. getUserMedia ungates full device info of more than one of a kind of device to choose from

I trust it's clear that if the same event fires for both 1 and 2 then that interferes with detecting 1. The problem use case I gave was getUserMedia racing with the user putting on their headset.

Number 2 is the norm frankly. Most devices (cellphones) have more than one device of a kind. In the spec, apps should assume they don't have the full set of information ahead of gUM (like in Safari), and not need an event to tell them this.

if/when to make a new getUserMedia call

Asking for getUserMedia twice is an anti-pattern. This is why we have constraints — and why Firefox has a picker — to help pick the correct device up front. Per-device permission models work poorly otherwise. I'm not going to call out specific VC apps, but you know who you are.

or to provide a notification to the user (e.g., to request permission again).

Using enumerateDevices to detect permission is also an antipattern. Hopefully this clarifies why I'm opposed to this change.

[youennf]

The problem use case I gave was getUserMedia racing with the user putting on their headset.

In that use case, what we are after is a way for the web application to know what triggered the devicechange event.
Given the web app has an empty list returned by enumerateDevices prior the getUserMedia call, it cannot compute this information by calling enumerateDevices again. Maybe there is a change of default input, or a removal of an unused device, how can the web app know?

To help the web application, there are different approaches we could consider, for instance:

1. Fire a first devicechange event just after gerUserMedia resolution, to allow the web application grab the list of devices that were considered as part of the first getUserMedia call. Then fire a second devicechange event for the newly introduced device.
2. Provide more info directly on devicechange event, say a reason attribute containing a list of enums ('default-change', 'device-addition', 'device-removal').

[guidou]

I'm asking about the use case, the real-world problem that requires this change that isn't possible today.

I've already mentioned it. Update the UI to reflect available devices, notify the user, know when/if to call getUserMedia again. Is your position that those are use cases we should not support via devicechange? We support them anyway via calling enumerateDevices() in a loop.

Earlier I clarified two reasons why the observable set may change:

1. a "real" device change (use case: app wants to react to a user having just inserted a new physical device)
2. getUserMedia ungates full device info of more than one of a kind of device to choose from

I trust it's clear that if the same event fires for both 1 and 2 then that interferes with detecting 1. The problem use case I gave was getUserMedia racing with the user putting on their headset.

Why? The preferred pattern is to use plan events, like devicechange and then let the application examine the updated state to determine what happened and/or what needs to be done to properly react to the event.
Do you also not support firing the event when the default device changes? Does that not interfere with detecting addition/removal?

Number 2 is the norm frankly. Most devices (cellphones) have more than one device of a kind. In the spec, apps should assume they don't have the full set of information ahead of gUM (like in Safari), and not need an event to tell them this.

if/when to make a new getUserMedia call

Asking for getUserMedia twice is an anti-pattern. This is why we have constraints — and why Firefox has a picker — to help pick the correct device up front. Per-device permission models work poorly otherwise. I'm not going to call out specific VC apps, but you know who you are.

What is wrong about calling gUM more than once? And what is wrong about updating a UI that lists available devices based on a change in the result of enumerateDevices()?

or to provide a notification to the user (e.g., to request permission again).

Using enumerateDevices to detect permission is also an antipattern. Hopefully this clarifies why I'm opposed to this change.

enumerateDevices() would be used to examine the updated state, just like with any other plain event. Not specifically to check permissions.
Let's say the user does something that makes a device unavailable (other than physically removing it). What is wrong about the application informing the user that it can't capture anymore, and waiting for the device to become available to call gUM() again?

[guidou]

2. Provide more info directly on devicechange event, say a reason attribute containing a list of enums ('default-change', 'device-addition', 'device-removal').

I would prefer to keep devicechange a plain event, to stay in line with https://www.w3.org/TR/design-principles/#state-and-subclassing.

[youennf]

1. Fire a first devicechange event just after gerUserMedia resolution

I guess we could also ensure enumerateDevices expose the set of devices without devicechange event at the time getUserMedia resolves. This still seems much more brittle and difficult to reason about.

[youennf]

I would prefer to keep devicechange a plain event, to stay in line with https://www.w3.org/TR/design-principles/#state-and-subclassing.

Agreed, I'd like if possible to keep allowing web apps to just resort on enumerateDevices to figure out what is happening.

[jan-ivar]

I think @youennf is on to our disconnect: he says what I want never worked. That is a key insight.

In that use case, what we are after is a way for the web application to know what triggered the devicechange event.

Yes.

Given the web app has an empty list returned by enumerateDevices prior the getUserMedia call, it cannot compute this information by calling enumerateDevices again.

That's not a satisfying answer, because apps will try even if they cannot compute it accurately, which seems like a design flaw.

In Firefox and Chrome, the app knows that a user-initiated action MUST have happened or it wouldn't have received the event. In Safari it doesn't have this information.

Maybe there is a change of default input,

I count that as a valid user-initiated action worth reacting to (e.g. macOS seems to reorder defaults in response me taking my AirPods out).

or a removal of an unused device, how can the web app know?

Here you've stumped me. Yes, it's possible that with 3 or more devices of a kind, removal of a 3rd device upon joining a meeting might be misinterpreted as insertion of the 2nd.

Your argument is basically: it won't work anyway. :)

But it probably works MOST of the time, right?

In any case, I prefer: it won't work anyway, so let's fix it. Not, it won't work anyway, so let's not.

Also, the web application needs anyway logic to understand why devicechange fired (say a device that is not capturing is removed...),

Yes. Let's talk about how complicated writing said code already is, and what a web compat nightmare it must be. Take this fiddle, but keep AirPods in their case before opening it, then put them on. This is on macOS Ventura:

Firefox:

Switching to inserted AirPods

Chrome:

ignoring duplicate event
Switching to inserted AirPods
AirPods removed; reverting to Default - AirPods
Default - MacBook Pro Microphone (Built-in) removed; reverting to Default - AirPods

Safari:

MacBook Pro Microphone removed; reverting to MacBook Pro Microphone
Switching to inserted AirPods

I think it's quite hard to reason about both this application code and browser behaviors here, and I suspect there's a lot we could do to improve this situation.

To start, just to cut down on races, it might help to mandate that browsers delay firing a devicechange event while a call to enumerateDevices is outstanding...

[jan-ivar]

I'd like if possible to keep allowing web apps to just resort on enumerateDevices to figure out what is happening.

I'd like for someone to show how an app can do that and cover all the edge-cases we've discussed so far (racing with gUM's device exposure gate, racing with enumerateDevices itself, relying on pre-gUM vs post-gUM baseline, handling pre-gUM insertion (from 0→1 devices).

The more I think about all the edge-cases I'm not confident apps will be able to decode device insertion correctly in all cases. Do we need a discrete deviceinserted event?

[guidou]

I'd like for someone to show how an app can do that and cover all the edge-cases we've discussed so far (racing with gUM's device exposure gate, racing with enumerateDevices itself, relying on pre-gUM vs post-gUM baseline, handling pre-gUM insertion (from 0→1 devices).

Races are still possible even if you restrict the event to insertion/removal. Sometimes they can be outside the UA (e.g., at the OS level, different processes can have different views of the set of devices). We can mitigate them with mechanisms like @youennf's proposal to delay firing a devicechange event while API calls are outstanding, but probably not eliminate them entirely.

The more I think about all the edge-cases I'm not confident apps will be able to decode device insertion correctly in all cases. Do we need a discrete deviceinserted event?

I'm open to having another event type, but I don't think that will completely solve the generic problem of races.
Also, I don't think these races are a major problem in practice for VC applications, since they are very rare and the consequences aren't particularly terrible.

Being able to present the correct choices in a UI when the set of available devices changes (no races) looks like a more important concern IMO.
I believe in most cases the application is more interested in knowing the latest state of devices than the causes that led to that state.

[jan-ivar]

I don't think it's accurate to say there are unavoidable races here. They're API races we designed. See #972.

I believe in most cases the application is more interested in knowing the latest state of devices than the causes that led to that state.

Then devicechange is not the event for them:

It clearly says the event denotes a change in devices available to the "User Agent" (i.e. an OS change), not to the web page.

[guidou]

It clearly says the event denotes a change in devices available to the "User Agent" (i.e. an OS change), not to the web page.

The proposal is to change the spec to make it denote a change in devices available to web page since that is more useful to Web applications.

[guidou]

I don't think it's accurate to say there are unavoidable races here. They're API races we designed. See #972.
Hopefully, you're right and we can find a way to avoid them or significantly mitigate them. I agree with discussing that in a different issue since that is separate from the discussion about firing the event when the result of enumerateDevices changes.

[guidou]

It clearly says the event denotes a change in devices available to the "User Agent" (i.e. an OS change), not to the web page.

Also, a change in devices available to the user agent covers more than just physical insertion/removal.
Many other changes occur at the OS level such as changes in labels, the device becoming enabled/disabled at the OS level, changes to the default device, etc.

[jan-ivar]

Many other changes occur at the OS level such as changes in labels,

When do labels change at the OS level?

the device becoming enabled/disabled at the OS level,

When is this NOT user-action driven? Insertion and removal (already antiquated USB terms in a Bluetooth world) already cover this since that's obviously all browsers can observe.

As long as the user acted, it's a reasonable app decision to assume intent and respond to those actions, e.g. if done during a live call or a selection screen.

changes to the default device, etc.

Again, when is this NOT user-action driven?

Hopefully, you're right and we can find a way to avoid them or significantly mitigate them. I agree with discussing that in a different issue since that is separate from the discussion about firing the event when the result of enumerateDevices changes.

It's not entirely separate, since in #972 I also show specifically how Safari's decision to fire devicechange from getUserMedia registers as a false positive, causing my demo page to react to a device having been inserted, when this did not happen.

What's proposed here would essentially declare Safari correct and the other browsers wrong, when the spec says the opposite.

I'm happy the way the spec is already written, and look forward to more implementations aligning to solving its primary use case better.

Regarding the problem use case of in-content UX not being updated, I'd like to understand specifically where the existing event falls short, and why you can't work around it.

[eric-carlson]

the device becoming enabled/disabled at the OS level,

When is this NOT user-action driven? Insertion and removal (already antiquated USB terms in a Bluetooth world) already cover this since that's obviously all browsers can observe.

This happens to me fairly frequently when the camera in my external monitor disappears while in use when the driver (or maybe USB bus?) crashes.

When this happens the OS fires an event when the camera disconnects, and another when it reconnects. The events are the same ones fired when a camera is connected/disconnected manually by the user.

[jan-ivar]

Thanks, it's interesting to explore real use cases. My first reaction is that this says it's a bug if it's not user-initiated. But then I suppose similar situations may happen if I temporarily move out of Bluetooth range during a meeting.

But in both cases, if this happens, I think most people would find it valid for apps to try to auto-switch back to the device when it comes back, so even if it's not 100% user-initiated, it seems desirable to categorize it as a devicechange worth reacting to.

[jan-ivar]

This was discussed in https://www.w3.org/2023/10/17-webrtc-minutes.html:

Conclusion: More discussion needed, but we seem to agree on the problems we need to solve.

[dontcallmedom-bot]

This issue was discussed in WebRTC Interim, May 21st – 21 May 2024 (Issue 966: Should devicechange fire when the device info changes? (Jan-Ivar))

[dontcallmedom-bot]

This issue had an associated resolution in WebRTC June 18 2024 meeting – 18 June 2024 (Should devicechange fire when the device info changes?):

RESOLUTION: Continue discussion on PR towards merging