WebXR Device API

[Manishearth]

Saluton TAG!

I'm requesting a TAG review of the WebXR Device API. We have previously requested review here and incorporated all the feedback; now that we are approaching CR I wanted to request review a second time in case the TAG wanted a chance to provide any final feedback.

The WebXR Device API provides access to input and output capabilities commonly associated with Virtual Reality (VR) and Augmented Reality (AR) devices. It allows you develop and host VR and AR experiences on the web.

• Explainer¹: https://github.com/immersive-web/webxr/blob/master/explainer.md
• Specification URL: https://immersive-web.github.io/webxr/
• Tests: https://github.com/web-platform-tests/wpt/tree/master/webxr
• Security and Privacy self-review²: We have an explainer and a spec section on security and privacy.
• GitHub repo (if you prefer feedback filed there): github.com/immersive-web/webxr
• Primary contacts (and their relationship to the specification):
  • @Manishearth (Editor, Mozilla)
  • @toji (Editor, Google)
  • @AdaRoseCannon (WG Chair, Samsung)
  • @cwilso (WG Chair, Google)
  • @Yonet (WG Chair, Microsoft)
• Organization(s)/project(s) driving the specification: Immersive-Web Working Group
• Key pieces of existing multi-stakeholder review or discussion of this specification: Previous TAG review at WebXR Device API #403
• External status/issue trackers for this specification (publicly visible, e.g. Chrome Status): https://www.chromestatus.com/feature/5680169905815552 , https://developer.mozilla.org/en-US/docs/Web/API/WebXR_Device_API#Browser_compatibility , it's also released on some versions of Firefox Reality (which does not have its own status page)

Further details:

• I have reviewed the TAG's API Design Principles
• Relevant time constraints or deadlines: I'm hoping we can request CR in a month or so, so ideally before that, but I understand if the wide review process takes longer. Review is the main remaining CR blocker.
• The group where the work on this specification is currently being done: Immersive web Working Group
• Major unresolved issues with or opposition to this specification: Not that I know of
• This work is being funded by: ?

You should also know that...

[please tell us anything you think is relevant to this review]

We'd prefer the TAG provide feedback as (please delete all but the desired option):

☂️ open a single issue in our GitHub repo for the entire review

[torgo]

Hi WebXR folks! Thanks for this opportunity. The TAG is very happy with the course this work has taken, and we are generally satisfied with the explainer and the shape of the API.

One critical comment however: We note that in Alice's original comment from a year ago she mentions accessibility as one of the areas of concern. It seems from the spec like no work has been done in this area - there is no mention of accessibility in the spec. We seem to remember a separate accessibility requirements document being worked on but there is no mention of this or sign of this in the top level repo. We feel like there should be some additional mention of this - even if it's non-normative - just to indicate what work has happened so far, considering this was one of the issues raised in the last review?

In the previous review comments Brandon specifically said "I think it would be more productive for us to outline our current thinking about accessibility in a separate doc which we'll link here."

Alice also had some specific questions that weren't answered.

• Could the Web Audio, Vibration and Gamepad APIs make use of XRViewerPose to provide this immersive experience? How does that work with the frame-based mechanism for updating the XRViewerPose? Could the explainer (or another document) provide some example code for linking these things together?
• For users who require magnification, might it make sense to have an option on the viewport to perform appropriate scaling automatically?

On a meta level: this is a new technology, and the very best time to think about accessibility is when things are being newly designed. It would be a true missed opportunity to exclude disabled people for years until the technology "catches up", when it could instead be designed with many more diverse needs in mind.

We believe there actually is on going work in the working group to address this; it would be enough just to have visible artifacts available to see exactly what work is being done, and to enable participation as appropriate.

[Manishearth]

For users who require magnification, might it make sense to have an option on the viewport to perform appropriate scaling automatically?

This does not make sense, since it's fundamentally an immersive experience and you cannot scale a reality.

Could the Web Audio, Vibration and Gamepad APIs make use of XRViewerPose to provide this immersive experience? How does that work with the frame-based mechanism for updating the XRViewerPose? Could the explainer (or another document) provide some example code for linking these things together?

No, however you can do it the other way around: You can use the WebXR API to drive WebAudio, and user-agents can disable visual rendering. We could add some text allowing this.

[torgo]

Hi folks @toji @Manishearth @AdaRoseCannon we're just picking this up at our virtual f2f to see if we can close it. I'll note that the accessibility issue that was raised a while back is still open. It would be great if you could put some additional energy there. We also have been thinking about the privacy issues related to fingerprinting. In the case where a shopping site, for example, is invoking the WebXR device API it may be possible for that site to determine that the user has the "latest and greatest" gear. In some cases this may not be beneficial to the user, as the site may then choose to jack up prices. Some sites are known to employ this kind of mechanism (arguably a "dark pattern"). It may be useful in some circumstances to allow the user to only advertise "basic support"... Have you considered this abuse case and possible mitigations?

[Manishearth]

Yeah, we've been meaning to write up that document, it's been taking some time.

We've worked with the privacy WG to figure out an acceptable solution for fingerprinting. Importantly: it takes a bunch of consent to get to the point where a website will know what hardware a user has (you basically have to start a session). The API that produces capability info without starting a session is extremely coarse grained ("vr support"/"ar support"/"none"), and may also have a consent prompt depending on the situation in the UA. Note that most mobile phones running Chrome have "ar support" now so this is not really a signal of affluence.

See https://immersive-web.github.io/webxr/#issessionsupported-fingerprinting

[torgo]

Just noting that work is progressing on the accessibility requirements which is great to see. On this basis we're happy to close this issue. We look forward to a WebXR future.

[Manishearth]

Thank you! 😄