Review request for WebVR

[toji]

Hello TAG!

I'm requesting a TAG review of:

• Name: WebVR
• Explainer: https://github.com/w3c/webvr/blob/master/explainer.md
• Primary contacts: @toji, @kearwood, @NellWaliczek

Further details:

• Relevant time constraints or deadlines: Our community group is having a Face to Face August 15th. Having feedback to review at that event would be ideal but not required.
• I have reviewed the TAG's API Design Principles

You should also know that...

WebVR has been reviewed by the TAG previously. The API described in the explainer is the result of complete refactor of the previously reviewed API (in part due to the previous TAG feedback), and is intended to replace the prior version of the API entirely. We occasionally refer to the latest API iteration as "2.0" within the community group for disambiguation, but it is not intended to be publicly presented as a 2.0 API, since that would imply the previous version was still accessible.

We'd prefer the TAG provide feedback as:

• open a single issue in our Github repo for the entire review

[travisleithead]

Taken up at the London F2F.

We were able to get half-way through reviewing the explainer (to the "mirroring" section).

To that point, we're happy with the explainer; very well written! So far, the framework of the WebVR API looks good to us (though we have some general feedback below that could impact that).

In general we have a couple of high-level concerns that don't seem to be described in the explainer so far--we suspect that the Web VR group may have plans/thoughts about these, but that they're just not written down yet.

The primary concerns are related to 1) isolation, and 2) peak performance.

Isolation

One of the two primary use cases described in the explainer is the "VR Enhanced" scenario. In these cases, we are worried that the embedded nature of Web VR will be result in a potentially poor VR experience because the VR render loop will be competing will all the other content being hosted on the page (which will undoubtedly include tens of ads in iframes--bringing the experience to a crawl :( ). In this respect, we think one of the high-order concerns should be isolation--isolating the Web VR experience in a way that UAs can easily de-prioritize non-VR content while the VR session is active. One way to cleanly create this isolation today is to use iframes. Iframe integration could be as loose as suggesting it as a best-practice, or as tight as a fundamental requirement to enabling VR. We don't have a concrete suggestion on how to enable the latter, but posit the idea for the group to consider.

Peak Performance

Regardless of the question of isolation, making the WebVR experience run as smoothly and "jank-free" as possible is critical--this is the concern of enabling peak performance. We notice that OffscreenCanvas is mentioned once in the explainer. We expect that developers will want to use Workers as a way to offload some rendering logic from the main thread. We were hoping to see more about how this scenario might work in the explainer. We note that Web VR interfaces only seem to be exposed in the main thread. Should they be exposed in a Worker? This would enable the vrSession.requestFrame to drive a render loop in a Worker, which is cool. If not, then Alex asks "where's my RAF?" :)

The "where's my RAF?" question is rather important; in general, this appears to be a missing feature from Workers that is critical to synchronizing rendering with OffscreenCanvas and the main thread. While not technically a requirement of WebVR, we do see WebVR having a big interest in ensuring that such a RAF in a worker would be able to run at the refresh rate required by a VR device. Related to the above paragraph, if the VR experience can be isolated into an Iframe, it might be more tenable to "upgrade" RAF for the entire frame (and it's workers) in a synchronized way (than provide some other coordination mechanism).

In addition, using a Worker adds lots of complexity to the question of how to get other main-thread things like user-input and audio (for the game-like scenario). As new input devices are considered, it would be good to think about how that input could be delivered into a Worker environment.

[travisleithead]

Finished our review of the explainer. Looks exciting, and we see how the door is open for AR additions too :).

Our primary concern remains what happens with the rest of the content in the Document (and what is the expected scenario regarding the document). E.g., in an exclusive session, does the browser window on a desktop machine continue rendering? If so, why? On a phone, in exclusive mode, there is probably no rendering going on for the page content, but is it still running (rAF, timers, etc.). We really need to understand what the group's intentions are regarding this, and how it might impact the model, and how it might impact the requestFrame vs. requestAnimationFrame question.

I'll put these into an issue on the WebVR repo as directed above.

[annevk]

Does it require secure contexts?

[travisleithead]

Around the room, lots of heads were nodding, though @slightlyoff notes major pushback from folks on the implementation team. I think the default answer is yes as it brings a new device category onto the web (VR hardware).

[travisleithead]

And [SecureContext] is already applied in their spec IDLs, so that's great: https://w3c.github.io/webvr/spec/latest/

[torgo]

@travisleithead currently crafting some issues to raise in their repo.

[dbaron]

We had an extended discussion of the rAF and timing issues in today's teleconference.

[travisleithead]

At TPAC, there was a lot of good progress made on our TAG feedback, specifically about the questions of isolation and the document.

• There was a proposal to generalize basic rAF so that it can be applied to other objects in the platform (besides just the window) as well as extended such that the callback's parameter might be extensible. This seemed like a good direction to me and @slightlyoff who were in the room. It was noted that WorkerGlobalScope might want rAF, as well as [potentially] the HTML media elements to enable content-framerate-based callback timing. Until this larger refactoring of rAF is made (which will probably track in HTML), the VR folks will spec a compatible signature to unblock themselves.
• Isolation: there was general acceptance that iframe is the tried and true (and best) way to aid in isolation. Use of iframe would not be required, but certainly recommended for embedding scenarios. We discussed how some kind of addition "high performance" mode flag/switch might be appropriate to use on the iframe to help signal the platform to optimize it's content (at the expense of the rest of the page, for example).

[travisleithead]

Some remaining things to follow-up on:

• How to know what framerate should be used for RaF (especially when in use in a OffscreenCanvas whose function is VR-related)
• Input and relation to Gamepad?
• 'high performance' (or similar) declaration for iframes

[travisleithead]

I'll note that the Gamepad spec has quite a number of issues related to a v2 that could address our follow question of "Input and relation to Gamepad" above:

https://github.com/w3c/gamepad/issues?q=is%3Aissue+is%3Aopen+label%3Av2

[travisleithead]

Hey @cvan and @toji (and others you'd like to invite), the TAG would love to follow-up with you on the latest developments around our previous feedback at our upcoming teleconference on May 8th. If you can make it, please follow-up with @torgo, myself, or @slightlyoff for telecon details. Thanks!

[dontcallmedom]

also paging @NellWaliczek and @TrevorFSmith

[TrevorFSmith]

@travisleithead @torgo I'll make time for this call. [email protected] is my email for follow-up.

Google I/O is May 8-10 so I would guess that the Googlers might be pretty busy.

[kearwood]

@torgo @travisleithead @slightlyoff I'll be available. My email is [email protected]

[torgo]

Suggestion on the table is to put this on our agenda for the 29th of May call (15:00 UTC). @travisleithead to reach out.

[torgo]

We discussed this long-open issue on the call 25-Sept 2018. We agreed to discuss any further topics relating to this issue at TPAC with a view to closing it out.

[cwilso]

@torgo I think we have concluded it makes the most sense to close this design review out, and that we would like in the future to request a TAG review of Web_XR_ Device API, but not until after TPAC. (We're still hammering out design decisions there.) . How does that sound?