API proposal

[dmarcos]

Follow up of immersive-web/webxr#517. I would like to propose a simple minimal mechanism for UAs to grant permission to enter immersive mode. This enables in-vr navigation, allowing VR and AR content to interlink other immersive content and for UAs to implement seamless transitions. Some of the valuable use cases previously discussed:

• Performance implications. Having different web pages load is sort of beneficial for not having to deal with offloading / onloading new models and environments or making your own interstitial. You can avoid frame drops by letting the browser handle the transition, and VR web pages load quick enough.
• Games that contains links to other games. Or links to friends' games.
• Personal blogs that link to affiliated VR sites or your friends blogs.
• Educational websites for a class, that link to the school's portal or other resources.
• User generated content linking to one another. Like Rec Room, VR Chat or Super Craft.
• Sponsored links. You have a VR world about photography and film-making that links to Photorama, B&H, GoPro's VR portals.
• Content portals in general (directories of games, immersive videos, collections).
• Personal storefronts that link to products hosted by eBay VR, Amazon VR. A page could easily link to product 3D models hosted externally in services the like of Sketchfab or Google Blocks.
• AR browsers where pages always present at load to display virtual objects in context.
• Open in immersive mode A browser in 2D mode could offer the possibility to open a link in immersive mode immediately.

To enable previous scenarios I can think of two different spec approaches:

1. Implicit, without any additional API. The spec could describe scenarios where entering immersive mode is exempt of the user gesture requirement. Like for example a page navigated from another that was already presenting.

2. sessiongranted event. UAs can fire an event that explicitly allows content to enter immersive mode. The event handler is not subject to the user gesture requirement. This is how the API could look like:

navigator.xr.addEventListener('sessiongranted', function (evt) {
   // One could check for the type of session granted.
   // Events notifies of session creation after navigation, UA action, or requestSession.
   // The session object is provided as part of this event.
   if (evt.session.mode === 'immersive-vr') {
      // set up app state for immersive vr, if that's what the app wants
   } else {
      // notify user that this app only works in immersive vr mode, if desired
   }
}

The spec could also include guidelines covering scenarios where user's security and privacy can be compromised. A series of high level recommendations without getting into the specifics on how browsers would work as it's not in the spec scope

e.g: During navigation in immersive mode UAs must guide and keep the user informed at all times. Possible mechanisms are:

1. Interstitial. When navigation occurs the UA can present the user a screen with the destination URL and a button to continue or return to previous site.
2. Overlays. UAs control compositing and can always render information about the navigated page over content.
3. Modal dialog. User can invoke the browser UI at all times to consult info about the current site. UAs UI can also be modal (similar to Oculus or Steam home spaces) and invoked with a reserved button or interaction that cannot be listened by content. To prevent spoofing, UAs could additionally display personal information that user can recognize and pages don't have access to (user generated doodles or environment choices)
4. Fall back to windowed mode. Some UAs might decide to retain a more traditional flow and always fallback to windowed / 2D mode on navigation.

I recommend reading @toji explainer for additional context and previous thoughts on the topic.

I'm happy to volunteer to put together a more formal proposal after general agreement.

Ping to participants in previous conversation @AlbertoElias @blairmacintyre @cvan @lincolnfrog @modulesio @rcabanier @klausw @avaer @n5ro @kfarr @disregardfiat @AdaRoseCannon @TrevorFSmith @Artyom17

[avaer]

Thank you for putting this together and getting the conversation started.

Overall I agree with the approach, though I'm slightly more in favor of option 2:

• it seems approach 1 would result in applications trying and failing to enter VR on load in some cases by design (i.e. we are not coming from VR), which is non-ideal for developer ergonomics
• sessiongranted is a vehicle for carrying over the context of the existing session (such as VR/AR mode, or permissions), which would be useful context for the target page to use -- for example to skip permisson UI
• WebVR has set precedent with vrdisplayactivate, which frameworks are designed to handle, so it would be a simpler loading architecture update

[blairmacintyre]

I'm in favor of the sessionGranted event. I've been a proponent of supporting UA-driven methods for entering/exiting XR mode (in addition to page initiated actions) and this aligns well with that.

I would add to your items that we should carefully consider the impact of same-origin vs cross-origin permissions and guidance to implementers. If I'm at a URL and follow a link to the same origin (bmaci.me/cool-webxr-thing1/mode1 -> bmaci.me/cool-webxr-thing1/variation2) the security is different then if I navigate to (diegossite.fun/xrgame)

[dmarcos]

@blairmacintyre Thanks. Sorry, I forgot to add the use cases you mentioned in the other issue. Edited above to include them. Let me know if there's any other scenarios we should cover.

[blairmacintyre]

I think it would be good to actually create a draft doc, if you’re up for it, where we can all comment on specifics. I don’t really want to have another long discussion thread here!

It’s pretty clear that there are some things we might all agree on, and some things that will be contentious.

[AlbertoElias]

Thanks a lot for putting this together @dmarcos!

I definitely think sessionGranted is the way to go, and I think it's a good name as well. I agree we should mention security concerns and possible solutions, but that a specific suggestion is out of scope.

I also remember from previous conversations the idea of a meta tag and/or HTTP Response Header. Browser vendors will be able to address this better, but could those optional add-ons improve performance? It does take a while to load and parse JavaScript, even more with WebXR where we probably have other libraries like Three.JS/Babylon.js/Polyfills.

I agree with @blairmacintyre that we should move this to a PR where we can comment on it better. Maybe as an update to the navigation explainer? And maybe a separate PR with the specific change to the WebXR API? Let me know what I can help with

[blairmacintyre]

@AlbertoElias I meant a PR on a document in this repo; it's premature to move things to the main repos until we sort out here what we can agree on!

So, create explainer.md here with the proposal and submit as a PR against this repo. Could have multiple files (e.g., one that is "things that will go to navigation explainer" and one about API./)

Heck, we could just copy the navigation explainer here and then do a PR against it, here.

[AlbertoElias]

Yup, that's what I meant, in this repo, but maybe based on those documents

[Utopiah]

To confirm what was clearly discussed earlier :

• yes kiosk mode is very important, from trade shows to school, the ability to remove 1 step to have an experience ready from boot to VR is extremely useful. Everybody here is highly skilled and is not afraid of technology in general. Other people though have to rely on those experiences in a professional context might not have the same enthusiasm and skills. Consequently having the ability to go directly to content is extremely valuable. I personal have clear need for this at UNICEF for our startups that want to provide educational content to teachers but also at the European Parliament where we try to demonstrate immersive content to users and their colleagues that have no time for those "small extra steps".
• entering from one experience to another is by definition the link. I find this shocking that is this is still discussed as secondary. We all, literally all of us, reached this very page through links. If my browser window was in fullscreen then it remains so. I think quite obviously the same principle of preserving a browsing context should be applied to immersive content.

[cabanier]

If my browser window was in fullscreen then it remains so.

That is incorrect. If you browser was in full screen, it would exit and go back to non-fullscreen mode upon a navigation

[Utopiah]

If you browser was in full screen, it would exit and go back to non-fullscreen mode upon a navigation

Does it? I just tried with latest Firefox, Chrome and Edge. I press F11, navigate from one link to another and remain in fullscreen. Am I doing it wrong?

[cabanier]

If you browser was in full screen, it would exit and go back to non-fullscreen mode upon a navigation

Does it? I just tried with latest Firefox, Chrome and Edge. I press F11, navigate from one link to another and remain in fullscreen. Am I doing it wrong?

There are different types of full screen :-)
If you use the full screen API (ie full screen YouTube), it will exit that mode upon navigation
If you make the browser itself fullscreen, notice how you can still see the url bar and navigation buttons.

WebXR is currently defined like the former

[Utopiah]

Well I think the behavior of actively entering VR or immersive mode is equivalent to pressing F11. If I consciously enable it I don't expect it to revert without me actively disabling it.

[dmarcos]

Continuing discussion in PR as requested #2

Looking forward to your feedback.