Include malicious sites and surveillance cameras in the PTZ explainer

[beaufortfrancois]

We should document in the PTZ explainer how the proposed API protects users against malicious sites and surveillance cameras in the explainer.

Let's use this issue to discuss how PTZ works today with the Field of View and differences between mechanical and digital PTZ.

@reillyeon @riju @kenchris @eehakkin

[kenchris]

Are we doing any restrictions on when PTZ parameters can be changes, like does the document have to be focused and the video stream be visible?

[riju]

A bit hand wavy now (will try to get more details) -

Pan and Tilt works within that bound field of view of that camera , most cameras and i think all webcams for personal use / video conf, pan and tilt are NOT mechanical, in very small case, zoom can be at most mechanical.

[kenchris]

Pan and Tilt works within that bound field of view of that camera , most cameras and i think all webcams for personal use / video conf, pan and tilt are NOT mechanical, in very small case, zoom can be at most mechanical.

But you cannot assume that is the case in the future and that this API will not apply to such cameras. Also I have been to many meeting rooms there the professional cameras could definitely pan and tilt mechanically.

[kenchris]

Even the Google professional Hangouts hardware has mechanical PTZ offerings and will presumable use the web versions of Hangouts (@beaufortfrancois can confirm):

[reillyeon]

The plan is to treat PTZ as separate permission from camera access. Requesting this permission will require the usual set of anti-abuse requirements such as a secure context and user gesture. This sufficient for camera access and we feel it is sufficient for PTZ as well.

I think @kenchris's suggestion to also require the page to be visible in order to change PTZ parameters is a good idea to explore but I am concerned that there are cases where it would cause trouble. For example, when running a presentation the video conferencing application may be hidden but should still be able to use PTZ to focus on the presenter's face as they move about the room.

[reillyeon]

There are some possible ways that User Agents could enhance the existing notifications that the camera is in use. For example,

1. Chrome provides a notification icon in the tab when recording is active. A preview of the camera's current field of view could be displayed when hovering over this icon.
2. A notification could be shown when the camera has been commanded to move or zoom.

The API as proposed seems sufficiently expressive to allow implementations to experiment with various anti-abuse mechanisms.

[riju]

User Gesture and Focused Frame (Page Visibility API) might hurt user experience and hamper some use cases, so we might prefer to keep this at the discretion of UA and add a NOTE to the spec. PTZ being a separate permission allows such hooks to the UA to experiment with what works best for their users.

[kenchris]

"Page Visibility API" - should probably be a check whether the video element is visible, like intersection observer wise

[beaufortfrancois]

Having access to the stream does not necessarily mean there is a <video> element associated to it, and visible as well.

const videoStream = await navigator.mediaDevices.getUserMedia({
  video: { pan: true, tilt: true, zoom: true }
});

// Code below is optional
const video = document.querySelector("video");
video.srcObject = videoStream;

[reillyeon]

User Gesture and Focused Frame (Page Visibility API) might hurt user experience and hamper some use cases, so we might prefer to keep this at the discretion of UA and add a NOTE to the spec. PTZ being a separate permission allows such hooks to the UA to experiment with what works best for their users.

Can you elaborate on your concerns here? To be clear I was assuming that these requirements would apply when permission was requested, not when the capability is used. It seems reasonable to require a user gesture when requesting camera + PTZ. The current specification does not require this for getUserMedia() but I have a feeling this is a legacy concession.

Requirements for using an API should be normatively specified. Allowing UA's to vary these requirements hurts interop.

[riju]

User Gesture and Focused Frame (Page Visibility API) might hurt user experience and hamper some use cases, so we might prefer to keep this at the discretion of UA and add a NOTE to the spec. PTZ being a separate permission allows such hooks to the UA to experiment with what works best for their users.

Can you elaborate on your concerns here? To be clear I was assuming that these requirements would apply when permission was requested, not when the capability is used. It seems reasonable to require a user gesture when requesting camera + PTZ. The current specification does not require this for getUserMedia() but I have a feeling this is a legacy concession.

When permission is requested "focus" is totally fine, actually it's mandatory as defined in the parent spec. I was thinking during usage (PTZ permission = true, but tab in background) more like ShouldSuspendUpdates we did for sensors. Presently gUM allows capture to happen even when tab is not in focus. Some might understand that for PTZ we might change the behaviour and disable the feature(panning tilting) if tab is not in focus. I was re iterating about the use case you presented earlier is important -

when running a presentation the video conferencing application may be hidden but should still be able to use PTZ to focus on the presenter's face as they move about the room

[beaufortfrancois]

#231 and #232 will hopefully address this issue.

[beaufortfrancois]

I think we can close this issue now that #231 and #232 have been merged.
@riju WDYT?