Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add pan and tilt constraints #182

Merged
merged 1 commit into from
Mar 20, 2019
Merged

Conversation

dsanders11
Copy link
Contributor

As promised in issue #177.

Only real thing I can think of worth debating is whether to have the setting value be degrees, or arc seconds. They're trivial to convert between, and in UVC it is defined as arc seconds, as well as in more of the Windows APIs.

Degrees would be more intuitive for users, but since the underlying API will be returning the max/min/step in arc seconds, the MediaSettingsRange may look quite odd if you convert it to degrees, like from -38.23 to 38.23 with steps of 0.453. That could also lead to rounding problems in the conversion if the step in arc seconds is say 1000, that would be 0.277778 degrees, which is icky to deal with. As long as the camera "snaps" to the nearest valid step the rounding problems matter less, although it's never a great idea to have the value returned be different than the one you just set if the set succeeds. It's unintuitive and could lead to user code failing. Since either setting could be mechanical it may not apply immediately, and any end user code which keeps checking to see if it has changed to the correct value might fail or infinite loop if they set 21.234 degrees and that snaps to 21.2335 or something similar, since a simple == check would not be true.

@YellowDoge, I'm fairly confident these aren't available in the Android API. Would it make sense to use an in the implementation status to indicate that feature will never be available? The blank space would indicate that the feature is just not implemented yet, but could be in the future, versus means it will never be available.

@yell0wd0g
Copy link
Member

Re. the x, I don't think it's needed: other platform columns, e.g. Mac or Linux, also have unexposed controls and we're just leaving those empty.

@anssiko, what's your opinion in the tilt/pan in arc seconds vs. degrees question ?

@anssiko
Copy link
Member

anssiko commented Jun 15, 2017

@anssiko, what's your opinion in the tilt/pan in arc seconds vs. degrees question ?

I'll loop in @alexshalamov who's been looking at the spec more closely recently.

@alexshalamov
Copy link

@anssiko I would vote for degrees. Most of the motorized heads or servo motors express rotation values in degrees (or radians). Easier to understand and use for web developers. Also, windows uses degrees for pan, roll and tilt for user space APIs and same units on the 'driver' level (tilt, roll, pan).

If someone would like to use Image Capture API e.g., for astrophotography and HW supports fine step increments, 15 decimal places (double) should be enough to represent fractions of a degree / arc-second.

@dsanders11 BTW, why just Pan and Tilt without Roll? I have a camera where pan, tilt and roll can be controlled separately. Also, I think it would be better to specify order for applying 'rotation' constraints, so that vendors can implement interoperable API (pan-tilt-roll, zyx).

@dsanders11
Copy link
Contributor Author

Easier to understand and use for web developers.

I agree degrees are more commonly used and intuitive, but I still worry about it leading to iffy code because of the rounding problems.

For example, a UI that lets the user control pan and tilt would probably want to wait for them to take effect before letting the user make another adjustment (or at least debounce their input) to prevent the camera from lagging behind and feeling out of sync. A developer might choose to implement that as:

input.oninput = function(event) {
  lockUI = true;  // Stand in for some method of locking the UI
  var panValue = event.target.value;
  track.applyConstraints({pan: panValue});

  var waitingForUpdate = true;

  while (waitingForUpdate) {
    if (track.getSettings().pan == panValue) {
      lockUI = false;
      waitingForUpdate = false;
    }
  }
}

Obviously that's not great code, but it's in the vein of code someone might write if they have to wait for a setting to take effect.

Now, if they had an input set up that used the pan value and step increments, with arc-degrees this would always be an integer and there is no rounding problem. With degrees, adding the step to the current value might lead to a very slightly different number then gets returned by getSettings(), which would break simple equality checking.

I may be overblowing the "risk" here, but I've definitely seen float rounding problems bite people in the real world, where order of operation leads to slightly different results. It's stuck with me enough to make me weary of situations where it could occur.

@YellowDoge, would the promise returned by applyConstraints() only resolve once the settings have taken effect (if pan takes 500ms to apply) or does it resolve once the constraint has been set, without waiting for it to take effect?

@dsanders11 BTW, why just Pan and Tilt without Roll? I have a camera where pan, tilt and roll can be controlled separately.

Roll can easily be added, in my experience it isn't commonly available in consumer webcams so I didn't add it at this time. Most webcams in laptops and phones aren't going to have support for pan/tilt/roll at all, but I know Logitech brand webcams have support for pan/tilt, but not roll.

Also, I think it would be better to specify order for applying 'rotation' constraints, so that vendors can implement interoperable API (pan-tilt-roll, zyx).

You'll have to expand on that a bit, I don't quite follow. There's not currently any notion of ordering constraints in this spec, I think adding it would change the API a decent amount unless you had an idea in mind?

@yell0wd0g
Copy link
Member

@YellowDoge, would the promise returned by applyConstraints() only resolve once the settings have taken effect (if pan takes 500ms to apply) or does it resolve once the constraint has been set, without waiting for it to take effect?

It would not wait, the Promise gets resolved upon passing the new settings successfully to the OS, and the user has to monitor, if desired, the correspondent setting. A similar issue happens with zoom (@HTA raised it in #126), if it's implemented by a mechanical device.

@dsanders11
Copy link
Contributor Author

Yea, so my main concern is that rounding issues with the floats might cause simple equality to not suffice, which would be an unfortunate side effect of using degrees.

Case in point, say the step value is 1000 arc-seconds. When dealing with degrees, that's:

step = 1000/3600;  // 0.2777777777777778
a = 45 + step*5;  // 46.388888888888886
b = 45 + step + step + step + step + step;  // 46.38888888888889

if (a != b) {
    alert("Here be dragons");
}

I guarantee that's going to mess with people's code. Anyone not familiar with the behavior of floats wouldn't have that kind of situation occur to them, and so they wouldn't check.

@alexshalamov
Copy link

alexshalamov commented Jun 16, 2017

Roll can easily be added, in my experience it isn't commonly available in consumer webcams so I didn't add it at this time. Most webcams in laptops and phones aren't going to have support for pan/tilt/roll at all, but I know Logitech brand webcams have support for pan/tilt, but not roll.

I'm not sure whether W3C APIs should be tailored for particular brand or type of device.

You'll have to expand on that a bit, I don't quite follow.

When object is rotated in 3D space, order of rotations can affect end result http://quaternions.online/ , maybe I'm wrong 😃

I guarantee that's going to mess with people's code. Anyone not familiar with the behavior of floats wouldn't have that kind of situation occur to them, and so they wouldn't check.

I think it is not related to arcseconds or degrees, it is generic rounding problem. Spec doesn't say what happens when step has decimal places e.g., 1.89.

@YellowDoge Maybe specification could add note about rounding problem or introduce different MediaSettingsRange interfaces.

For example:
color temperature, iso, brightness, contrast, saturation, sharpness, zoom => unsigned long long
focus, exposure comp => unrestricted double (focus can be +Infinity)
tilt, pan, roll => long (+-180deg)

@dsanders11 my personal opinion is that pan, tilt and roll are niche features and not directly related to photo / video capture capabilities. Those are the features of camera accesoires: motorized heads, 3-axis gimbals, conference webcam or security IP cams chassis. Looks like UVC and Windows APIs have those features for some legacy reasons, I'm not sure whether new specs should replicate same pattern.

Also, Windows is a good example of what not to do. Half of the methods get / set arcseconds, while other half uses degrees.

@dsanders11
Copy link
Contributor Author

dsanders11 commented Jun 16, 2017

I'm not sure whether W3C APIs should be tailored for particular brand or type of device.

I'm not trying to tailor it to a particular brand or type of device, I was simply using a popular consumer brand as a way to gauge how common the roll control is in the wild.

I think it is not related to arcseconds or degrees, it is generic rounding problem.

It's a generic problem with floats, but the conversion from arcseconds to degrees is what causes it to be more prevalent than it would be for other controls. Most other controls are integers, or have a couple decimal places. The conversion from arcseconds to degrees (dividing by 3600) leads to repeating decimals like the one in my example, which are likely to run into the rounding issue.

So while not specific to arcseconds, by their nature you're more likely to run into repeating decimals due to the conversion.

I won't put up too much of a fight here, I just think a spec should do its best to protect users from sharp edges if possible.

@dsanders11 my personal opinion is that pan, tilt and roll are niche features and not directly related to photo / video capture capabilities. Those are the features of camera accesoires: motorized heads, 3-axis gimbals, conference webcam or security IP cams chassis. Looks like UVC and Windows APIs have those features for some legacy reasons, I'm not sure whether new specs should replicate same pattern.

I disagree with this assessment. They are not features limited to those listed camera accessories, they are features supported by many off the shelf webcams, of various brands, just not as common on cameras in tiny packages like phones or tablets. Roll is the more niche feature, which is why this PR is only for pan and tilt to begin with.

If you Google search "bmControls" pantilt you can see some of the various webcams that support pan and tilt thanks to people posting the output of lsusb online. Webcams I've identified that support pan and tilt (but not roll) in about 5 minutes of looking:

I believe there to be broad support for pan and tilt among laptop integrated webcams and external USB webcams, without any sort of accessory required.

@dsanders11 my personal opinion is that pan, tilt and roll are niche features and not directly related to photo / video capture capabilities.

Pan and tilt, especially when paired with zoom, have good applicability for use with face tracking, eye tracking, and other such use cases.

@alexshalamov
Copy link

alexshalamov commented Jun 16, 2017

@dsanders11 Thanks for info, those links basically prove that pan and tilt is a niche segment. I checked links for integrated cameras, looks like laptop manufacturers just soldered 3rd party components and took drivers that report pan/tilt capability. I doubt cameras installed in a window would be able to pan and tilt properly.

Some external webcams or security cameras do have such capability and I mentioned that before. Do you have rough estimate for market share of external cameras having pan/tilt capabilities and all mobile devices + laptop with integrated cameras?

As for degrees vs arcseconds, it would be nice to wait for reply from @YellowDoge, in my opinion, using double for all settings might be not an ideal solution.

@alexshalamov
Copy link

@dsanders11 Do you see any problems with using degrees vs arcseconds? For example, do external camera motors support less than 1 degree steps?

I forgot to mention that other w3c APIs use degrees to express rotation, DeviceOrientation, ScreenOrientation use degrees. OrientationSensor uses quaternion that can be converted to degrees, maybe it would be better to 'sync' units that express rotations across w3c apis.

@dsanders11
Copy link
Contributor Author

dsanders11 commented Jun 17, 2017

@dsanders11 Thanks for info, those links basically prove that pan and tilt is a niche segment. I checked links for integrated cameras, looks like laptop manufacturers just soldered 3rd party components and took drivers that report pan/tilt capability.

Again, I disagree with this assessment. I don't see any evidence that they're misreporting having a pan and tilt functionality which they don't have. Look through the information in those links again, no two of them are reporting having the exact same set of controls, they're all slightly different. This isn't a case of generic drivers misreporting features.

I doubt cameras installed in a window would be able to pan and tilt properly.

I think this is the root of our different understandings. I believe when you hear pan and tilt you're imagining mechanical pan and tilt, with motors, of the sort you mentioned in your description of accessories: "motorized heads, 3-axis gimbals, conference webcam or security IP cams chassis."

This is not the only way that cameras can provide the pan tilt functionality. Many of these cameras reporting pan and tilt have it as a feature of the CMOS chip used, like this one here. The datasheet for that CMOS chip (which is a tiny 4.6mm x 3.7mm) lists "Electronic Pan, Tilt, and Zoom" as a feature of the chip. Cameras using chips with electronic pan and tilt don't need anything mechanical inside their housing and simply have a lens over the CMOS chip. Their range of pan and tilt is much smaller, for example, 10 degrees in any direction, but it exists.

Here on the Logitech site they cover controls for their C920 camera. Of particular interest are these two bullets:

  • Pan/Tilt controls — Use the left, right, up, and down arrows to fine-tune the field of vision.
  • Follow my face — Click on the checkbox to allow the camera lens to automatically pan and tilt in order to follow your face.

This is what a teardown of that webcam looks like. There's nothing mechanical in there, the pan and tilt are being provided electronically.

Notice that the Logitech software has one of the features I was mentioning adding these controls to the spec would allow web developers to write - using the pan and tilt controls for face tracking.

I'll gladly concede the arc-degrees versus degrees in deference of other specs using degrees and if that's what others want. I will not, however, concede that pan and tilt are too niche to be in the spec.

EDIT: Doing some more research, it's a bit of a jumble on what this is called in literature, but it's usually electronic or digital pan/tilt/zoom. Sometimes called ePTZ or digital PTZ.

Notice in the links in my earlier comment which have the lsusb output for various cameras with pan tilt, they all also have zoom. That's because the the three always go together when it's digital PTZ, hence the term PTZ (pan, tilt, zoom).

@dsanders11 dsanders11 force-pushed the pan-tilt-constraints branch from c5d8aa7 to 28e9a21 Compare June 18, 2017 03:46
@alexshalamov
Copy link

Many of these cameras reporting pan and tilt have it as a feature of the CMOS chip used, like this one here.

That sensor has max 1 degree rotation. Lets say, with 50-60 deg FOV for webcam, that would not be much of a pan / tilt.

EDIT: Doing some more research, it's a bit of a jumble on what this is called in literature, but it's usually electronic or digital pan/tilt/zoom. Sometimes called ePTZ or digital PTZ.

If I remember correctly term digital PTZ is mostly used for security IP cams with ultra wide FOV. Stream from such cameras can be projected (perspective projection) to normal FOV and users can pan / tilt digitally.

@dsanders11 Do you think pan / tilt rotations need to be with less than 1 degree steps? Can long be used instead of double? If you think that roll can be added later, I think it is fine. In my opinion, roll can be important for leveling horizon when mechanical pan/tilt/roll is used, gimbals, drones, etc. BTW, did you check my question about pan tilt roll order? I still think it is important, that is why it is PanTilt or PanTiltRoll, standard Euler angles rotation order (ZYX, yaw-pitch-roll)

@dsanders11
Copy link
Contributor Author

dsanders11 commented Jun 19, 2017

That sensor has max 1 degree rotation. Lets say, with 50-60 deg FOV for webcam, that would not be much of a pan / tilt.

Webcams I've dealt with that were providing electronic pan tilt were in the range of +10 to -10 degrees.

@dsanders11 Do you think pan / tilt rotations need to be with less than 1 degree steps? Can long be used instead of double?

I think it needs support for less than 1 degree steps because there's nothing that says it can't be less than 1 degree. While in practice it may be uncommon, how would the API act if there is a camera with 0.5 degree steps? I suppose this spec could round step to the nearest degree, but that could again run into a camera that has something like 0.75 degree step which would mean a rounded degree value is often not a valid value.

The underlying APIs that are used for the camera control (UVC and Windows getRange_Pan) return the step value as arc degrees so I think we have to assume that it can be less than 1 degree.

BTW, did you check my question about pan tilt roll order? I still think it is important, that is why it is PanTilt or PanTiltRoll, standard Euler angles rotation order (ZYX, yaw-pitch-roll)

You'll have to provide an example of how to accommodate this in the spec, I don't see how enforcing an order can easily be added to the current API.

Wouldn't order only matter if using relative pan tilt roll? So far everything in the spec (correct me if I'm wrong) is using absolute values for constraints, not relative values. While order certainly matters when making relative transformations, does it matter when the values are absolute? Doing pan 5 degrees, roll 5 degrees, pan -5 degrees, roll -5 degrees would not get you back to default position when using relative values, doing pan to 5 degrees, roll to 5 degrees, pan to 0 degrees, roll to 0 degrees will always get you back to default position since they're absolute and the last value set was 0.

@yell0wd0g
Copy link
Member

To add my POV to this discussion, PanTilt is exposed by the UVC 1.5 Spec, Sec. "4.2.2.1.14 PanTilt (Absolute) Control" and "4.2.2.1.15 PanTilt (Relative) Control"; UVC is the underlying bus for all Desktop OSs, including Windows (whereas mobile platforms use MIPI). IMHO, that gives a large and stable enough deployed base to justify its inclusion in the Spec, also taking into account that if these are not supported by the concrete hardware, they won't appear in the MediaStreamTrackCapabilities. Also, the closer the Spec looks like the UVC the easier they'll match; UVC defines the units as Signed Number of size 4 bytes, so: int?.

Re. pantilt order, I guess @alexshalamov is aiming at including, or not, the path to get from a given "coordinate" to another as part of the Spec, e.g. from pan=0, tilt=0, to pan=1, tilt=1, the underlying hardware could manipulate pan first, then tilt, or viceversa, or both at the same time, and, e.g. at what speed. Looking at the UVC 1.5 Spec I don't see much definition for these issues, so we might simply need to explicit that the way "coordinates" in the pantilt space are reached is UA specific.

Also note that if roll is included, it changes the game since the application of pt and roll would not commutative and we'd be in the deep waters of the quaternions. Since roll is separated in the UVC spec, perhaps we should leave it for another PR, and then specify some order. Personally I haven't found any webcams with roll, so we might also need some evidence of implementations in the wild.

@dsanders11
Copy link
Contributor Author

Looking at the UVC 1.5 Spec I don't see much definition for these issues, so we might simply need to explicit that the way "coordinates" in the pantilt space are reached is UA specific.

Well, more than that, the UVC 1.5 Spec defines the PanTilt as a single control, you set both in one USB command. While you could use the relative version to enforce an order, setting 0 for the relative movement of the non-target control, the ImageCapture spec here is using absolute values, and there's no way in the UVC 1.5 Spec to set the absolute pan and tilt values separately. You have to define them at the same time, so the order in which they are applied is entirely undefined in the spec. The implementation could perform the pan and tilt in any order when it has been set both at the same time.

Personally I haven't found any webcams with roll, so we might also need some evidence of implementations in the wild.

You can find some if you search it particularly on the second page but it seems like they're less common than webcams with pan tilt.

@yell0wd0g
Copy link
Member

so the order in which they are applied is entirely undefined in the spec.

Precisely. Could you then add literature to this effect in the PR please? A <div class="note"> should be enough.
Also note that you have to rebase implementation-status.md.

@dsanders11 dsanders11 force-pushed the pan-tilt-constraints branch from 28e9a21 to cd6eacd Compare June 23, 2017 05:38
@dsanders11
Copy link
Contributor Author

@YellowDoge, note added and I did a force push to the branch because something had gotten wonky and it was showing commits not in upstream even though it didn't have any effect on the changes in the branch. Should be back to a single clean commit.

The language in the note may be a bit wordy but was trying to convey what has been discussed here. With absolute values I still see no reason to worry about the ordering, but hopefully the note is enough to get this PR back on track.

@yell0wd0g
Copy link
Member

Cool, thanks! @jan-ivar , @alexshalamov ptal.

@jan-ivar
Copy link
Member

@YellowDoge I think you meant @dsanders11. 😏

Reading through this, I don't have a strong opinion on rounding errors, but I would like to point out the several other ways that @dsanders11's example won't work:

  1. Busy-spinning on getSettings() waiting for it to change will always hang, because implementations generally queue tasks to update JS state. Use await.
  2. track.applyConstraints({pan}) is the same as track.applyConstraints({pan: {ideal: pan}}) and does not guarantee a value. Your page may be one of several tabs accessing the singular device simultaneously, and the browser chooses the final value. Use track.applyConstraints({pan: {exact: pan}}) to force a value or fail.
  3. Does applyConstraints() wait for the motor? I dunno. I've filed Does getSettings() reflect configured or actual setting? mediacapture-main#470.

@dsanders11
Copy link
Contributor Author

Reading through this, I don't have a strong opinion on rounding errors, but I would like to point out the several other ways that @dsanders11's example won't work

It also won't work because it simply sets a magical lockUI variable. It wasn't meant to be a working example (which is why I said it wasn't great code) but quick pseudo-code to highlight the potential problem lurking due to the float rounding.

It's not an example being inserted into the spec, I don't see how it is useful in moving this discussion forward to pick it apart on ancillary details.

The point being illustrated was that a seemingly intuitive use of simple equality may unexpectedly fail. You also used a simple equality check in the code you supplied in the issue you filed regarding applyConstraints(). You have to go out of your way to write code that would be immune to the float rounding problem. We have an opportunity to insulate end users from that problem, one many would not be aware of.

@jan-ivar
Copy link
Member

@dsanders11 Sure, my point was there were other assumptions here, not just that doubles match. The big issue is w3c/mediacapture-main#470 which questions whether getSettings() can be used like this at all. i.e. whether it returns an actual live hardware value, or just the last configured value (that the hardware is presumably moving towards asap). If it's the latter, then this is not a use-case showing that rounding is a problem. Not saying there aren't others. Just saying.

@dsanders11
Copy link
Contributor Author

@jan-ivar, I contend that even if getSettings() returns the last configured value immediately, this is still a use case highlighting the rounding problem.

As you pointed out, if you don't specify exact then the constraint may not actually be applied. How would one check if it had been applied? The promise returned by applyConstraints resolves to void if successful, so it doesn't provide you the information you need. A logical way to check would be to use getSettings() to see if your ideal settings had been set. For example, I may want: {pan: {ideal: pan}, tilt: {ideal: tilt}, width: {exact: 640}, height: {exact: 480}}.

If applying those constraints does not cause the promise to reject then I can be sure width and height were fine, but I don't know if pan or tilt took. So I check them with getSettings() and change the UI accordingly if they were not accepted, and we're back to the float rounding problem with simple equality. My check may say they weren't applied when in actuality they were, just my equality check failed due to the rounding.

Sure, this can be avoided by writing the code differently, such as applying each constraint one at a time with exact so that it is clear when a constraint fails. There's always ways around shortcomings, if you know the shortcomings.

If we go with degrees over arc-seconds then I'll likely add a note to the spec warning about these rounding problems and simple equality checks, although most casual developers are not going to be reading the spec itself.

@jan-ivar
Copy link
Member

FWIW, since the UA actually stores the constraints, could implementations fix these rounding errors?

E.g. If track.getConstraints().pan.ideal is 21.234, then track.getSettings().pan could return 21.234 instead of 21.2335.

Even if it changed back to 21.2335 once you removed the constraint, it wouldn't be that weird (who knows, maybe the UA decided to move it a tiny bit).

There seems to be some allowance for spec'ing this sort of thing in this mediacapture spec note.

@dsanders11
Copy link
Contributor Author

@jan-ivar, I don't think there's anything about being stateful in this spec. Your suggestion would require the implementation of the spec to keep a separate state, where as I believe current implementations just pass the values to the OS-level APIs and read them back accordingly. The problem with keeping state is that the state could be changed by something outside of the implementation, or simply another tab, which could lead to inconsistency.

The note you linked to, while interesting, is more about 1-to-1 mappings it would seem. The end also seems to suggest that getSettings() would still return the true value (although it doesn't say it has to).

I think that solution would slightly side-step the issue, but isn't an absolute fix, so I'm not sure it would be worth doing a half-measure if it adds extra complexity to implementations. The main issue is that order of operation when using step can cause rounding problems with floats, so while the mapping might save the programmer some of the time, a different path through the code may perform the math in a different order which would result in a different value, and we're back to square one.

@jan-ivar
Copy link
Member

I believe current implementations just pass the values to the OS-level APIs and read them back accordingly

@dsanders11 That's too expensive because track.getSettings() is synchronous. In Firefox at least, we don't access devices on the main thread. Instead we queue a task on the main thread when state changes, to store it there, where JS can read it.

Regardless, even if we accessed it directly, the only state we'd need to round values before returning them is track.getConstraints() which is already stored state according to the spec.

Adding complexity to implementations is warranted if it removes complexity for users.

@dsanders11
Copy link
Contributor Author

@jan-ivar, if the spec stated that pan and tilt were in degrees with at most 2 decimal points, that could remove some of the sharp edge from the user side. It's unlikely anyone would ever need less than 0.01 of a degree, and users could do comparisons with simple JavaScript thanks to type coercion:

track.getSettings().pan == pan.toFixed(2)

There's still the sharp edge where they need to know to use toFixed(2), but it's at least a simple and clean solution to the problem in JavaScript.

The only way to fully remove the sharp edge is to use arc-degrees, but it seems like there's not much support for that.

@yell0wd0g
Copy link
Member

This Pr seems to have gone quiet for a while now :-)
Any resolution, @dsanders11, @jan-ivar ?

@yell0wd0g
Copy link
Member

@dsanders11 , @jan-ivar, friendly ping :-)

Note that in Chrome we already have a private API where the Range is not specified, i.e. developers would use whatever the driver says.

@alvestrand
Copy link
Contributor

How should this API relate to the WebUSB API?
Given that the PTZ controls mentioned so far are all USB controls, perhaps the driver should be stated in terms of USB primitives?

(BTW, the spec HAS to admit to the fact that PTZ is not instant - at any given moment, the position of the camera is pointing in a definite direction, but that direction may or may not be the target position.)

@jan-ivar
Copy link
Member

@YellowDoge I summarized my position earlier. I don't have any new info.

perhaps the driver should be stated in terms of USB primitives?

@alvestrand What do you mean by "driver" and "stated" here?

BTW, the spec HAS to admit to the fact that PTZ is not instant

We fixed this is in https://github.com/w3c/mediacapture-main/pull/512/files to say settings are target values, not measured. Is that normatively sufficient? A note might help of course.

@riju
Copy link
Collaborator

riju commented Mar 8, 2019

Thanks @dsanders11 for the PR. As you rightly pointed out, both the UVC spec (section 4.2.2.1.14) as well as Windows getRange_Pan API use arc seconds as the unit for pan and tilt settings.

@jan-ivar summarized the options for the spec succinctly and based on the discussion in this PR, related issue, and data provided it seems the option “1. Use arc seconds” is the option everyone can live with. As a further data point, I also reviewed some native applications like gucview, and observed they also use arc seconds as the unit.

Given pan and tilt are slightly niche features, my expectation is web developers who’ll make use of these features in their web apps are comfortable working with arc seconds.

I implemented this feature in Chromium (video - note: big file) according to this spec PR to try it out, and noticed no issues. I would like to hear if people have any concerns in merging this PR.

CC : @anssiko

chromium-wpt-export-bot pushed a commit to web-platform-tests/wpt that referenced this pull request Mar 8, 2019
Pan and Tilt Constraints are part of the UVC spec.
For Linux/CrOS, we can use V4L2 controls like
(V4L2_CID_PAN_ABSOLUTE and V4L2_CID_TILT_ABSOLUTE).

Spec: w3c/mediacapture-image#182

Test Page: https://riju.github.io/WebCamera/samples/panTilt/

Bug:934063
Change-Id: I552c4c8be717c3b67c4d91f826a1f16850430fa4
chromium-wpt-export-bot pushed a commit to web-platform-tests/wpt that referenced this pull request Mar 11, 2019
Pan and Tilt Constraints are part of the UVC spec.
For Linux/CrOS, we can use V4L2 controls like
(V4L2_CID_PAN_ABSOLUTE and V4L2_CID_TILT_ABSOLUTE).

Spec: w3c/mediacapture-image#182

Test Page: https://riju.github.io/WebCamera/samples/panTilt/

Bug:934063
Change-Id: I552c4c8be717c3b67c4d91f826a1f16850430fa4
chromium-wpt-export-bot pushed a commit to web-platform-tests/wpt that referenced this pull request Mar 11, 2019
Pan and Tilt Constraints are part of the UVC spec.
For Linux/CrOS, we can use V4L2 controls like
(V4L2_CID_PAN_ABSOLUTE and V4L2_CID_TILT_ABSOLUTE).

Spec: w3c/mediacapture-image#182

Test Page: https://riju.github.io/WebCamera/samples/panTilt/

Bug:934063
Change-Id: I552c4c8be717c3b67c4d91f826a1f16850430fa4
@yell0wd0g
Copy link
Member

@dsanders11 are we then OK with option “1. Use arc seconds” where everyone seems happy?

@dsanders11
Copy link
Contributor Author

@riju, @YellowDoge, apologies for the late response. Yes, arc-seconds is great by me.

chromium-wpt-export-bot pushed a commit to web-platform-tests/wpt that referenced this pull request Mar 20, 2019
Pan and Tilt Constraints are part of the UVC spec.
For Linux/CrOS, we can use V4L2 controls like
(V4L2_CID_PAN_ABSOLUTE and V4L2_CID_TILT_ABSOLUTE).

Spec: w3c/mediacapture-image#182

Test Page: https://riju.github.io/WebCamera/samples/panTilt/

Bug:934063
Change-Id: I552c4c8be717c3b67c4d91f826a1f16850430fa4
@yell0wd0g yell0wd0g merged commit e5533df into w3c:master Mar 20, 2019
chromium-wpt-export-bot pushed a commit to web-platform-tests/wpt that referenced this pull request Mar 22, 2019
Pan and Tilt Constraints are part of the UVC spec.
For Linux/CrOS, we can use V4L2 controls like
(V4L2_CID_PAN_ABSOLUTE and V4L2_CID_TILT_ABSOLUTE).

Spec: w3c/mediacapture-image#182

Test Page: https://riju.github.io/WebCamera/samples/panTilt/

Bug:934063
Change-Id: I552c4c8be717c3b67c4d91f826a1f16850430fa4
chromium-wpt-export-bot pushed a commit to web-platform-tests/wpt that referenced this pull request Mar 22, 2019
Pan and Tilt Constraints are part of the UVC spec.
For Linux/CrOS, we can use V4L2 controls like
(V4L2_CID_PAN_ABSOLUTE and V4L2_CID_TILT_ABSOLUTE).

Spec: w3c/mediacapture-image#182

Test Page: https://riju.github.io/WebCamera/samples/panTilt/

Bug:934063
Change-Id: I552c4c8be717c3b67c4d91f826a1f16850430fa4
chromium-wpt-export-bot pushed a commit to web-platform-tests/wpt that referenced this pull request Mar 22, 2019
Pan and Tilt Constraints are part of the UVC spec.
For Linux/CrOS, we can use V4L2 controls like
(V4L2_CID_PAN_ABSOLUTE and V4L2_CID_TILT_ABSOLUTE).

Spec: w3c/mediacapture-image#182

Test Page: https://riju.github.io/WebCamera/samples/panTilt/

Intent to Implement and Ship discussions:
https://groups.google.com/a/chromium.org/d/msg/blink-dev/j-Q08QgBipM/F3a5sau1BwAJ

Bug:934063
Change-Id: I552c4c8be717c3b67c4d91f826a1f16850430fa4
chromium-wpt-export-bot pushed a commit to web-platform-tests/wpt that referenced this pull request Jul 26, 2019
Pan and Tilt Constraints are part of the UVC spec.
For Linux/CrOS, we can use V4L2 controls like
(V4L2_CID_PAN_ABSOLUTE and V4L2_CID_TILT_ABSOLUTE).

Spec: w3c/mediacapture-image#182

Test Page: https://riju.github.io/WebCamera/samples/panTilt/

Putting Pan/Tilt feature behind a flag:
chrome --enable-blink-features MediaCapturePanTilt

Intent to Implement and Ship discussions:
https://groups.google.com/a/chromium.org/d/msg/blink-dev/j-Q08QgBipM/F3a5sau1BwAJ

Bug:934063
Change-Id: I552c4c8be717c3b67c4d91f826a1f16850430fa4
chromium-wpt-export-bot pushed a commit to web-platform-tests/wpt that referenced this pull request Jul 26, 2019
Pan and Tilt Constraints are part of the UVC spec.
For Linux/CrOS, we can use V4L2 controls like
(V4L2_CID_PAN_ABSOLUTE and V4L2_CID_TILT_ABSOLUTE).

Spec: w3c/mediacapture-image#182

Test Page: https://riju.github.io/WebCamera/samples/panTilt/

Putting Pan/Tilt feature behind a flag:
chrome --enable-blink-features=MediaCapturePanTilt

Intent to Implement and Ship discussions:
https://groups.google.com/a/chromium.org/d/msg/blink-dev/j-Q08QgBipM/F3a5sau1BwAJ

Bug:934063
Change-Id: I552c4c8be717c3b67c4d91f826a1f16850430fa4
aarongable pushed a commit to chromium/chromium that referenced this pull request Jul 26, 2019
Pan and Tilt Constraints are part of the UVC spec.
For Linux/CrOS, we can use V4L2 controls like
(V4L2_CID_PAN_ABSOLUTE and V4L2_CID_TILT_ABSOLUTE).

Spec: w3c/mediacapture-image#182

Test Page: https://riju.github.io/WebCamera/samples/panTilt/

Putting Pan/Tilt feature behind a flag:
chrome --enable-blink-features=MediaCapturePanTilt

Intent to Implement and Ship discussions:
https://groups.google.com/a/chromium.org/d/msg/blink-dev/j-Q08QgBipM/F3a5sau1BwAJ


Bug: 934063
Change-Id: I552c4c8be717c3b67c4d91f826a1f16850430fa4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1508519
Reviewed-by: Guido Urdaneta <[email protected]>
Reviewed-by: Kinuko Yasuda <[email protected]>
Reviewed-by: Miguel Casas <[email protected]>
Reviewed-by: Reilly Grant <[email protected]>
Commit-Queue: Rijubrata Bhaumik <[email protected]>
Cr-Commit-Position: refs/heads/master@{#681332}
chromium-wpt-export-bot pushed a commit to web-platform-tests/wpt that referenced this pull request Jul 26, 2019
Pan and Tilt Constraints are part of the UVC spec.
For Linux/CrOS, we can use V4L2 controls like
(V4L2_CID_PAN_ABSOLUTE and V4L2_CID_TILT_ABSOLUTE).

Spec: w3c/mediacapture-image#182

Test Page: https://riju.github.io/WebCamera/samples/panTilt/

Putting Pan/Tilt feature behind a flag:
chrome --enable-blink-features=MediaCapturePanTilt

Intent to Implement and Ship discussions:
https://groups.google.com/a/chromium.org/d/msg/blink-dev/j-Q08QgBipM/F3a5sau1BwAJ

Bug: 934063
Change-Id: I552c4c8be717c3b67c4d91f826a1f16850430fa4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1508519
Reviewed-by: Guido Urdaneta <[email protected]>
Reviewed-by: Kinuko Yasuda <[email protected]>
Reviewed-by: Miguel Casas <[email protected]>
Reviewed-by: Reilly Grant <[email protected]>
Commit-Queue: Rijubrata Bhaumik <[email protected]>
Cr-Commit-Position: refs/heads/master@{#681332}
chromium-wpt-export-bot pushed a commit to web-platform-tests/wpt that referenced this pull request Jul 26, 2019
Pan and Tilt Constraints are part of the UVC spec.
For Linux/CrOS, we can use V4L2 controls like
(V4L2_CID_PAN_ABSOLUTE and V4L2_CID_TILT_ABSOLUTE).

Spec: w3c/mediacapture-image#182

Test Page: https://riju.github.io/WebCamera/samples/panTilt/

Putting Pan/Tilt feature behind a flag:
chrome --enable-blink-features=MediaCapturePanTilt

Intent to Implement and Ship discussions:
https://groups.google.com/a/chromium.org/d/msg/blink-dev/j-Q08QgBipM/F3a5sau1BwAJ

Bug: 934063
Change-Id: I552c4c8be717c3b67c4d91f826a1f16850430fa4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1508519
Reviewed-by: Guido Urdaneta <[email protected]>
Reviewed-by: Kinuko Yasuda <[email protected]>
Reviewed-by: Miguel Casas <[email protected]>
Reviewed-by: Reilly Grant <[email protected]>
Commit-Queue: Rijubrata Bhaumik <[email protected]>
Cr-Commit-Position: refs/heads/master@{#681332}
moz-v2v-gh pushed a commit to mozilla/gecko-dev that referenced this pull request Aug 5, 2019
…nd wire in Linux/CrOS., a=testonly

Automatic update from web-platform-tests
[ImageCapture] Add pan/tilt constraint and wire in Linux/CrOS.

Pan and Tilt Constraints are part of the UVC spec.
For Linux/CrOS, we can use V4L2 controls like
(V4L2_CID_PAN_ABSOLUTE and V4L2_CID_TILT_ABSOLUTE).

Spec: w3c/mediacapture-image#182

Test Page: https://riju.github.io/WebCamera/samples/panTilt/

Putting Pan/Tilt feature behind a flag:
chrome --enable-blink-features=MediaCapturePanTilt

Intent to Implement and Ship discussions:
https://groups.google.com/a/chromium.org/d/msg/blink-dev/j-Q08QgBipM/F3a5sau1BwAJ

Bug: 934063
Change-Id: I552c4c8be717c3b67c4d91f826a1f16850430fa4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1508519
Reviewed-by: Guido Urdaneta <[email protected]>
Reviewed-by: Kinuko Yasuda <[email protected]>
Reviewed-by: Miguel Casas <[email protected]>
Reviewed-by: Reilly Grant <[email protected]>
Commit-Queue: Rijubrata Bhaumik <[email protected]>
Cr-Commit-Position: refs/heads/master@{#681332}

--

wpt-commits: c019cf8328f1b985a37936ff2d617053f09aa3c1
wpt-pr: 15741
xeonchen pushed a commit to xeonchen/gecko that referenced this pull request Aug 5, 2019
…nd wire in Linux/CrOS., a=testonly

Automatic update from web-platform-tests
[ImageCapture] Add pan/tilt constraint and wire in Linux/CrOS.

Pan and Tilt Constraints are part of the UVC spec.
For Linux/CrOS, we can use V4L2 controls like
(V4L2_CID_PAN_ABSOLUTE and V4L2_CID_TILT_ABSOLUTE).

Spec: w3c/mediacapture-image#182

Test Page: https://riju.github.io/WebCamera/samples/panTilt/

Putting Pan/Tilt feature behind a flag:
chrome --enable-blink-features=MediaCapturePanTilt

Intent to Implement and Ship discussions:
https://groups.google.com/a/chromium.org/d/msg/blink-dev/j-Q08QgBipM/F3a5sau1BwAJ

Bug: 934063
Change-Id: I552c4c8be717c3b67c4d91f826a1f16850430fa4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1508519
Reviewed-by: Guido Urdaneta <[email protected]>
Reviewed-by: Kinuko Yasuda <[email protected]>
Reviewed-by: Miguel Casas <[email protected]>
Reviewed-by: Reilly Grant <[email protected]>
Commit-Queue: Rijubrata Bhaumik <[email protected]>
Cr-Commit-Position: refs/heads/master@{#681332}

--

wpt-commits: c019cf8328f1b985a37936ff2d617053f09aa3c1
wpt-pr: 15741
natechapin pushed a commit to natechapin/wpt that referenced this pull request Aug 23, 2019
Pan and Tilt Constraints are part of the UVC spec.
For Linux/CrOS, we can use V4L2 controls like
(V4L2_CID_PAN_ABSOLUTE and V4L2_CID_TILT_ABSOLUTE).

Spec: w3c/mediacapture-image#182

Test Page: https://riju.github.io/WebCamera/samples/panTilt/

Putting Pan/Tilt feature behind a flag:
chrome --enable-blink-features=MediaCapturePanTilt

Intent to Implement and Ship discussions:
https://groups.google.com/a/chromium.org/d/msg/blink-dev/j-Q08QgBipM/F3a5sau1BwAJ

Bug: 934063
Change-Id: I552c4c8be717c3b67c4d91f826a1f16850430fa4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1508519
Reviewed-by: Guido Urdaneta <[email protected]>
Reviewed-by: Kinuko Yasuda <[email protected]>
Reviewed-by: Miguel Casas <[email protected]>
Reviewed-by: Reilly Grant <[email protected]>
Commit-Queue: Rijubrata Bhaumik <[email protected]>
Cr-Commit-Position: refs/heads/master@{#681332}
gecko-dev-updater pushed a commit to marco-c/gecko-dev-comments-removed that referenced this pull request Oct 4, 2019
…nd wire in Linux/CrOS., a=testonly

Automatic update from web-platform-tests
[ImageCapture] Add pan/tilt constraint and wire in Linux/CrOS.

Pan and Tilt Constraints are part of the UVC spec.
For Linux/CrOS, we can use V4L2 controls like
(V4L2_CID_PAN_ABSOLUTE and V4L2_CID_TILT_ABSOLUTE).

Spec: w3c/mediacapture-image#182

Test Page: https://riju.github.io/WebCamera/samples/panTilt/

Putting Pan/Tilt feature behind a flag:
chrome --enable-blink-features=MediaCapturePanTilt

Intent to Implement and Ship discussions:
https://groups.google.com/a/chromium.org/d/msg/blink-dev/j-Q08QgBipM/F3a5sau1BwAJ

Bug: 934063
Change-Id: I552c4c8be717c3b67c4d91f826a1f16850430fa4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1508519
Reviewed-by: Guido Urdaneta <guidouchromium.org>
Reviewed-by: Kinuko Yasuda <kinukochromium.org>
Reviewed-by: Miguel Casas <mcasaschromium.org>
Reviewed-by: Reilly Grant <reillygchromium.org>
Commit-Queue: Rijubrata Bhaumik <rijubrata.bhaumikintel.com>
Cr-Commit-Position: refs/heads/master{#681332}

--

wpt-commits: c019cf8328f1b985a37936ff2d617053f09aa3c1
wpt-pr: 15741

UltraBlame original commit: 05861c142528c472fe9cec6ea2f5828f75071612
gecko-dev-updater pushed a commit to marco-c/gecko-dev-wordified that referenced this pull request Oct 4, 2019
…nd wire in Linux/CrOS., a=testonly

Automatic update from web-platform-tests
[ImageCapture] Add pan/tilt constraint and wire in Linux/CrOS.

Pan and Tilt Constraints are part of the UVC spec.
For Linux/CrOS, we can use V4L2 controls like
(V4L2_CID_PAN_ABSOLUTE and V4L2_CID_TILT_ABSOLUTE).

Spec: w3c/mediacapture-image#182

Test Page: https://riju.github.io/WebCamera/samples/panTilt/

Putting Pan/Tilt feature behind a flag:
chrome --enable-blink-features=MediaCapturePanTilt

Intent to Implement and Ship discussions:
https://groups.google.com/a/chromium.org/d/msg/blink-dev/j-Q08QgBipM/F3a5sau1BwAJ

Bug: 934063
Change-Id: I552c4c8be717c3b67c4d91f826a1f16850430fa4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1508519
Reviewed-by: Guido Urdaneta <guidouchromium.org>
Reviewed-by: Kinuko Yasuda <kinukochromium.org>
Reviewed-by: Miguel Casas <mcasaschromium.org>
Reviewed-by: Reilly Grant <reillygchromium.org>
Commit-Queue: Rijubrata Bhaumik <rijubrata.bhaumikintel.com>
Cr-Commit-Position: refs/heads/master{#681332}

--

wpt-commits: c019cf8328f1b985a37936ff2d617053f09aa3c1
wpt-pr: 15741

UltraBlame original commit: 05861c142528c472fe9cec6ea2f5828f75071612
gecko-dev-updater pushed a commit to marco-c/gecko-dev-wordified-and-comments-removed that referenced this pull request Oct 4, 2019
…nd wire in Linux/CrOS., a=testonly

Automatic update from web-platform-tests
[ImageCapture] Add pan/tilt constraint and wire in Linux/CrOS.

Pan and Tilt Constraints are part of the UVC spec.
For Linux/CrOS, we can use V4L2 controls like
(V4L2_CID_PAN_ABSOLUTE and V4L2_CID_TILT_ABSOLUTE).

Spec: w3c/mediacapture-image#182

Test Page: https://riju.github.io/WebCamera/samples/panTilt/

Putting Pan/Tilt feature behind a flag:
chrome --enable-blink-features=MediaCapturePanTilt

Intent to Implement and Ship discussions:
https://groups.google.com/a/chromium.org/d/msg/blink-dev/j-Q08QgBipM/F3a5sau1BwAJ

Bug: 934063
Change-Id: I552c4c8be717c3b67c4d91f826a1f16850430fa4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1508519
Reviewed-by: Guido Urdaneta <guidouchromium.org>
Reviewed-by: Kinuko Yasuda <kinukochromium.org>
Reviewed-by: Miguel Casas <mcasaschromium.org>
Reviewed-by: Reilly Grant <reillygchromium.org>
Commit-Queue: Rijubrata Bhaumik <rijubrata.bhaumikintel.com>
Cr-Commit-Position: refs/heads/master{#681332}

--

wpt-commits: c019cf8328f1b985a37936ff2d617053f09aa3c1
wpt-pr: 15741

UltraBlame original commit: 05861c142528c472fe9cec6ea2f5828f75071612
blueboxd pushed a commit to blueboxd/chromium-legacy that referenced this pull request May 6, 2020
Pan and Tilt is already present on Linux and CrOS devices.
This CL adds the platform support for the DirectShow backend
for Windows platform. We need to pass the flag
--disable-features=MediaFoundationVideoCapture to chrome.

Spec: w3c/mediacapture-image#182

Test Page: https://riju.github.io/WebCamera/samples/panTilt/

Pan/Tilt feature is behind a flag:
chrome --enable-blink-features=MediaCapturePanTilt

Intent to Implement and Ship discussions:
https://groups.google.com/a/chromium.org/d/msg/blink-dev/j-Q08QgBipM/F3a5sau1BwAJ

Bug: 934063
Change-Id: I8981692647e30b79fb20c8359c8883c194c055d7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2014916
Commit-Queue: Eero Häkkinen <[email protected]>
Reviewed-by: Guido Urdaneta <[email protected]>
Cr-Commit-Position: refs/heads/master@{#765992}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants