Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clarifying docs on XRInputSource.handedness property #124

Closed
m-alkalbani opened this issue May 22, 2024 · 10 comments · Fixed by #125
Closed

Clarifying docs on XRInputSource.handedness property #124

m-alkalbani opened this issue May 22, 2024 · 10 comments · Fixed by #125

Comments

@m-alkalbani
Copy link
Contributor

The Hand Input Explainer does not explicitly mention the handedness attribute, and this causes the code samples provided to not work as expected (you end up with one visible hand when drawing the skeleton).

While it's clear in the Device API spec that handedness is a property of XRInputSource, I feel it should be mentioned in the explainer, perhaps in the Accessing this API or Displaying hand models using this API sections? This would make it clear for developers that a hand doesn't determine handedness, and would also provide the relevant info on how to access the handedness attribute if required.

I am happy to create a PR to do the following:

  1. Mention handedness in the relevant parts of the explainer

  2. Mention handedness in a note in Section 3.1 of the Hand Input Module spec

Any thoughts? I am away early next week but I would be able to work on this later in the week.

@cabanier
Copy link
Member

I think it's fine to add some clarification to the explainer.
Can you explain why only 1 hand is visible?

@Maksims
Copy link

Maksims commented May 23, 2024

Another quick note: Meta's XR emulator does provide handedness, but not at the moment of input source creation. While the real device (e.g. Quest 3), provides a specific handedness at the moment the XRInputSource became available. Which makes it slightly better to work with.
Specs do not mention this, and it is assumed that handedness can change for an input source at any point.

@AdaRoseCannon
Copy link
Member

One of the editors correct me if so am wrong but I’m pretty sure handedness shouldn’t change on the fly and that if an input source needed to change handedness it would be removed and added with the new handedness.

@Maksims
Copy link

Maksims commented May 23, 2024

One of the editors correct me if so am wrong but I’m pretty sure handedness shouldn’t change on the fly and that if an input source needed to change handedness it would be removed and added with the new handedness.

That is the behaviour on Quest 3 as I believe.
But not in the emulator 😕

@cabanier
Copy link
Member

One of the editors correct me if so am wrong but I’m pretty sure handedness shouldn’t change on the fly and that if an input source needed to change handedness it would be removed and added with the new handedness.

Yes, the chromium code generates new inputsources each time anything in the inputsource changes.

@m-alkalbani
Copy link
Contributor Author

I think it's fine to add some clarification to the explainer. Can you explain why only 1 hand is visible?

I attempted to draw both hands using a custom component in A-Frame by following the structure in the explainer, but this results in rendering the skeleton on one hand:

for (const inputSource of inputSources) {
    if (inputSource.hand) {
        const hand = inputSource.hand;
        for (const finger of orderedJoints) {
            for (const jointName of finger) {
                const joint = hand.get(jointName);
                if (joint) {
                    const jointPose = this.frame.getJointPose(joint, this.referenceSpace);
                    const position = jointPose.transform.position;
                    if (!this.spheres[jointName]) {
                        this.spheres[jointName] = this.drawSphere(jointPose.radius, position);
                    } else {
                        this.spheres[jointName].object3D.position.set(position.x, position.y, position.z);
                    }
                }
            }
        }
    }
}

Both hands are detected, but the problem is that the spheres are drawn (and overwritten) on the last detected hand when looping through inputSources. I am able to work around this if I refer to handedness, but this took some effort to figure out as it is not mentioned in the relevant docs (explainer/spec/other as above). It's currently not obvious for developers how to use handedness if they need it.

@m-alkalbani
Copy link
Contributor Author

One of the editors correct me if so am wrong but I’m pretty sure handedness shouldn’t change on the fly and that if an input source needed to change handedness it would be removed and added with the new handedness.

Just to clarify, this is not about changing handedness during runtime, but mainly about clarifying in the docs that handedness is a property of XRInputSource. This would make it easier for developers to find and use handedness if needed (for example recognising a bimanual gesture where the left fist is closed and the right one is open).

@Maksims
Copy link

Maksims commented May 29, 2024

I think it's fine to add some clarification to the explainer. Can you explain why only 1 hand is visible?

I attempted to draw both hands using a custom component in A-Frame by following the structure in the explainer, but this results in rendering the skeleton on one hand:

for (const inputSource of inputSources) {
    if (inputSource.hand) {
        const hand = inputSource.hand;
        for (const finger of orderedJoints) {
            for (const jointName of finger) {
                const joint = hand.get(jointName);
                if (joint) {
                    const jointPose = this.frame.getJointPose(joint, this.referenceSpace);
                    const position = jointPose.transform.position;
                    if (!this.spheres[jointName]) {
                        this.spheres[jointName] = this.drawSphere(jointPose.radius, position);
                    } else {
                        this.spheres[jointName].object3D.position.set(position.x, position.y, position.z);
                    }
                }
            }
        }
    }
}

Both hands are detected, but the problem is that the spheres are drawn (and overwritten) on the last detected hand when looping through inputSources. I am able to work around this if I refer to handedness, but this took some effort to figure out as it is not mentioned in the relevant docs (explainer/spec/other as above). It's currently not obvious for developers how to use handedness if they need it.

You index spheres by joint name: this.spheres[jointName], so you get an override because left hand and right hand have joints that share same names. This is your application logic issue, handedness on input sources work as intended.

@m-alkalbani
Copy link
Contributor Author

handedness on input sources work as intended.

I agree handedness on input sources works as intended. The problem is that handedness is not mentioned in the explainer, nor the hand input spec. For new developers using the explainer as a guide, it would be helpful for us to point out that handedness exists, and is present on XRInputSource.

Would anyone object to me making a PR for your review?

@cabanier
Copy link
Member

Would anyone object to me making a PR for your review?

No, PRs are always welcome

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants