Add a mouse based cursor / raycaster and applies to the link traversal example #2861

dmarcos · 2017-07-11T06:30:38Z

It adds a mouse cursor mode to the cursor component similar to https://github.com/mayognaise/aframe-mouse-cursor-component
It modifies the raycaster component to accommodate the mouse cursor use case
It makes the link traversal example usable on desktop
It adds link traversal example to the index.html

ngokevin · 2017-07-13T19:38:36Z

docs/components/cursor.md

@@ -80,6 +80,7 @@ AFRAME.registerComponent('cursor-listener', {
 | downEvents  | Array of additional events on the entity to *listen* to for triggering `mousedown` (e.g., `triggerdown` for vive-controls).  | []                               |
 | fuse        | Whether cursor is fuse-based.                                                                                              | false on desktop, true on mobile |
 | fuseTimeout | How long to wait (in milliseconds) before triggering a fuse-based click event.                                             | 1500                             |
+| mode     | Camera (gaze) or mouse controlled. | gaze


I'd just make it a boolean based for mouse. mouseMode: true|false. The cursor can also be used for controllers as well, not just for gaze.

but you also might want mouse and not gaze. Also the raycaster operates differently for each mode: raycaster from the entity or from the mouse cursor on the screen. If you want gaze + mouse you can add two cursors to the same or different entities. In the link traversal examples the cursor is added to the a-scene

Don't know if my comment was understood. mode: gaze doesn't really do anything (and the non-mouse cursor is not just for gaze). What we want to represent is either the cursor has mouse capabilities or it doesn't, represented as a boolean mouseMode: true or mouseMode: false. This would be consistent to how it's represented in the code as well. Also better not to use strings when possible.

The problem is that gaze and mouse are mutually exclusive because the raycaster configuration is different (hence the mode property). A cursor as it is, cannot operate in both gaze and mouse modes. A mouseMode flag is confusing because it conveys an additional functionality on top of the normal behavior that can be toggled. We could make the raycaster a multiple component and the cursor component could add two of them: One for gaze and the other for mouse. This might introduce confusion since configuring the raycaster by the user would not be as simple as:

el.setAttribute('raycaster', 'objects', '.clickable')

Two different raycasters would have to be configured instead:

el.setAttribute('raycaster__gaze', 'objects', '.clickable'); el.setAttribute('raycaster__mouse', 'objects', '.clickable');

An alternative would be to add an objects property to the cursor component so the user does not configure the raycaster anymore

A boolean mouseMode property properly represents the mutual exclusiveness, that the entire component is flipped to a mouse mode or it's not. "Modes" usually mean an entire configuration, not "additional functionality".

"Gaze mode" does not make sense as the cursor can be used with controllers as well, and in the component code, "gaze mode" doesn't trigger any logic. It's only when setting to mouse mode does any different code run.

How I could see this component being used (although I'm not too amped on including a mouse mode in the first place):

<a-scene> <a-mixin id="raycaster" raycaster="objects: .clickable"></a-mixin> <a-entity mixin="raycaster" cursor="mouseMode: true"></a-entity> <a-camera> <a-entity mixin="raycaster" cursor></a-entity> </a-camera> <a-entity mixin="raycaster" laser-controls</a-entity> </a-scene>

I think the mode sort of makes sense, but I don't really like having to provide strings, that gaze isn't the only other way to use the cursor, and that gaze is not represented in the code as doing anything.

Maybe we have different expectations on what a flag means. When seeing mouseMode: true enabled many people will interpret it as I can use the mouse cursor as well to select things in addition to the normal behavior. A mode word indicates... modality 😄 We can replace the mode names mouse \ gaze with screen \ entity to indicate where the cursor performs the raycasting from. A cleaner solution is probably adding two raycasters, an objects property and a mouseMode flag (false by default) so the modes are not mutually exclusive. The API would be then:

<a-entity cursor="mouseMode: true; objects: .clickable">

The cursor would operate in both modes simultaneously and use two raycasters: raycaster__mouse and raycaster__entity

Thinking about it, yeah that makes sense as long as it's not named gaze. I need to think about operating in both simultaneously, it would make it less work for developers and possibly more performant (one cursor vs. two cursors), although it's more magic and complex.

dmarcos · 2017-07-17T23:11:01Z

I replaced mode with rayOrigin with two options entity / mouse

ngokevin · 2017-07-17T23:37:04Z

So not going to try the idea of having a cursor work with both simultaneously?

dmarcos · 2017-07-17T23:44:00Z

It needs more thought and probably not a priority right now.

ngokevin

r+wc

ngokevin · 2017-07-18T20:38:03Z

src/components/raycaster.js

@@ -27,7 +27,8 @@ module.exports.Component = registerComponent('raycaster', {
    objects: {default: ''},
    origin: {type: 'vec3'},
    recursive: {default: true},
-    showLine: {default: false}
+    showLine: {default: false},
+    worldCoordinates: {default: false}


I'd name this isWorldSpace (or areWorldCoordinates) to not sound like it takes coordinates.

ngokevin · 2017-07-18T20:38:24Z

src/components/cursor.js

+    var origin = new THREE.Vector3();
+    var direction = new THREE.Vector3();
+    return function (evt) {
+      var camera = this.el.sceneEl.camera;


Need a test for this block

ngokevin · 2017-07-18T20:38:34Z

src/components/cursor.js

+      mouse.y = -(evt.clientY / window.innerHeight) * 2 + 1;
+      origin.setFromMatrixPosition(camera.matrixWorld);
+      direction.set(mouse.x, mouse.y, 0.5).unproject(camera).sub(origin).normalize();
+      this.el.setAttribute('raycaster', {origin: origin, direction: direction});


Reuse an object for this

ngokevin · 2017-07-18T20:41:47Z

src/components/cursor.js

+  },
+
+  update: function () {
+    this.updateMouseEventListeners();


I'd only call this if rayOrigin changes

ngokevin · 2017-07-18T20:42:27Z

docs/components/raycaster.md

@@ -58,6 +58,7 @@ AFRAME.registerComponent('collider-check', {
 | origin    | Vector3 coordinate of where the ray should originate from relative to the entity's origin.                                                            | 0, 0, 0       |
 | recursive | Checks all children of objects if set. Else only checks intersections with root objects.                                                              | true          |
 | showLine  | Whether or not to display the raycaster visually with the [line component][line].                                                                                         | false          |
+| worldCoordinates  | Determine if origin and direction are given in local or world coordinates.                                                                                         | false          |


Whether the raycaster origin and direction properties are specified in world space.

ngokevin · 2017-07-18T20:43:02Z

docs/components/cursor.md

@@ -80,6 +80,7 @@ AFRAME.registerComponent('cursor-listener', {
 | downEvents  | Array of additional events on the entity to *listen* to for triggering `mousedown` (e.g., `triggerdown` for vive-controls).  | []                               |
 | fuse        | Whether cursor is fuse-based.                                                                                              | false on desktop, true on mobile |
 | fuseTimeout | How long to wait (in milliseconds) before triggering a fuse-based click event.                                             | 1500                             |
+| rayOrigin     | if the intersection ray is cast from the entity or the mouse | entity


Where the intersection ray is cast from (i.e., entityormouse)

ngokevin · 2017-07-18T20:45:47Z

tests/components/cursor.test.js

@@ -76,6 +76,16 @@ suite('cursor', function () {
        done();
      });
    });
+
+    suite('update', function () {
+      test('attach mousemove event listener on mouse mode', function () {


I'd also test removing mousemove event listeners

…l example

…l example (#2861)

stephanedemotte · 2017-08-03T14:46:24Z

I can change the rayOrigin from entity to mouse dynamically, but when i try to change from mouse to entity nothing append :/

dmarcos · 2017-08-03T15:35:13Z

@stephanedemotte can you open an issue?

stephanedemotte · 2017-08-03T16:36:24Z

done ;) #2929

stephanedemotte · 2017-08-21T20:04:23Z

@dmarcos do you have time to check the example ? #2929
Need to know if it's a bug or my code :/

Thank you !

dmarcos added the work in progress label Jul 11, 2017

dmarcos force-pushed the screenMouse branch 2 times, most recently from a8edde4 to cb821a5 Compare July 13, 2017 02:37

dmarcos added needs review and removed work in progress labels Jul 13, 2017

dmarcos force-pushed the screenMouse branch 2 times, most recently from 9e348bb to 18c14da Compare July 13, 2017 17:49

ngokevin reviewed Jul 13, 2017

View reviewed changes

dmarcos force-pushed the screenMouse branch from 18c14da to 7a1f9b4 Compare July 17, 2017 23:10

ngokevin reviewed Jul 18, 2017

View reviewed changes

dmarcos force-pushed the screenMouse branch 2 times, most recently from 261be15 to 6f0f7c8 Compare July 19, 2017 05:14

Add a mouse based cursor / raycaster and applies to the link traversa…

e976440

…l example

dmarcos force-pushed the screenMouse branch from 6f0f7c8 to e976440 Compare July 19, 2017 05:23

dmarcos merged commit a4b91bc into aframevr:master Jul 19, 2017

ngokevin pushed a commit that referenced this pull request Jul 19, 2017

Add a mouse based cursor / raycaster and applies to the link traversa…

e8bc421

…l example (#2861)

JonathanWade mentioned this pull request Aug 2, 2017

Conflict with... cursor="rayOrigin: mouse" jeromeetienne/AR.js#148

Closed

PlumCantaloupe mentioned this pull request Sep 18, 2018

Update GitHub description to say it's deprecated and now a core feature in A-Frame. mayognaise/aframe-mouse-cursor-component#35

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add a mouse based cursor / raycaster and applies to the link traversal example #2861

Add a mouse based cursor / raycaster and applies to the link traversal example #2861

dmarcos commented Jul 11, 2017 •

edited

Loading

ngokevin Jul 13, 2017

dmarcos Jul 14, 2017

ngokevin Jul 17, 2017 •

edited

Loading

dmarcos Jul 17, 2017 •

edited

Loading

ngokevin Jul 17, 2017

ngokevin Jul 17, 2017

dmarcos Jul 17, 2017 •

edited

Loading

ngokevin Jul 17, 2017

dmarcos commented Jul 17, 2017

ngokevin commented Jul 17, 2017

dmarcos commented Jul 17, 2017

ngokevin left a comment

ngokevin Jul 18, 2017

ngokevin Jul 18, 2017

ngokevin Jul 18, 2017

ngokevin Jul 18, 2017

ngokevin Jul 18, 2017

ngokevin Jul 18, 2017

ngokevin Jul 18, 2017

stephanedemotte commented Aug 3, 2017

dmarcos commented Aug 3, 2017

stephanedemotte commented Aug 3, 2017 •

edited

Loading

stephanedemotte commented Aug 21, 2017

Add a mouse based cursor / raycaster and applies to the link traversal example #2861

Add a mouse based cursor / raycaster and applies to the link traversal example #2861

Conversation

dmarcos commented Jul 11, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ngokevin Jul 17, 2017 • edited Loading

Choose a reason for hiding this comment

dmarcos Jul 17, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dmarcos Jul 17, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dmarcos commented Jul 17, 2017

ngokevin commented Jul 17, 2017

dmarcos commented Jul 17, 2017

ngokevin left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stephanedemotte commented Aug 3, 2017

dmarcos commented Aug 3, 2017

stephanedemotte commented Aug 3, 2017 • edited Loading

stephanedemotte commented Aug 21, 2017

dmarcos commented Jul 11, 2017 •

edited

Loading

ngokevin Jul 17, 2017 •

edited

Loading

dmarcos Jul 17, 2017 •

edited

Loading

dmarcos Jul 17, 2017 •

edited

Loading

stephanedemotte commented Aug 3, 2017 •

edited

Loading