Raycasting to objects that are not the closest

[diarmidmackenzie]

Description:

I've been hitting various circumstances in which I want to access raycast entities that are not the closest to the camera.

Specific cases are:

• In VR, I have a control system where I can grab an object with the raycaster and move it around. While grabbed, I want the raycast line to always point to the grabbed object, even when it passes behind a closer raycastable object.
• I have some small objects that appear always on top (using depthTest: false). I want raycasting experience with these to match what the user sees, so when they can see an object, they can interact with it, even when it is further away than another object.
• Plus one more case involving a transparent plane that I want to be able to grab with a raycast, but also want to be able to raycast through.

So far I have been working around this in my application code:

• I have a customized version of "raycaster-line" that picks the correct item to draw a line to.
• I have code to grab objects that steps through the list of raycast matches to find the top priority one.

However I'm now needing to solve a similar problem in a situation where I use cursor.js.

I could make some modifications to cursor.js, but it occurred to me that a more elegant solution would be to allow raycaster to have a configurable object priority order to use, instead of distance order.

This would mean that I could define my precedence order in one place, and avoid having custom code in each of:

• raycaster-line
• cursor
• my own application code.

I have made some code changes to raycaster.js, and it's all working fine apart from one fix I needed to make to cursor as well.

PR to follow soon...

[dmarcos]

In VR, I have a control system where I can grab an object with the raycaster and move it around. While grabbed, I want the raycast line to always point to the grabbed object, even when it passes behind a closer raycastable object.

Have you played with the objects filter to determine the entities that get raycasted? Once you have grabbed the object you can change the filter to only track the entity of interest.

I have some small objects that appear always on top (using depthTest: false). I want raycasting experience with these to match what the user sees, so when they can see an object, they can interact with it, even when it is further away than another object.

Similar the objects filter might be helpful in this case. You can perhaps add a class to those entities that have depthTest false? Multiple raycasters to track different objects could also be an option I think and prioritize hits from one over the other.

Plus one more case involving a transparent plane that I want to be able to grab with a raycast, but also want to be able to raycast through.

Iterating over the list of raycasted objects is not good enough?

Better to make sure the current API cannot cover your use case before introducing changes.

Deriving your own more specialized cursor from cursor.js is a good path. We've done that in a few experiments.

[diarmidmackenzie]

Thanks for the reply.

Have you played with the objects filter to determine the entities that get raycasted? Once you have grabbed the object you can change the filter to only track the entity of interest.

Thinking about this case some more, probably the best solution is to add an additional class to the entity when it is grabbed, and then only raycast against this class while it is grabbed.

That would avoid hover behaviours on other entities as you pass over them, which is probably an improvement, even though nobody has complained about it yet.

Similar the objects filter might be helpful in this case. You can perhaps add a class to those entities that have depthTest false? Multiple raycasters to track different objects could also be an option I think and prioritize hits from one over the other.

I don't think the objects filter helps in this case. We want all objects to be raycastable all the time, but we want non-distance-based prioritization when multiple objects are under the raycaster/cursor.

Introducing multiple raycasters seems to make things more complicated:

• To get the correct raycaster line drawn, we'd need to disable lines on one raycaster based on the intersection state of another.
• Cusors map 1-1 to raycasters, so we'd need multiple cursor components, but then we'd want to disable "click" on one cursor based on the hover state of another...
• Hard to do this in a way that doesn't involve significant code change whenever we want to add a new priority class.

Iterating over the list of raycasted objects is not good enough?

Yes, this can work. I had been doing it this way, and it absolutely can work, but I realized I was writing the same piece of code in 3 different places:

• in our VR controller code (rouhhgly equivalent to laser-controls, but custom)
• in our line-display code (a customized version of raycaster-line)
• in a customized version of cursor.

The fact that all these different users of raycaster are having ot do the same processing on its output suggests to me that a better place to put this sorting function is into the raycaster component itself.

Deriving your own more specialized cursor from cursor.js is a good path. We've done that in a few experiments.

I was quite happy to take a copy of raycaster-line into our codebase and modify there, as it's pretty small and unlikely to change much.

cursor.js is quite a bit bigger, and so I'd much prefer to keep this aligned with core A-Frame if possible, to reduce future maintenance costs.

If you'd consider an alternative PR to put the proposed sortOrder function as an option on cursor, laser-controls & raycaster-line, I'd be happy to go ahead in that direction. But it seems more elegant to offer this as an option on the base raycaster component, rather than implementing the same function in each of these dependent components.