Picking misses plane at specific angles with orthographic projection

[dmlary]

I'm using a plane centered on the origin to get x/z coords. I discovered that with an orthographic projection, raycasting will miss the bottom edge of the plane if the camera has some rotation. Looking straight (or nearly so) down, and the problem goes away.

Screen.Recording.2023-05-17.at.8.21.44.PM.mov

In this video, the green rectangle is the only Pickable entity. The plane is correctly picked for most of the screen, but when I mouse over it on the lower portion of the screen, it no longer picks the plane.

If I then rotate the camera to look down, picking works correctly.

This doesn't occur when using perspective projection.

[aevyrie]

What picking backend are you using? It looks like the mod_raycast backend?

[dmlary]

I believe it is the mod_raycast backend. I'm using the default features for the crate, and DefaultPickingPlugins.

[dmlary]

Looking at the implementation of the raycast backend, this does look like it's most likely a bug in bevy_mod_raycast.

I'm planning to dig deeper into it tomorrow. I've confirmed the angle of the camera in reference to the plane does have an impact on where the raycasting stops. It doesn't matter how large the plane is (working with 10,000x10,000 normally) at a 30 degree angle to the plane, the raycast misses the bottom 1/4th of the screen. The plane is absolutely present there.

Do you have any suggestions of where to start looking, or any additional data that you'd like to see to help figure this out? I've started reading the raycast code, but any direction would be appreciated.

[dmlary]

Oh, this has something to do with the depth of the hit! I enabled debugging and used the mouse to trace where raycasting stops on the plane. The only thing that stood out is that once depth should go below 0.0 due to mouse movement, the raycast stops hitting the plane.

In this attached video, there is a 10000x10000 (x, z) plane at Y = 0 that is the picking target. It takes up the entire screen when not transparent.

raycast-depth.mov

[dmlary]

Digging further, I started dumping the results of Ray3d::from_screenspace and found that it stops detecting the plane once the Ray3d.origin y-value goes negative.

For a moment I thought maybe the problem was that using shape::Plane there is no "back" of the plane. You can see this when you flip to the other side of the plane and it disappears. I added a second pickable plane flipped on the x-axis to give the plane a back to pick from if the ray is cast from y < 0. This did not resolve the problem.

I'm curious now about the calculated Ray3d.origin() value. From an naive perspective, I don't really understand why the Y value for the origin goes negative in this case. Maybe it's because this is an orthographic projection, not perspective?

Note that the problem reproduces with the camera under the plane, but on the top of the screen instead of the bottom.

picking-two-sided.mov

[aevyrie]

it stops detecting the plane once the Ray3d.origin y-value goes negative.

Is this the input ray? My initial suspicion is that the hit is behind the camera (negative depth), which should be ignored, but orthographic view is rendering objects behind the camera.

[dmlary]

That's exactly what is going on. I was just typing up a new issue in bevy_mod_raycast for this.

Should update_raycast() not prune these hits when doing ortho projection?

[dmlary]

Here's the simplest demonstration of it I stumbled across. Two xz planes: pink is at y=0, tan is at y=10. The camera is at y=2.5, and pointed at the origin. With ortho projection, that tan plane is visible on a third of the screen, but it's technically behind the camera. Of course picking prunes the hits to the tan plane at Y=10 when I mouse over it, because its behind. Instead the hit is for the pink plane at Y=0

[dmlary]

Resolution

Alright, a couple of the problems I had in my code:

• The Camera was 5.0 units from the camera focus
• I had increased projection near to -100 because I was seeing clipping of tiles at the bottom of the screen as a result of the camera being so close

Fix / Workaround

Increase the distance of the camera from the focus. For my case, I increased the distance of the camera to 20, with no change in the on-screen projection. This also stopped my camera from rendering objects behind it.

This will resolve raycast picking issues, and allow a switch to near: 0.

[aevyrie]

To add to this - setting the near plane negative resulted in objects behind the camera being rendered. Raycasting does not consider objects behind the camera to be hittable. More precisely, it can raycast against these objects, but will ignore them if the hit is behind the camera.

[dmlary]

This is resolved by aevyrie/bevy_mod_raycast#84