rs2_deproject_pixel_to_point(), does argument "depth" have to be the actual depth of target surface?

[Boatsure]

---

Required Info
Camera Model	D435
Operating System & Version	Linux (Ubuntu 18.06)
Platform	NVIDIA Jetson
Language	python

The key step of obtaining the 3D coordinates of a pointed pixel in the RGB frame is using rs2_deproject_pixel_to_point(intrinsics, [pixel_x, pixel_y], pixel_depth), where pixel_depth comes from pixel_depth = depth_frame.get_distance(pixel_x, pixel_y), right?

So can I get the 3D coordinates at the x&y position of (pixel_x, pixel_y) but z position of (pixel_depth/2) through rs2_deproject_pixel_to_point(intrinsics, [pixel_x, pixel_y], pixel_depth/2)? Does argument "depth" have to be the actual depth of the target position surface?

[MartyG-RealSense]

Hi @surefyyq Usually when rs2_deproject_pixel_to_point is being used to obtain 3D xyz coordinates, you first have to align depth to color. A successful example of a Python script for obtaining xyz coordinates with this method can be found at #6749 (comment)

In that script, their depth value used in the rs2_deproject_pixel_to_point equation comes from the instruction depth = depth_frame.get_distance(x, y)

The SDK instruction get_distance() can be used to obtain the z-distance of a coordinate in meters.

However, if you are creating a pointcloud by performing depth to color alignment and then obtaining the 3D real-world point cloud coordinates with rs2_deproject_pixel_to_point then the use of alignment may result in inaccuracies. Using points.get_vertices() instead to generate the point cloud and then store the vertices in a numpy array whose values can be printed out should provide better accuracy. This subject is discussed in detail in the Python case at #4315

[Boatsure]

Hi@MartyG-RealSense Thanks for your reply.

The process of obtaining 3D xyz coordinates is no problem and I can get the right 3D xyz coordinates of pointed position of object surface. According to your explaination and quote, I reckon that there is no way to get the 3D coordinates of point which locates between object surface and realsense camera by rs2_deproject_pixel_to_point(intrinsics, [pixel_x, pixel_y], pixel_depth/2), right?

I took a test about it:

point_on_the_surface = rs2_deproject_pixel_to_point(intrinsics, [pixel_x, pixel_y], pixel_depth)
ponit_in_the_air = rs2_deproject_pixel_to_point(intrinsics, [pixel_x, pixel_y], pixel_depth/2)

and I found that the value of ponit_in_the_air xyz coordinates all are halved, where xy are excepted to be identical.

[MartyG-RealSense]

If you wanted to halve the pixel depth value but not affect x and y then I would try amending the pixel_depth/2 to isolate it from xy by adding a couple more brackets.

ponit_in_the_air = rs2_deproject_pixel_to_point(intrinsics, [pixel_x, pixel_y], (pixel_depth/2))

This should ensure that only the pixel_depth value is halved and not [pixel_x, pixel_y].

[MartyG-RealSense]

Hi @surefyyq Do you require further assistance with this case, please? Thanks!

[Boatsure]

@MartyG-RealSense Sorry for the delay, I tried more brackets like you advised but still, all values are halved.

This question comes from the requirement as follows:
The target is an upward convex arc in space. I want to obtain the real coordinates of the highest point of the arc surface H and another point which is on the same plane height with H but not at the arc surface.

[MartyG-RealSense]

I considered your requirement carefully. As you require two distance points and not one, you could generate a 3D depth pointcloud of the scene and retrieve the real-world Z-distance from the camera for the coordinates corresponding to the two points (the highest arc point and the one that is not at the arc surface).

[Boatsure]

Thanks for your advice @MartyG-RealSense
I will try the pointcloud way and before more information, I should close the issue for now.

[MartyG-RealSense]

You are very welcome, @surefyyq - thanks very much for the update!