ralsense D455 identify feature points

[junmoxiao11]

• Before opening a new issue, we wanted to provide you with some useful suggestions (Click "Preview" above for a better view):

  • Consider checking out SDK examples.
  • Have you looked in our documentations?
  • Is you question a frequently asked one?
  • Try searching our GitHub Issues (open and closed) for a similar issue.

• All users are welcomed to report bugs, ask questions, suggest or request enhancements and generally feel free to open new issue, even if they haven't followed any of the suggestions above :)

---

Required Info
Camera Model	D455
Firmware Version	(Open RealSense Viewer --> Click info)
Operating System & Version	Ubuntu20.04
Kernel Version (Linux Only)	(e.g. 4.14.13)
Platform	PC/Raspberry Pi/ NVIDIA Jetson / etc..
SDK Version	{ legacy / 2.. }
Language	{C/C#/labview/nodejs/opencv/pcl/python/unity }
Segment	{Robot/Smartphone/VR/AR/others }

Issue Description

How do I use realsense viewer to identify feature points

[MartyG-RealSense]

Hi @junmoxiao11 Are you seeking the coordinates of points on the image? If you are then coordinates are displayed in the bottom corner of the RealSense Viewer as the mouse cursor is moved over the image when in the 3D pointcloud mode of the Viewer. The '3D' option to switch to pointcloud display mode is in the top corner of the Viewer..

[junmoxiao11]

Thank you. I don't need to see the coordinates. I need to capture the feature points of the target object,which may require some code.

[MartyG-RealSense]

Could you provide more information please about what 'feature points' means in your particular situation?

[junmoxiao11]

I need to use realsense D455 depth camera to capture the movement of the object. I want to capture the movement of one point to get the whole movement. This point is the feature point what I mentioned. I want to get the depth information of the point as I observe it and record the depth data. This process may require some code, if you can help.

[MartyG-RealSense]

A single point would be too small to track the movement of in real-time, unfortunately. The tracked object should be a group of pixels that is at least 9 pixels large on the image. #4175 provides more info about this subject.

The RealSense Viewer tool can update depth in real-time as an object moves but it does not have the built-in capability for automatic tracking of the position of a particular group of moving pixels.

[junmoxiao11]

I will learn something according to your suggestion. Thank you very much for your help.

[junmoxiao11]

I read the information you gave me. But it doesn't seem to have what I want. I might need to capture feature points and record their depth information. It is best to save this information in a file. The feature point could be what you call a few pixels. I need to record their depth over a period of time.

[MartyG-RealSense]

You can record a bag file in the Viewer using the Record button in 2D mode. A bag file is like a video recording of camera data that stores the data of all enabled streams over a period of recording time. When the bag is played back, it is treated as though it is a live camera. So you can follow the movement of certain pixels over time during playback, though all pixels will be displayed rather than just a few pixels.

[MartyG-RealSense]

Hi @junmoxiao11 Do you require further assistance with this case, please? Thanks!

[junmoxiao11]

How can I figure out which pixels I need out of these pixels.

[MartyG-RealSense]

If the object that is being tracked has a particular color (such as a white ball) then it is possible to track position by a specific color using OpenCV. Google opencv track color for more details.

The RealSense Viewer does not have the capability to track particular coordinates automatically though.

[junmoxiao11]

How to implement the method you said, are there specific steps? And according to your method, can I get the coordinate information file of these pixels.

[MartyG-RealSense]

There is not a RealSense step by step guide for OpenCV-based tracking, unfortunately.

If you have not created RealSense scripting with OpenCV integration before then Intel's beginner guide at the link below may be helpful.

https://dev.intelrealsense.com/docs/opencv-wrapper

In regards to getting coordinates with a RealSense OpenCV application, please see the example shared by a RealSense user at #6239 (comment)

[MartyG-RealSense]

Hi @junmoxiao11 Do you require further assistance with this case, please? Thanks!

[MartyG-RealSense]

Case closed due to no further comments received.

[junmoxiao11]

When I was shooting with the D455 camera, I noticed a lot of black depth noise on the image. Is there any way to calibrate the camera for noise reduction?

[junmoxiao11]

Is there any way I can learn how to use the D455 camera more?

[junmoxiao11]

I want to reopen this issue. I found the code to capture the feature points, but there's a lot of depth noise on the image. This severely affected the use of the algorithm. So I needed to calibrate the camera. See if the results can be a little more accurate.

[MartyG-RealSense]

Rather than re-open this issue, let's continue the discussion at the new issue that you have created at #12689

In regard to learning more about your camera, links to useful resources are listed below.

Official RealSense blog
https://www.intelrealsense.com/blog/

Official RealSense YouTube channel
https://www.youtube.com/@IntelRealSense

White-Paper guides
https://dev.intelrealsense.com/docs/whitepapers