How can I get the aligned infrared image?

[hosea7456]

---

Required Info
Camera Model	D400
Firmware Version	(Open RealSense Viewer --> Click info)
Operating System & Version	Win 10
Kernel Version (Linux Only)	(e.g. 4.14.13)
Platform	PC
SDK Version	2.54.1
Language	C++
Segment	others

Issue Description

How can I get the aligned infrared image but not the infrared image overlaid with the depth image(as following image)?

My code is:
rs2::frameset frames1 = pipeline.wait_for_frames();
rs2::align align_to_depth(RS2_STREAM_DEPTH);
rs2::frameset frames = align_to_depth.process(frames1);
rs2::frame infrared_frame = frames.get_infrared_frame();
cv::Mat infrared_img(cv::Size(frame_width, frame_height), CV_8UC1, (void*)infrared_frame.get_data(), cv::Mat::AUTO_STEP);

[MartyG-RealSense]

Hi @hosea7456 There is no need to perform alignment between infrared and depth if you do not want a combined image. This is because the left infrared image is already perfectly time-syncronized and pixel-perfect aligned with the depth image by default, This is because the left infrared sensor is also the origin point of the depth image.

[hosea7456]

Hi @hosea7456 There is no need to perform alignment between infrared and depth if you do not want a combined image. This is because the left infrared image is already perfectly time-syncronized and pixel-perfect aligned with the depth image by default, This is because the left infrared sensor is also the origin point of the depth image.

Thank you for your prompt response! I got it, I've been using "rs2::frame infrared_frame = frames.get_infrared_frame(2)" by default.

[MartyG-RealSense]

You are very welcome. Index '2' is the right-side infrared stream, whilst '1' is the left stream that is aligned with depth. You do not need to use an index number if only using one infrared stream though as the left stream is used by default when there is no index number specified.

[MartyG-RealSense]

Hi @hosea7456 Do you require further assistance with this case, please? Thanks!

[MartyG-RealSense]

Case closed due to no further comments received.

[jwwang0-0]

Just wondering, is it possible to align the infrared frame to the color frame? The image I got from this process seems to be off with the color frame.

            frames = self.pipeline.wait_for_frames()       
            # Align the depth frame to color frame
            align = rs.align(rs.stream.color)
            aligned_frames = align.process(frames)
        
            # Get aligned frames
            aligned_depth_frame = aligned_frames.get_depth_frame()
            color_frame = aligned_frames.get_color_frame()
            ir_frame = aligned_frames.get_infrared_frame(1)

[MartyG-RealSense]

Hi @jwwang0-0 When using align_to, one of the two streams that is being aligned has to be depth. However, because the infrared stream is already aligned with the depth stream it is possible to produce a combined depth, color and infrared image. An example of doing so in Python is at #5093

[jwwang0-0]

Thanks! Your answer is quick, great and useful 😆. Sorry if I may ask one more question, is the focal point located at (0,0,0) of the depth frame?

[MartyG-RealSense]

When depth is unaligned, its 0,0,0 origin is the center-line of the left infrared sensor. But when depth is aligned to color, the depth origin becomes the center-line of the RGB color sensor.

[jwwang0-0]

Cool! Thank you for your explanation.

[jwwang0-0]

The last question: I am doing some accuracy tests of the D455. The official document said that its accuracy is less than 2% within 4m. From my test result, an object moved 20mm in real; based on realsense, it moved 18mm. I think it is quite good in this case. Do you have more data about the accuracy of it? 1m with 2% is 2cm which is quite poor.

[MartyG-RealSense]

Error starts at around zero at the front glass of the camera and increases linearly as distance of an observed object / surface from the camera increases. This is called RMS Error.

The D455 camera model has 2x the accuracy over distance of the D435 models. For example, at 6 meters distance from the camera the D455 has the same accuracy as a D435 has at only 3 meters.