Aligning all streams: rgb, depth, and IR

[cassie101]

Hi, I have read about rs.align and learnt that it only aligns two streams of input (depth and rgb, or depth and ir). However, for my application, I need the alignment from 3 streams (depth, rgb, and ir). Has anyone faced similar problem before and could share the experience or provide some direction?

Camera model: SR300 and D435 (but I'm only using SR300 for now)

I have read about the comment with similar issue here but did not understand the process exactly
#1556 (comment)

[agrunnet]

Depth and IR (left camera) are aligned already because they are derived from the same camera. The third RGB camera is the one you need to align to. Once you do that you have all three. Same applies for both SR300 and D4xx series.

[cassie101]

Hi @agrunnet, thank you for your reply.
I have tried to align all inputs to depth:

#align_to = rs.stream.color
align_to = rs.stream.depth #align to depth
align = rs.align(align_to)
...
frames = pipeline.wait_for_frames()
aligned_frames = align.process(frames)
aligned_depth_frame = aligned_frames.get_depth_frame()
color_frame = aligned_frames.get_color_frame()
infrared_frame = aligned_frames.get_infrared_frame()

This way, all 3 frames seem to be aligned. However, there are artifacts in the rgb and ir images (originally not seen when frames are aligned to rgb).
Is this expected when we align frames to depth?

[RealSenseCustomerSupport]

---

The artifacts may be caused by false positive depth data. Please see #5030

This example may also be useful https://github.com/IntelRealSense/librealsense/tree/master/examples/align
Please let us know if further assistance is needed. Thank you.