D455 CameraInfo incorrect distortion model

[oceanusxiv]

Hello, I'm currently using the Realsense D455 camera, and I noticed that for the D455, the ROS driver reports a distortion_model of plumb_bob for the CameraInfo topic of /camera/color/camera_info.

However, reading issues like IntelRealSense/librealsense#8810 would seem to suggest that the actual distortion model used by the color camera of the D455 is in fact the inverse brown conrady, e.g. the inverse of plumb bob, rather than plumb bob itself. Indeed looking through the code in

realsense-ros/realsense2_camera/src/base_realsense_node.cpp

Line 1936 in e4938bb

if (intrinsic.model == RS2_DISTORTION_KANNALA_BRANDT4)

it appears to me that no check was ever added to see if the distortion model is inverse brown conrady, rather it defaults to plumb bob.

As such, I believe that the ROS wrapper is actually outputting incorrect calibration information when it comes to the Realsense D455 color stream. Reporting the distortion model to be plumb_bob instead of its inverse.

[oceanusxiv]

[MartyG-RealSense]

Hi @oceanusxiv There are previous discussions of the D455's color distortion model at IntelRealSense/librealsense#7734 and IntelRealSense/librealsense#9551

On most RealSense 400 Series models the coefficients are all artificially set to zero for reasons provided at IntelRealSense/librealsense#1430 (comment)

However, on the D455 camera model the color coefficients are non-zero.

Despite the unusual nature of the D455's color coefficients compared to other 400 Series models, they do not cause problems.

[oceanusxiv]

@MartyG-RealSense hi, yes I have read those previous discussions, but what I don't think has been clarified in those previous discussions, exactly, is whether these coefficients are useable by external undistort programs such as opencv. Since these distortion models are nonstandard, e.g. not simple Brown Conrady.

The documentation on projection would seem to indicate that inverse Brown Conrady requires special undistort functions in order to make it work, librealsense seems to have the necessary functions within its API, but that is of course no accessible by programs outside the driver. Im simply concerned that these camera coefficients represent a nonstandard distortion model while presenting itself as a standard one to external ROS consumers.

[MartyG-RealSense]

The KANNALA_BRANDT4 reference in the code that you quoted at the beginning of this case is apparently a check for whether the Kannala-Brandt distortion model that is used for the 'fisheye' type lenses of the RealSense T265 Tracking Camera model is the one that is being applied.

IntelRealSense/librealsense#3880 (comment)

The best rule to apply in this situation may be that if the undistorted image looks correct then the reported distortion model is likely correct.

[oceanusxiv]

@MartyG-RealSense The distortion parameters at play here are all somewhat low, so I do not think the naked eye would be able to really easily tell whether an image is correctly undistorted or not. It does still matter though since I am using it as input to various computer vision and SLAM algorithms, where accuracy of calibration parameters is key. I would highly appreciate some official clarification on whether it is in fact valid to use the inverse brown conrady coefficients with opencv's plumb bob undistort function.

[MartyG-RealSense]

One of my Intel RealSense colleagues is going to be checking the correctness of the Brown-Conrady distortion model on D455 as part of the case at IntelRealSense/librealsense#10811 so it would be worth following that discussion to see what emerges from my colleague's checks.

[MartyG-RealSense]

Hi @oceanusxiv Do you require further assistance with this case, please? Thanks!

[MartyG-RealSense]

Case closed due to no further comments received.