Pointcloud projected is off in scale I think

[FPSychotic]

I just want warn about an issue I found in d435 and d435 cameras. As it is evident something is absolutely off in the depth to pointcloud projection, not sure if in the realsense sdk or ros wrapper. In the image you can see, the lidar/d435's depth images fits and match in scale,size of the things, obviously so better the lidar one, as do the color image, which will colourise lidar and d435 correctly, but the realsense point cloud from Depth projection (aligned to color,) is totally off, it is not displaced as it looks in a fast view, it is out of scale, because that looks displaced, quite worse thing that only displaced, displacement could be fixed, scale likely not.
[18:20]
if you use realsense in a critical application related with distance measurement , I would check that. Tested with 3 d435 and d435i, I don't know in what moment that went wrong, but obviously wasn't like that time ago. Could be the lidar , but in that case would be impossible match and calibrate color with lidar, so it is the depth projection what is wrong. The frame axes doesn't appear in the picture but they in the same point. Measurement with lidar is OK in size using the RVIZ measurement tool. d435 fov is bigger but that should not affect to the depth, wall should be in the same line.

Noetic
last realsense apt packages.

[MartyG-RealSense]

Hi @FPSychotic Are you using the ROS1 or ROS2 wrapper, please? And what is your ROS launch instruction?

Does color map onto depth correctly in the 3D pointcloud mode of the RealSense Viewer tool if you enable the depth stream first and then the RGB stream secondly to automatically map color onto the pointcloud depth coordinates?

[FPSychotic]

In SDK doing what you said with this setup, I think standard still happens. The table is 205cm, and as you see the measurement tool says 265cm. color and depth always match, but the depth is off, but the lateral distance too, so it is the scale what is off.

{ "colorizer.Color Scheme": "0", "colorizer.Frames Queue Size": "16", "colorizer.Histogram Equalization Enabled": "0", "colorizer.Max Distance": "6", "colorizer.Min Distance": "0", "colorizer.Stream Filter": "1", "colorizer.Stream Format Filter": "1", "colorizer.Stream Index Filter": "-1", "colorizer.Visual Preset": "0", "colorizer.enabled": "1", "notifications.fw_update_alert.5.14.0.0.845112071035.next": "1672333145", "performance.show_fps": "1", "ply.encoding": "1", "ply.mesh": "0", "ply.normals": "1", "viewer_model.is_3d_view": "1", "viewer_model.is_measuring": "0", "viewer_model.output_open": "0", "viewer_model.post_processing.Intel RealSense D435I.RGB Camera.Decimation Filter.Filter Magnitude": "2", "viewer_model.post_processing.Intel RealSense D435I.RGB Camera.Decimation Filter.Frames Queue Size": "16", "viewer_model.post_processing.Intel RealSense D435I.RGB Camera.Decimation Filter.Stream Filter": "2", "viewer_model.post_processing.Intel RealSense D435I.RGB Camera.Decimation Filter.Stream Format Filter": "0", "viewer_model.post_processing.Intel RealSense D435I.RGB Camera.Decimation Filter.Stream Index Filter": "-1", "viewer_model.post_processing.Intel RealSense D435I.RGB Camera.Decimation Filter.enabled": "0", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Decimation Filter.Filter Magnitude": "2", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Decimation Filter.Frames Queue Size": "16", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Decimation Filter.Stream Filter": "1", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Decimation Filter.Stream Format Filter": "1", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Decimation Filter.Stream Index Filter": "-1", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Decimation Filter.enabled": "1", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Depth Huffman Decoder.Frames Queue Size": "16", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Depth Huffman Decoder.Stream Filter": "1", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Depth Huffman Decoder.Stream Format Filter": "28", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Depth Huffman Decoder.Stream Index Filter": "-1", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Depth Huffman Decoder.enabled": "1", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Depth to Disparity.enabled": "1", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Disparity to Depth.enabled": "1", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Filter By Sequence id.Frames Queue Size": "16", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Filter By Sequence id.Sequence Id": "1", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Filter By Sequence id.enabled": "0", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.HDR Merge.Frames Queue Size": "16", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.HDR Merge.enabled": "1", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Hole Filling Filter.Frames Queue Size": "16", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Hole Filling Filter.Holes Fill": "1", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Hole Filling Filter.Stream Filter": "1", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Hole Filling Filter.Stream Format Filter": "1", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Hole Filling Filter.Stream Index Filter": "-1", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Hole Filling Filter.enabled": "0", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Spatial Filter.Filter Magnitude": "2", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Spatial Filter.Filter Smooth Alpha": "0.5", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Spatial Filter.Filter Smooth Delta": "20", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Spatial Filter.Frames Queue Size": "16", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Spatial Filter.Holes Fill": "0", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Spatial Filter.Stream Filter": "1", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Spatial Filter.Stream Format Filter": "1", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Spatial Filter.Stream Index Filter": "-1", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Spatial Filter.enabled": "1", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Temporal Filter.Filter Smooth Alpha": "0.4", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Temporal Filter.Filter Smooth Delta": "20", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Temporal Filter.Frames Queue Size": "16", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Temporal Filter.Holes Fill": "3", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Temporal Filter.Stream Filter": "1", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Temporal Filter.Stream Format Filter": "1", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Temporal Filter.Stream Index Filter": "-1", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Temporal Filter.enabled": "1", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Threshold Filter.Frames Queue Size": "16", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Threshold Filter.Max Distance": "4", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Threshold Filter.Min Distance": "0.1", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Threshold Filter.Stream Filter": "1", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Threshold Filter.Stream Format Filter": "1", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Threshold Filter.Stream Index Filter": "-1", "viewer_model.post_processing.Intel RealSense D435I.Stereo Module.Threshold Filter.enabled": "1", "viewer_model.sdk_version": "25301", "viewer_model.settings_tab": "1", "viewer_model.show_stream_details": "1", "window.height": "656", "window.is_fullscreen": "0", "window.maximized": "1", "window.position_x": "72", "window.position_y": "64", "window.saved_pos": "1", "window.saved_size": "1", "window.width": "1208", "y411.Frames Queue Size": "16", "y411.Stream Filter": "0", "y411.Stream Format Filter": "0", "y411.Stream Index Filter": "-1", "y411.enabled": "1", "yuy2rgb.Frames Queue Size": "16", "yuy2rgb.Stream Filter": "0", "yuy2rgb.Stream Format Filter": "0", "yuy2rgb.Stream Index Filter": "-1", "yuy2rgb.enabled": "1" }

As I said ROS1 with noetic , ubuntu 20.04, last apt packages in x86 machine.
`

/camera/stereo_module/emitter_enabled: 1

  <arg name="enable_pointcloud"        value="$(arg enable_pointcloud)"/>
  <arg name="pointcloud_texture_stream" value="$(arg pointcloud_texture_stream)"/>
  <arg name="pointcloud_texture_index"  value="$(arg pointcloud_texture_index)"/>
  <arg name="enable_sync"              value="$(arg enable_sync)"/>
  <arg name="align_depth"              value="$(arg align_depth)"/>

  <arg name="fisheye_width"            value="$(arg fisheye_width)"/>
  <arg name="fisheye_height"           value="$(arg fisheye_height)"/>
  <arg name="enable_fisheye"           value="$(arg enable_fisheye)"/>

  <arg name="depth_width"              value="$(arg depth_width)"/>
  <arg name="depth_height"             value="$(arg depth_height)"/>
  <arg name="enable_depth"             value="$(arg enable_depth)"/>

  <arg name="confidence_width"         value="$(arg confidence_width)"/>
  <arg name="confidence_height"        value="$(arg confidence_height)"/>
  <arg name="enable_confidence"        value="$(arg enable_confidence)"/>
  <arg name="confidence_fps"           value="$(arg confidence_fps)"/>

  <arg name="color_width"              value="$(arg color_width)"/>
  <arg name="color_height"             value="$(arg color_height)"/>
  <arg name="enable_color"             value="$(arg enable_color)"/>

  <arg name="infra_width"              value="$(arg infra_width)"/>
  <arg name="infra_height"             value="$(arg infra_height)"/>
  <arg name="enable_infra"             value="$(arg enable_infra)"/>
  <arg name="enable_infra1"            value="$(arg enable_infra1)"/>
  <arg name="enable_infra2"            value="$(arg enable_infra2)"/>
  <arg name="infra_rgb"                value="$(arg infra_rgb)"/>

  <arg name="fisheye_fps"              value="$(arg fisheye_fps)"/>
  <arg name="depth_fps"                value="$(arg depth_fps)"/>
  <arg name="infra_fps"                value="$(arg infra_fps)"/>
  <arg name="color_fps"                value="$(arg color_fps)"/>
  <arg name="gyro_fps"                 value="$(arg gyro_fps)"/>
  <arg name="accel_fps"                value="$(arg accel_fps)"/>
  <arg name="enable_gyro"              value="$(arg enable_gyro)"/>
  <arg name="enable_accel"             value="$(arg enable_accel)"/>

  <arg name="publish_tf"               value="$(arg publish_tf)"/>
  <arg name="tf_publish_rate"          value="$(arg tf_publish_rate)"/>

  <arg name="filters"                  value="$(arg filters)"/>
  <arg name="clip_distance"            value="$(arg clip_distance)"/>
  <arg name="linear_accel_cov"         value="$(arg linear_accel_cov)"/>
  <arg name="initial_reset"            value="$(arg initial_reset)"/>
  <arg name="reconnect_timeout"        value="$(arg reconnect_timeout)"/>
  <arg name="wait_for_device_timeout"  value="$(arg wait_for_device_timeout)"/>
  <arg name="unite_imu_method"         value="$(arg unite_imu_method)"/>
  <arg name="topic_odom_in"            value="$(arg topic_odom_in)"/>
  <arg name="calib_odom_file"          value="$(arg calib_odom_file)"/>
  <arg name="publish_odom_tf"          value="$(arg publish_odom_tf)"/>
  <arg name="stereo_module/exposure/1" value="$(arg stereo_module/exposure/1)"/>
  <arg name="stereo_module/gain/1"     value="$(arg stereo_module/gain/1)"/>
  <arg name="stereo_module/exposure/2" value="$(arg stereo_module/exposure/2)"/>
  <arg name="stereo_module/gain/2"     value="$(arg stereo_module/gain/2)"/>

  <arg name="allow_no_texture_points"  value="$(arg allow_no_texture_points)"/>
  <arg name="ordered_pc"               value="$(arg ordered_pc)"/>
  
</include>

`

with ordered true, pointcloud filter, align true will happen the same.

Firmware version 05.13.00.50
but viewer 2.53.1 is asking 5.14.00 , so it could be the firmware, I'm going to update and check. but even if that is the issue, you should add a warning because in a depth camera, used in drones, robotics etc, scale is off is a very very bad secondary effect by only have a previous firmware version. No good at all.

Edit: updated the firmware to 5.14.00 and the problem persist. I note as a wide distorded fov, it looks as it is taken the d455 fov or something like that, I never changed the calibration, it is stock, going to test another d435.

[FPSychotic]

I tried with a d435 instead d435i , and it just in first impression is evident the image has less fov and the measurement is right. Depth is quite better too I think. Could be the SDK confusing a d435 with a d455?

[MartyG-RealSense]

Thanks very much @FPSychotic for your updates above.

It should not be possible for a D435 to be mis-identified for a D455 as they have different internal circuit boards. D435 has a D430 depth module and V1 version of the Vision Processor D4. D455 has a D450 depth module, an IMU and V3 version of the Vision Processor D4. The firmware driver bases its identification of the model number based on the camera components that it can detect.

If the D435 that you tried in the comment above has correct measurement, could you try resetting the seemingly incorrect D435i to its default factory-new calibration in the RealSense Viewer please to see whether this positively affects the D435i's measurements. This procedure can be performed using the instructions at IntelRealSense/librealsense#10182 (comment)

[FPSychotic]

It worked, also I tested 2 extra d435i and they where off too. I didnt use any calibration tool on them, I only did in a d435 once due to a 4d module swap after fried it (now it is a d430 + a rgb camera without calibration). No clue how the calibration values where changed as no other person have access to the cameras
Fixing all of them now.
Thanks by the support.

[MartyG-RealSense]

You are very welcome. That's great to hear that a factory reset solved the issue for you!