The issue of "No stream match for pointcloud chosen texture Process"

[Pran-Seven]

This issue has been posted multiple times and I have tried most of the fixes suggested but none seem to work.
System specs: Real sense D415 cameras (2), ubuntu 18.04, ros melodic
Everything works great on real sense viewer, but when try to launch 2 cameras via rs_rgbd or rs_camera it either fails to start depth stream - hardware error or when I try to select the dept point cloud from rviz I get no stream match error and nothing is visualized. Sometimes one of the camera publishes and that degrades as well within a few minutes. Tried with multiple computers, multiple cameras but the issue persists. How to go about fixing this issue?

[MartyG-RealSense]

Hi @Pran-Seven The warning No stream match for pointcloud chosen texture process - Color can appear every time that there is an unavailable texture, likely because of a frame drop. The point cloud will not be published in the frame that the message occurs.

In a previous case of this error, a RealSense ROS user was using a laptop running off the battery only and found that the error disappeared if the laptop was connected to a mains power socket.

Are you using a battery-driven laptop, please?

[Pran-Seven]

Hey @MartyG-RealSense thanks for the prompt response. Yes, i could gather that from the previous issues posted, but I am facing an issue where the error appears and the stream does not even start. In some case like you mentioned, the stream is running and I get the warning once in a while and I can see the frame drop on rviz then, but I encounter the stream does not start issue more than this.

The laptop is connected to a mains socket during most of my testing and I have tested it on a Beelink PC which runs only on power and still face the same issue.

Update: Camera works fine @ 15fps when connected to a USB 2.0 interface but does not work on a 3.0 interface. Unfortunately could not test with both connected to USB 2.0 interfaces due to unavailability of ports.
For reference: librealsense version - 2.50.0
realsense-ros version - 2.3.2

[MartyG-RealSense]

Have you tried launching the two cameras with a single roslaunch command using rs_multiple_devices.launch

When launching with rs_camera.launch, are you launching the two cameras in separate ROS terminals as recommended by the multiple camera section of the ROS wrapper documentation?

https://github.com/IntelRealSense/realsense-ros#work-with-multiple-cameras

Also, does the camera work on a USB 3.0 interface if you use the custom launch command below to set both depth and color to 640x480 and 30 FPS:

roslaunch realsense2_camera rs_camera.launch depth_width:=640 depth_height:=480 depth_fps:=30 color_width:=640 color_height:=480 color_fps:=30

[Pran-Seven]

Yeah I am launching it as per the documentation.
The custom launch command also exhibits the same error behaviour with stream not loading due to no stream match for the chosen texture.

[MartyG-RealSense]

Are you using the official 1 meter cable supplied with the camera, please? There have been cases where a USB 2 cable has unknowingly been used instead of a USB 3 one when selecting an own choice of cable.

As you have tested with multiple cameras and multiple computers and still have the problem, that would suggest that the camera hardware, the computer and the ROS wrapper are not at fault (as you would have had to install the wrapper individually on each computer). The cameras also work correctly in the RealSense Viewer tool, indicating that the librealsense SDK installation is okay too.

What method do you use to install librealsense and the ROS wrapper on your computers, please? And which Ubuntu kernel version are you using?

[MartyG-RealSense]

Hi @Pran-Seven Do you require further assistance with this case, please? Thanks!

[Pran-Seven]

Will get back shortly regarding this

[MartyG-RealSense]

Thanks very much @Pran-Seven for the update. I look forward to your next report. Good luck!

[MartyG-RealSense]

Hi @Pran-Seven Do you have an update about this case that you can provide, please? Thanks!

[Pran-Seven]

Yeah after multiple trials, somehow changing the realsense-ros and librealsense versions and cables seemed to fix the issue. There is a new issue I am facing now, when I am trying to launch two D415's using rs_rgbd.launch and passing an external manager, only one camera gets launched, any possible fixes for this? Or any insight into what lets realsense_camera_manager launch both cameras and run the nodelet thread without any issues.

[MartyG-RealSense]

Have you tried launching the two cameras with rs_rgbd.launch in separate ROS terminals as suggested in the ROS wrapper documentation, please?

https://github.com/IntelRealSense/realsense-ros#work-with-multiple-cameras

[Pran-Seven]

That would again launch with the realsense_camera_manager. The issue arises when we try to pass an external manager, when launching from our launch file which points to rs_rgbd.launch. The launch file is as attached.

<launch> 
 <node pkg="robot_localization" type="ekf_localization_node" name="ekf_localization" clear_params="true">
        <rosparam command="load" file="$(find seven_robot_localization)/params/ekf.yaml"/>
 </node>

 <node pkg="nodelet" type="nodelet" name="pcmergeManager" args="manager" output="screen"/> 
 <arg name="realsense_enabled"/>
    <group if="$(arg realsense_enabled)">
     <include file="$(find realsense2_camera)/launch/rs_camera.launch">
       <arg name="camera" value="sensor_d415_right"/>
       <arg name="external_manager" value="true"/>
       <arg name="manager" value="/pcmergeManager"/> 
       <arg name="serial_no" value="746112061710"/>
       <arg name="enable_pointcloud" value="true"/>
       <arg name="enable_depth" value="true"/>
       <arg name="depth_height" value="720"/>
       <arg name="depth_width" value="1280"/>
       <arg name="color_width" value="1280"/>
       <arg name="color_height" value="720"/>
       <arg name="bond" value="true" />
      
     </include>
     <include file="$(find realsense2_camera)/launch/rs_camera.launch">
       <arg name="camera" value="sensor_d415_left"/>
       <arg name="external_manager" value="true"/>
       <arg name="manager" value="pcmergeManager"/> 
       <arg name="serial_no" value="746112060570"/>
       <arg name="enable_pointcloud" value="true"/>
       <arg name="enable_depth" value="true"/>
       <arg name="depth_height" value="720"/>
       <arg name="depth_width" value="1280"/>
       <arg name="color_width" value="1280"/>
       <arg name="color_height" value="720"/>
       <arg name="bond" value="true" />
       
     </include>
     <!-- <rosparam>
     /sensor_d415_right/stereo_module/inter_cam_sync_mode : 1 
     /sensor_d415_left/stereo_module/inter_cam_sync_mode : 2 
     /sensor_d415_right/global_time_enabled : false
     /sensor_d415_left/global_time_enabled : false
     /sensor_d415_right/auto_exposure_priority: false
     /sensor_d415_left/auto_exposure_priority: false
     /sensor_d415_right/auto_exposure: true
     /sensor_d415_left/auto_exposure: true
     </rosparam> -->
    </group>
  
  <include file="$(find seven_robot_vision)/launch/pointCloudMerger.launch">
    <!-- <arg name="external_manager" value="pcmergeManager"/> -->
  </include>


</launch>

When I launch this with the parameter external manager, only one camera gets launched.

[MartyG-RealSense]

I note that you define the two cameras with two sets of <include> </include> one after the other. I wonder whether this approach is telling the wrapper to look at one camera with the first set and then telling it to look at a camera with a different serial number with the second set, so that the launch ignores the first camera because it is finding a camera that has the serial of the second set instead. In other words, overriding the instructions in the first set with the instructions in the second set.

[Pran-Seven]

But the same lines with both includes right after each other seem to work when realsense_camera_manager is the nodelet manager.

[Pran-Seven]

Hey was able to fix it, I think this issue can be closed thank you. But I am getting a bond broken error between the two camera topics and a different nodelet subscribing to it, but I don't suppose that would be a question for this forum right?
Found a few links for the bond broken as well but they don't resemble our situation so yeah.

[MartyG-RealSense]

Thanks very much, I was just conducting further research on your case. It's great to hear that you were successful. :)

I would recommend defining depth_fps and color_fps too in your external manager. Since wrapper 2.2.22 onwards, if you do not provide 3 factors (width, height and FPS) in a custom stream configuration then the ROS launch deems the custom configuration invalid and instead applies the default stream profile of the particular RealSense camera model being used.

[Pran-Seven]

Well thank you! I did try settig the fps at launch but I am still plagued by the same issue of bond broken after a few seconds.

[MartyG-RealSense]

There was a bond broken case with the ROS1 wrapper where decreasing the resolution resolved the error in that particular case, as described at #1619 (comment)

[Pran-Seven]

Tried that as well, but the results are the same

[MartyG-RealSense]

Someone else with the error used the workaround of a script at #934 (comment) that automatically repeated the roslaunch if the launch exited because of bond broken.

Have you tested whether the error still occurs if you add initial_reset:=true to your roslaunch instruction to reset the camera at launch? In your case it may be best to add it as an arg to your external manager.

<arg name="initial_reset" value="true"/>

[MartyG-RealSense]

Hi @Pran-Seven Do you require further assistance with this case, please? Thanks!

[Pran-Seven]

Yeah, I was able to figure out that the issue lies with different frame rates being published by the two cameras and I am looking for a hardware sync solution, but even after connecting the cameras I still can't get them to sync. Looking at the thread hardware_sync and other related for solutions

[MartyG-RealSense]

Hardware sync will not synchronize the frame rate (FPS) of cameras, but instead synchronizes their timestamps with a master camera or trigger pulse. Hardware sync would therefore not be a solution for differences in FPS between one camera and another.

Normally, if auto_exposure is set to true and auto_exposure_priority is set to false then the FPS rates of depth and color should be forced to try to maintain the same FPS rate instead of being permitted to vary the FPS.

I note that in your launch file above you are applying these two settings. However, these particular settings should be specified for the particular type of camera sensor (stereo_module for the depth sensor and rgb_camera for the color sensor), and auto_exposure_priority is a color sensor option. So I wonder whether the settings should be defined like this:

/sensor_d415_right/stereo_module/auto_exposure: true
/sensor_d415_left/stereo_module/auto_exposure: true
/sensor_d415_right/rgb_camera/auto_exposure: true
/sensor_d415_left/rgb_camera/auto_exposure: true

/sensor_d415_right/rgb_camera/auto_exposure_priority: false
/sensor_d415_left/rgb/camera/auto_exposure_priority: false

[Pran-Seven]

The reason i need a fps sync is I am trying to stitch two pointclouds together. Since they are coming at different frequencies, the code I am using keeps breaking with the error bond broken. The Approximate Time Sync methods also don't seem to be working which is why we went for an hardware sync. I did change he settings in the launch file as you mentioned but no luck. Would you happen to any robust methods/codebases for pointcloud stitching in ros that does not pertain to the conix solutions or others mentioned in the common threads?

[MartyG-RealSense]

Intel have a RealSense ROS guide for stitching together pointclouds from two cameras attached to the same computer at the link below.

https://github.com/IntelRealSense/realsense-ros/wiki/Showcase-of-using-2-cameras

There is also a guide for three cameras spread across two computers.

https://github.com/IntelRealSense/realsense-ros/wiki/showcase-of-using-3-cameras-in-2-machines

[Pran-Seven]

Yeah I have gone through those and have implemented them and they work well, but I can't incorporate those into my ros project, therefore looking for other alternatives

[MartyG-RealSense]

Some ROS users have taken the approach of combining laser scans into a single pointcloud. There is a ROS package that has been used by some RealSense ROS users called pointcloud_to_laserscan

http://wiki.ros.org/pointcloud_to_laserscan

A guide at the link below discusses using pointcloud_to_laserscan to combine laserscans into a single pointcloud.

https://medium.com/@amritgupta1999/merging-data-from-multiple-lidar-s-in-ros-e890fb60cbbf

[MartyG-RealSense]

Hi @Pran-Seven Do you require further assistance with this case, please? Thanks!

[Pran-Seven]

Nope, thanks a lot. You have been very helpful.

[MartyG-RealSense]

You are very welcome, @Pran-Seven - thanks very much for the update!