Is hardware sync for multi camera configuration needed in ROS?

[SatishAkula91]

I'm planning to use three d435's along with one t265 tracking module in multi camera configuration in Ros to configure this hardware synchronization is really needed if yes how do i sync t265 with them or is there any other methods to do that ?

[MartyG-RealSense]

T265 does not have hardware sync pins. Advice on syncing T265 pose frames with 400 Series depth frames via a software-based method is given in the link below though.

IntelRealSense/librealsense#3670 (comment)

Intel has published multi-camera guides for using ROS with two cameras or three cameras, with the inter-connection based on camera serial numbers:

Two cameras
https://github.com/IntelRealSense/realsense-ros/wiki/Showcase-of-using-2-cameras

Three cameras
https://github.com/IntelRealSense/realsense-ros/wiki/showcase-of-using-3-cameras-in-2-machines

[SatishAkula91]

Thanks for the reply, I have gone through the showcase-of-using-2-cameras and 3-cameras already in those two processes they didn't mention anything about hardware synchronization for the depth cameras.

[MartyG-RealSense]

I interpreted your question as asking if it was possible to sync without using hardware sync. The links I provided describe how to set up a multi-camera system through software. I apologize if I have misunderstood your question.

[doronhi]

What is the purpose of synchronization that you need?
Just to clarify the meaning of hardware synchronization - Usually each camera works separately, say at 30Hz (i.e an image every 33.3 mSec). That means that there could be up to 33.3/2=16.6 mSecs difference between images taken by different cameras.
Hardware synchronization causes all cameras to take their images at the same time. This is useful if you want to reconstruct a scene from every set of image etc. If you are going to use 3 D435 cameras as a mean to avoid obstacles or map the scene I doubt that you actually need to hardware sync them.
T265 usually works at much higher frequencies. There is no way to make it work at the D435's images' frequencies and make it transmit it's position in correlation with the D435's images but usually, interpolation a position given at 200Hz to the time of a specific image is accurate enough.

So to summery - there is no method to hardware sync T265 and D435 camera.

I would like to point out that each message from both T265 and D435 has a timestamp that is based on hardware clock of the device, synced with the host computer. They are meant to be used for synchronization between devices.

[SatishAkula91]

@MartyG-RealSense no need to apologize you have understood in the right way only the reason why I'm getting confused is in Links for the multicamera configuration that you have mentioned is without the hardware sync. I tried that approach but the point cloud is not good, double frames, misalignments are there and finding transformations is very difficult that's why I got the doubt that is it possible to do multicamera configuration without the hardware sync?

@doronhi Thank you very much for the clear clarification from your answer what understood is we need to do hardware synch for the cameras and make it one cam as master and rest as slaves for multi-camera configuration.

Thanks.

[SatishAkula91]

@doronhi you have mentioned about the way to do hardware sync through ROS driver/wrapper in the below comment
https://github.com/IntelRealSense/librealsense/issues/2637#issuecomment-434203140

Can I know the status of that?

[doronhi]

I didn't find my name there :)
However, the ROS driver support that option. You can set the camera to be master or slave using inter_cam_sync_mode dynamic parameter.

[SatishAkula91]

Sorry, it's my mistake I thought both are the same names.
So to get master-slave behavior I have to pass the inter_cam_sync_mode parameter while starting the cameras launch file in ROS. am I right and how to validate that the cameras are in master-slave mode?

[RealSenseCustomerSupport]

---

@vnsiot - Is there anything further needed on this thread?

[Alex-Beh]

@SatishAkula91 @doronhi May I know how to set inter_cam_sync_mode via roslaunch file?

I can find the information from librealsense repo about but don't how to set it via ROS api.

        enum inter_cam_sync_mode
        {
            INTERCAM_SYNC_DEFAULT    = 0,
            INTERCAM_SYNC_MASTER     = 1,
            INTERCAM_SYNC_SLAVE      = 2,
            INTERCAM_SYNC_FULL_SLAVE = 3,
            INTERCAM_SYNC_MAX        = 258 // 4-258 are for Genlock with burst count of 1-255 frames for each trigger
        };

[MartyG-RealSense]

The opening comment of the case in the link below provides an example of setting inter_cam_sync_mode with a dynamic reconfigure instruction during runtime.

#871

I believe that a simplified one-instruction version of that complex command sequence would look something like this:

Default
rosrun dynamic_reconfigure dynparam set /camera/stereo_module inter_cam_sync_mode 0

Master
rosrun dynamic_reconfigure dynparam set /camera/stereo_module inter_cam_sync_mode 1

Slave
rosrun dynamic_reconfigure dynparam set /camera/stereo_module inter_cam_sync_mode 2

An option that can be set with dynamic reconfigure can also be set with a launch file rosparam instruction.
Since it is referencing the Stereo Module, I think that the rosparam would be something like this (change the end number for the mode that you want to set):

<rosparam> /camera/stereo_module/inter_cam_sync_mode : 1 </rosparam>

[r91andersson]

Im running librealsense v.2.38.1 and tried to set the inter cam sync mode after starting rs_camera.launch , but keep getting these errors:

Command:
rosrun dynamic_reconfigure dynparam set /camera/stereo_module inter_cam_sync_mode 1

Result:
Traceback (most recent call last): File "/opt/ros/melodic/lib/dynamic_reconfigure/dynparam", line 238, in <module> elif cmd == 'set': do_set() File "/opt/ros/melodic/lib/dynamic_reconfigure/dynparam", line 122, in do_set set_params(node, values_dict, timeout=options.timeout) File "/opt/ros/melodic/lib/dynamic_reconfigure/dynparam", line 203, in set_params client.update_configuration(params) File "/opt/ros/melodic/lib/python2.7/dist-packages/dynamic_reconfigure/client.py", line 224, in update_configuration raise DynamicReconfigureCallbackException('service call failed') dynamic_reconfigure.DynamicReconfigureCallbackException: service call failed

Result from launch file rs_camera:
[ERROR] [1602855357.012967348]: Exception thrown while processing service call: object doesn't support option #42

[MartyG-RealSense]

Is there any difference if you use a decimal number instead of a whole number for the mode number?

rosrun dynamic_reconfigure dynparam set /camera/stereo_module inter_cam_sync_mode 1.0

[r91andersson]

@MartyG-RealSense I tried, but it gave me this error:
error updating parameters: can't set parameter 'inter_cam_sync_mode' of <type 'int'>: invalid literal for int() with base 10: '3.0'

However, I tried updating the inter_cam_sync_mode to 4 from realsense-viewer console, but got the error below (seems to arise whenever I change a parameter):

I then closed realsense-viewer and launched the rs_camera.launch, and ran the command
rosrun dynamic_reconfigure dynparam get /camera/stereo_module

The result shows that the inter_cam_sync_mode was successfully set to 4:
{'laser_power': 150.0, 'groups': {'laser_power': 150.0, 'parent': 0, 'emitter_always_on': False, 'global_time_enabled': True, 'inter_cam_sync_mode': 4, 'gain': 16, 'groups': {}, 'id': 0, 'output_trigger_enabled': False, 'exposure': 8500, 'name': 'Default', 'parameters': {}, 'emitter_enabled': 1, 'enable_auto_exposure': True, 'error_polling_enabled': True, 'state': True, 'visual_preset': 0, 'type': '', 'frames_queue_size': 16}, 'emitter_always_on': False, 'enable_auto_exposure': True, 'global_time_enabled': True, 'inter_cam_sync_mode': 4, 'visual_preset': 0, 'error_polling_enabled': True, 'gain': 16, 'emitter_enabled': 1, 'frames_queue_size': 16, 'output_trigger_enabled': False, 'exposure': 8500}

So somehow, I can set the intercam sync mode from within realsense viewer(even though I keep getting errors as in the picture above), but I cant set value properly with the rosrun dynparam set command.

My setup:
librealsense 1.38.1
realsense-ros wrapper 2.2.17
Camera Firmware: 05.12.08.200

[MartyG-RealSense]

I do not know whether the inter_cam_sync_mode function of the RealSense ROS wrapper has been updated to support the additional sync modes that were introduced when the External Synchronization (Genlock) feature was released in librealsense in June 2020. Up until then, the available modes were 0 to 2 (Default, Master, Slave).

@doronhi Can you verify please what the correct instructions are for accessing inter_cam_sync_mode in the ROS wrapper with dynamic reconfigure and with launch-fle rosparam, please? Thank you.

[r91andersson]

@MartyG-RealSense A wierd thing; The next day, after a rebooting my Xavier, the realsense-viewer won't recognize my camera (nor the realsense-ros wrapper), I have also tested with one of my other realsense cameras (aslo a D435i, but with different firmware). I keep getting these errors:
09:23:51 [Warn] /home/dev/librealsense/src/ds5/ds5-factory.cpp:1119 - DS5 group_devices is empty.
09:23:54 [Info] /home/dev/librealsense/src/linux/backend-hid.cpp:1139 - Failed to read busnum/devnum. Device Path: /sys/bus/iio/devices/iio:device1

The camera are connected through the accompanying usb cable directly into the one of the Xaviers USB3 ports.

Worth mention is that nothing has changed, I have just restarted the Xavier basically.

[MartyG-RealSense]

@r91andersson It has been a couple of years since I last had a question about a busnum/devnum error. The error usually occurs on some devices that are single-board computers, rather than full PCs such as a laptop or desktop. Examples of devices it has been reported on include small devices with Intel Atom processors such as the original Up Board and the Intel Compute Stick.

In the past it has typically occurred due to insufficient power supplied to the camera when plugging the camera in via a USB OTG micro-size USB port. It is not so usual for it to occur when plugged into a full-size USB port. Can you confirm please that it is undetectable even if you unplug and replug the camera a few times, please?

[r91andersson]

@MartyG-RealSense Sorry for late reply, but I managed to solve it by reinstalling the camera drivers! I found out that the jetsonhacks had an install script (initially for Jetson Nano) that worked for me!

However, how are progress with setting the inter_cam_sync_mode via ROS wrapper?

[MartyG-RealSense]

I'm pleased to hear that you found a solution @r91andersson :)

Another RealSense user using genlock was having problems with inter_cam_sync_mode and getting the error Exception thrown while processing service call: object doesn't support option 42. They developed a workaround in an edit at the bottom of of the comment linked to below. Also see the response to the workaround from doronhi the RealSense ROS developer below that comment.

#1431 (comment)