Global timestamp sync loss after update_samples_base

[oren-hinkis]

---

Required Info
Camera Model	D435i
Firmware Version	5.13.0.50
Operating System & Version	Linux 16.04
Kernel Version (Linux Only)	4.15.0-136
Platform	PC
SDK Version	2.50.0
Language	C++ Ros Kinetic
Segment	Robot

Issue Description

I am running two D435i cameras using ROS Kinetic (v.3.2.3) and RSUSB backed. Built both from source since apt install for kinetic targets 2.45.0.

Please see the following debug logs. It seems that the call to update_samples_base coincides with the start of the timestamp errors.
realsense_debug_log.txt

The following is a graph of the timestamps of both of the cameras. One follows time linearly while the other one spikes up to an error of approximately 1 second and then settles back down, this starts precisely on the frame after update_samples_base.

[MartyG-RealSense]

Hi @oren-hinkis Could you confirm please which RealSense ROS wrapper version you are using. The 3.2.3 version mentioned above is for ROS2, whilst the most recent ROS1 version for Kinetic is 2.3.2.

https://github.com/IntelRealSense/realsense-ros/releases/tag/2.3.2

The RSUSB backend method of building librealsense from source code is not best suited to multiple camera applications and works better for single camera projects. The V4L backend, which involves patching the kernel before building from source with RSUSB = false, is the best librealsense build type for multicam.

https://github.com/IntelRealSense/librealsense/blob/master/doc/installation.md

For a comparison of the advantages and disadvantages of these two installation methods (RSUSB vs kernel patching), you can visit #5212 (comment) and scroll down through the linked-to comment to the section headed What are the advantages and disadvantages of using libuvc vs patched kernel modules?

[oren-hinkis]

Thanks for the quick reply!

My apologies, I am using 2.3.2, not 3.2.3.

We have attempted a V4L installation along with the ROS driver which had a multitude of its own issues, some which we were able to resolve and some which have us stumped. We were unable to get global timestamps in the ROS driver working accurately using V4L, we had an immense amount of jitter and future dated timestamps and as a result in our V4L version we opted to calculate timestamps through frame metadata.

The RSUSB attempts have been slightly more forgiving and we have seem to have narrowed it down to our final issue of this strange time skip. I will attempt to recreate the timestamp issue with RSUSB while only using a single camera to ensure that the issue is our multi-cam use.

[oren-hinkis]

We ran a small test set of dual camera V4L robots using global stamping to see if the issue is present there. It seems that it is, however instead of just one camera getting the synchronization issues, all Realsense sensors and data experience the shift in unison.

We were unable to catch the realsense logs for this specific event, but I will update with logs on the next occurrence.

A few more details about our V4L setup
Camera 1 - IR1@90FPS/848x480 and Gyro@200fps
Camera 2 - IR1@30FPS/1280x720
Installation of librealsense 2.50.0 was done through apt get (dkms, dev, udev, utils), only one version of librealsense is installed (checked via dpkg -l | grep "realsense")
Installation of Ros Driver (v2.3.2) from source. Operating using rs_multiple_devices.launch
NTP sync to a single cloud server is enabled, rtcsync is enabled

[MartyG-RealSense]

IntelRealSense/realsense-ros#1756 (comment) suggests a solution for robots with timestamps that are in the future.

Does performance improve if you add initial_reset:=true to your roslaunch instruction to reset the camera at launch?

You could also try disabling global timestamps with the method described at IntelRealSense/realsense-ros#1906 to see whether it makes a positive difference.

[oren-hinkis]

We have attempted with initial_reset:= true as well and the behavior persists.

If I understood the suggestion correctly it means accepting poorly stamped data into the system and only referencing it when it is no longer future stamped. However my main concern is not the avoidance of TF errors, it is that data timestamp is incorrect by a significant margin (~1 second)!

We have attempted turning off the global timestamps feature altogether and the result was simply a growing time difference between the frame stamp and the system time, as expected and similar to the result mentioned in the thread. In an attempt to circumvent this we modified the hardware timestamp calculation by applying a patch similar to this commit in which the frame timestamp is calculated using metadata. Utilizing RS2_FRAME_METADATA_BACKEND_TIMESTAMP for V4L and RS2_FRAME_METADATA_TIME_OF_ARRIVAL for RSUSB.

I am still hoping that we can utilize the global timestamps feature instead of relying on what is likely be an issue prone alternative approach. Perhaps this PR is relevant, it seems like a similar issue? I'm still working through understanding this PR, and will likely attempt it once I do.

[MartyG-RealSense]

Your timestamp graph does seem to be visually similar to the one in the PR, yes.

[oren-hinkis]

The PR fix seems to work

[MartyG-RealSense]

That's great to hear, @oren-hinkis - thanks very much for the update!

[MartyG-RealSense]

Hi @oren-hinkis Do you require further assistance with this case, please? Thanks!

[MartyG-RealSense]

Case closed due to no further comments received.