Inter Cam HW Sync prevents from RGB/D sync on each device

[pichu380]

Hi,

I have managed to have perfect time-synchronized flow (all RGB+Depth at 30 FPS) between two cameras with the realsense viewer app. But when I try to do the same without the app GUI, RGB wants always to be the master over depth (29.8 FPS instead of 30 FPS) and only the depth flows of the two cameras seems time-synced, not the rgb flows since the RGB and depth flow of the same device are not synced. I have tried to launch depth sensor before rgb sensor with sensor.start but without success.

Is there any way to force the Depth sensor as the master over RGB?

Note: I am using hardware sync between the two cameras as explained in the multicamera white paper

[RohithKuditipudi]

If the depth flows of the two cameras are synced and the RGB stream is the master one one of the cameras, then wouldn't that RGB stream also be synced to the depth stream on the other (i.e. slave) camera? Seems like the only issue then would be that the slave camera's RGB stream might be out of sync.

[pichu380]

I have checked and the RGB stream is not synced with depth streams whether the camera I look at.

In more details, the INTER-camera sync (one D415 with other one D415) only works for depth streams which are time synced according global timestamps. To have also INTER-camera sync for both RGB streams, the INTRA-camera sync (RGB stream synced with the depth stream of the same D415 camera) has to work for each camera since for INTER-camera sync, it is the depth streams which are connected with the sync trigger clock. So if the depth streams are not master over RGB streams, they do not share the sync trigger clock with RGB.

The problem is that, by default, the RGB stream is the master over depth stream on the same D415 device for INTRA-camera sync. I could bypass this problem in the realsense-viewer GUI by simply launch the depth sensor before the RGB sensor in order to force the depth to be master for INTRA-camera sync. But I did not succeed when trying to reproduce it with my own code (depth_sensor.start() followed by rgb_sensor.start() with AE priority off and same frame rates).

I think it is the same problem as #2637 which reports (see end of issue) good depth synced but wrong rgb synced. Since I could do it in the viewer GUI (https://www.youtube.com/watch?v=DSm7NYG5bZE&feature=youtu.be) I know there is a way to do it but I do not know how. I will try to do hardware reset and put some sleep time as in #2281.

In the post #2637, there is this explanation about INTRA and INTER camera sync if I was not clear above:

"Furthermore, since I'm unable to set any flags on RGB, how does the RGB image sensor get hardware synced?

As far as I know D415 allows Depth+Color sync to another Depth+Color. D435 allows Depth to Color sync, and Depth to another Depth sync, but not Color to another Color, so at least one (or both) of the color sensors will not be synced. Perhaps firmware / hardware people will be able to find a solution in future update, but for now it is a limitation. For the D415, setting option on the depth sensor will also propagate to color (assuming regular checklist for sync is green - AE Priority Off and same FPS)"

Thank you for your help!

[pichu380]

I solved the problem thanks to #2281 by adding some hardware_reset and sleep(3) before to setup the sensors. It seems that without the hardware reset, it is not possible to force the depth stream as master. I have now the same as in the realsense-viewer GUI.