Syncing depth with color on D435 and triggering an external device?

[RohithKuditipudi]

Required Info
Camera Model	{ D400 }
Firmware Version	05.11.06.200
Operating System & Version	{Ubuntu 18.04}
Kernel Version (Linux Only)	4.15
Platform	NVIDIA Nano
SDK Version	{ 2.23 }
Language	{C++ }
Segment	{SLAM }

Issue Description

Hello, I am working on a SLAM project where I would like to sync an external IMU with the D435 camera's color stream. In particular we would like to send the sync pulse from the D435's sync out pin to our IMU's sync in pin. My understanding is that it's not possible to generate this signal from the color stream. However, based on the response below it seems like as long as we run depth and color at the same frame rate and generate a sync out pulse from the depth stream, our IMU will effectively be synced to both the depth and color streams since the color stream would actually be triggering the depth stream on the D435 and therefore implicitly triggering the IMU? Is my understanding of the situation correct? And if not is there any alternative way I can sync an external IMU with the D435's color stream?

See below for the response I was referencing:

How does the Depth to Color sync work in D435? More specifically, what do I need to do to enable it?

When working with a single camera, hardware sync between color and depth kicks-in once they run at the same actual FPS. To ensure it, you can disable Auto Exposure Priority control under RGB Camera > Controls (this way the camera won't reduce FPS in a dark room to get more light)
The way it works, as far as I know, is by color sensor sending sync signal to the depth sensor. This is why it is problematic to set it up in multi-camera scenario. Color sensor always wants to be the master.

Another hardware quirk is that color frames get their timestamp at a different time (mid exposure, I think?), so the timestamps are not perfectly aligned. We try to work around that in syncer class, that is also used inside each pipeline.

Also, if color can't be synced to depth, does that mean I should have a separate pipeline for each depth sensor, which are all synced to each other (one pipeline per thread), and a separate set of pipelines for the color images, which will arrive asynchronously? If there aren't separate pipelines (as now), does that mean the wait_for_frames() will return every time the depth frame arrives (with whatever the last color frame was), and similarly for color (returning whatever the last depth frame to arrive was)?

Yes, if you enable all depth streams (synced) and all color streams, the color streams might introduce some unwanted latency to the depth streams (since every wait for frame will try to match depth frame to a color frame). This should not create delays greater than the frame interval, but if this is unacceptable, you can switch to directly controlling each sensor (see sensor-control example)
The main question is what your application prioritizes? Having some color together with each depth frame or having all depth frames together?

Originally posted by @dorodnic in #2637 (comment)

[ev-mp]

@RohithKuditipudi , the above description is correct and you cannot generate a sync pulse from the RGB stream. Using the multi-cam sync depth pulse in the best-effort option as in certain cases the internal depth->rgb sync will be disabled like, for instance, when the RGB exposure priority is activated and the RGB stream switches to its own pace.

[RohithKuditipudi]

Thanks!