WithLatestFrom Frame Rate

[tarheltur]

Hello,

I'm currently using two The Imaging Source cameras (this model, 238 fps) and am trying my best to synchronize them without I/O cables. I have seen some previous posts like this and this, but they still don't seem to answer my question, or perhaps I'm just not seeing it.

When I use WithLatestFrom to attempt to match the camera frames, the actual (if I can call it) synchronization (checked with filming a timer and frame-by-frame comparison of cameras filming continuous movements) seems to be okay, but no matter what I do (eg. try to adjust the "output" frame rate in VideoWriter or the frame rate in the camera software IC Capture), the output frame rate of these videos is always 159.9 or 160. If the frame rate is set to lower in IC Capture, the output videos will repeat frames to compensate for it, and if it's higher, frames will be skipped. My question is just where is this number coming from?

I have a second set of cameras for another setup (these, 539 fps) in which the constant frame rate is at 528 fps (which is higher than I want and crashes Bonsai unless I have a cropped section, understandably because it's a lot for the computer to handle). I'm just trying my best to understand how this value comes about because it seems to only depend on the maximum capacity/frame rate of the camera, but anything else I do has no effect on this number.

When I use Zip to try and synchronize, the value is the same, and when using CombineLatestFrom, the value is double with the two "synchronized" videos essentially taking turns repeating a frame (which makes sense when reading the documentation of both).

I am asking to understand if I could in any way change this value that seems to be set in stone no matter what because I cannot adjust the video frame rate, and setting the frame rate to lower via the IC Capture software causes issues with frame repeats. Therefore, I'm kind of "stuck" with the frame rate somehow "decided" by WithLatestFrom.

Also, I tried the same thing with VideoCaptureDevice which allows me to set a higher resolution and therefore bigger FOV (up to 1440x1080 while CameraCapture only allows 640x480), but it provides me with set frame rate options under properties which also has no effect on the 159.9 fps I get whenever I use WithLatestFrom. Also, the frame rate seems to be less stable when using a bigger FOV with VideoCaptureDevice which is why I'm using CameraCapture.

I'm very much a beginner with Bonsai and have limited coding experience so please excuse if I overlooked some obvious paths to the answer to my question.

Thanks a lot in advance!

[bruno-f-cruz]

Please add the full workflow file. Thanks!

Also check this discussion and see if it answers any of the questions: https://github.com/orgs/bonsai-rx/discussions/1112