Epipolar Geometry and Camera Poses

[luislglp]

Thanks for sharing this impressive project, @jyjblrd

About the camera pose computation, specifically when the fundamental matrix is being calculated*, how many correspondence points are required in this step?
I found this guideline stating that at least 8 (also for the RANSAC option):
https://amroamroamro.github.io/mexopencv/matlab/cv.findFundamentalMat.html

Does that mean at least 8 IR markers to perform the camera poses? If so, any hint on how to arrange them?

*F, _ = cv.findFundamentalMat(camera1_image_points, camera2_image_points, cv.FM_RANSAC, 1, 0.99999)

[jyjblrd]

Thanks @luislglp

To calibrate the camera positions, simply click the Collect points for camera pose calibration button on the web interface, and move a single marker / IR led randomly throughout the capture area. Each frame where the marker can be seen by 2+ cameras will be recorded as a correspondence point.

Counterintuitively, I found that it works better when there are only a couple dozen points captured. Probably due to bad RANSAC parameters or something. More tips on camera pose calibration is in this discussion post: #23 (comment)

[luislglp]

Many thanks for the reply!

[luislglp]

One more question if I may, @jyjblrd. I would say more out of curiosity =)

So as it is the camera pose calibration will only work with a single reference marker, is that right?

Usually mocap systems have a standing calibration kits with some additional markers (https://www.qualisys.com/accessories/calibration-kits/), but I suppose that this would require finding correspondences first and ruling out images that the correspondences are dubious. I am not sure if this would improve the calibration results

[jyjblrd]

Yeah, normally the calibration wand will have 2-3 markers at specific distances so that we can calibrate the camera poses and room scale at the same time.

However, to simplify things I split this up into two operations. Firstly, you calculate the camera poses with a single marker. Then, you need to wave around a wand with two markers spaced 20cm apart (I think – need to double check that distance in the code) to calibrate the room scale.

Finally, for professional systems they use a "L frame" that is placed on the floor to calibrate the floor location and origin. This is also split into two operations in my system. Firstly a marker is rolled around on the floor to calibrate the floor location. Then, a marker is placed at the origin to calibrate the world origin location.

I should add more documentation to this process, as right now they are just a few buttons on the front end without much explanation!