Intrinsic Calibration

This page reviews how to intrinsically calibrate a Kinect v1, Kinect v2, or a Mesa SwissRanger. This process will allow OpenPTrack to correct for any inconsistencies that were introduced during the manufacturing process.

Completing this process for each imager before moving on to Camera Network Calibration is highly recommended to produce the best tracking results.

Single Kinect (Intrinsic Calibration)

This section describes how to intrinsically calibrate a single Kinect v1. This information will be used by the OpenPTrack master to determine distortion of this Kinect v1. This should be done to each Kinect v1 in your OpenPTrack network:

Intrinsic camera calibration of a single Kinect can be performed with ROS package camera_calibration.

Calibration requires moving a rigid checkerboard pattern in front of the camera, as suggested by the GUI, until calibration is complete. The checkerboard must be moved and inclined in all directions until all GUI bars become green. Then, intrinsic parameters are computed by pressing the Calibrate button. And by pressing Commit, all parameters are saved to the ~/.ros/camera_info/ directory in the rgb_<serial>.yaml file, where <serial> is the serial number of the camera. This file is loaded whenever the OpenNI camera driver is launched from the computer attached to that camera, in order to use the estimated intrinsic parameters.

N.B.: For some Kinects, the driver cannot read the serial number. Therefore, the file created during the calibration is rgb_00000000000000.yaml.

N.B. 2: Intrinsic Calibration is not yet possible with the Mesa Imager.

Example of usage:

After launching the camera driver

roslaunch detection openni.launch

or

roslaunch detection freenect.launch

depending on which driver your system uses.

in a separate terminal, run:

rosrun camera_calibration cameracalibrator.py --size 6x5 --square 0.080 image:=/camera/rgb/image_color camera:=/camera/rgb 

For the above, 6x5 is the number of interior vertex points of the checkerboard (not the number of squares) in the two dimensions, and 0.080 is the dimension (in meters) of the squares.

For a step-by-step tutorial on performing intrinsic calibration, please refer to this page.

N.B.: In order to avoid singularities in the automatic estimation of checkerboard orientation, use a checkerboard with an even number of squares in one direction and an odd number of squares in the other direction. The more numerous and bigger the squares, the better the calibration.

Move calibration files to the OpenPTrack folder:

After a successful calibration (if the Kinect has a serial number), to move the calibration file to opt_calibration/camera_info/rgb_<serial>.yaml, run:

roslaunch opt_calibration save_camera_info.launch with_serial:="true" serial:=<serial>

If unsuccessful, i.e., the driver does not acquire the serial number, run:

roslaunch opt_calibration save_camera_info.launch with_serial:="false" camera_id:=<camera_id>

This way, the calibration file will be renamed as rgb_<camera_id>.yaml and moved to opt_calibration/camera_info/.

N.B.: <camera_id> must be the ID of the sensor as reported in camera_network.yaml.

People tracking test with one Kinect and without extrinsic calibration:

After intrinsic calibration (above), with a Microsoft Kinect for XBOX360 connnected, run:

roslaunch tracking detection_and_tracking.launch

Appearing will be an RGB image with people detections surrounded by white rectangles and an ROS visualizer displaying a top view of people centroids.

Single Swiss Ranger (Intrinsic Calibration)

This section describes how to intrinsically calibrate a single SwissRanger 4500. This information will be used by the OpenPTrack master to determine distortion of this SwissRanger 4500. This should be done to each SwissRanger 4500 in your OpenPTrack network:

Intrinsic camera calibration of a single Swiss Ranger can be performed with ROS package camera_calibration.

Calibration requires moving a rigid checkerboard pattern in front of the camera, as suggested by the GUI, until calibration is complete. The checkerboard must be moved and inclined in all directions until all GUI bars become green. Then, intrinsic parameters are computed by pressing the Calibrate button. And by pressing Commit, all parameters are saved to a file and loaded whenever the driver is launched, in order to use the estimated intrinsic parameters.

Example of usage:

You can perform the calibration by running:

roslaunch opt_calibration sr4500_calibration.launch device_ip:=192.168.1.31 size:=5x6 square:=0.12 camera_id:=swiss_ranger

For the above:

• 192.168.1.31 is the IP address of the Swiss Ranger.
• 5x6 is the number of interior vertex points of the checkerboard (not the number of squares) in the two dimensions.
• 0.12 is the dimension (in meters) of the squares.
• swiss_ranger is the ID of the sensor as reported in camera_network.yaml.

After a successful calibration, the calibration file created is opt_calibration/camera_info/<camera_id>.yaml.

N.B.: In order to avoid singularities in the automatic estimation of checkerboard orientation, use a checkerboard with an even number of squares in one direction and an odd number of squares in the other direction. The more numerous and bigger the squares, the better the calibration.

Single Kinect v2 (Intrinsic Calibration)

N.B.: The Kinect v2 is currently only supported by the Development branch of OpenPTrack.

This section describes how to intrinsically calibrate a single Kinect v2. This information will be used by the OpenPTrack master to determine distortion of this Kinect v2. This should be done to each Kinect v2 in your OpenPTrack network:

N.B. You may have to execute the following code to run the Kinect v2 commands (you may receive an error when you run this code, but it can be ignored.):

 cd ~/workspace/ros/catkin/devel/lib/kinect2_bridge
 sudo ./kinect2_bridge

1. Create temporary folder for storing calibration images:

 mkdir ~/tmp

2. Launch Kinect v2 Imager:

 roslaunch kinect2_bridge kinect2_bridge.launch

3. Color Image Capture:

 rosrun kinect2_calibration kinect2_calibration record color chess6x5x0.12 color /kinect2_head/mono/image ir /kinect2_head/ir/image interval 1 ~/tmp/

NB: be sure to insert the right checkerboard parameters in this string “chess6x5x0.12” when launching the executable. The first two numbers are the number of horizontal and vertical chess intersections, while the third number is the size of a square in meters. This is valid for all the operations described in this guide.

4. Move The Checkerboard: Move the checkerboard to different positions, angles, and distances from the sensor ensuring that the checkerboard is correctly detected. The checkerboard is being detected correctly when different colored lines overlay the checkerboard's image.

5. End Color Image Capture: Kill the color calibration process. The files are automatically saved to the ~/tmp/ folder.

6. Calibrate Color:

 rosrun kinect2_calibration kinect2_calibration calibrate color chess6x5x0.12 color /kinect2_head/mono/image ir /kinect2_head/ir/image interval 1 ~/tmp/

6. Infrared Image Capture:

 rosrun kinect2_calibration kinect2_calibration record ir chess6x5x0.12 color /kinect2_head/mono/image ir /kinect2_head/ir/image interval 1 ~/tmp/

7. Move The Checkerboard: Move the checkerboard to different positions, angles, and distances from the sensor ensuring that the checkerboard is correctly detected. The checkerboard is being detected correctly when different colored lines overlay the checkerboard's image.

8. End Infrared Image capture: Kill the color calibration process. The files are automatically saved to the ~/tmp/ folder.

9. Calibrate IR:

 rosrun kinect2_calibration kinect2_calibration calibrate ir chess6x5x0.12 color /kinect2_head/mono/image ir /kinect2_head/ir/image interval 1 ~/tmp/

10. Calibrate Poses Between IR and Color:

 rosrun kinect2_calibration kinect2_calibration record sync chess6x5x0.12 color /kinect2_head/mono/image ir /kinect2_head/ir/image interval 1 ~/tmp/

11. Move the Checkerboard: Move the checkerboard at different positions, angles, and distances from the sensor making sure that the checkerboard is correctly detected in both color and IR images, that is in both images the colored lines overlay the checkerboard image. When the checkerboard is detected, images are automatically saved every second to the destination folder (~/tmp).

12. Color-IR Calibration:

 rosrun kinect2_calibration kinect2_calibration calibrate sync chess6x5x0.12 color /kinect2_head/mono/image ir /kinect2_head/ir/image interval 1 ~/tmp/

13. Move The Calibration Files to the Correct Location:

Create the folder that contains Kinect 2 calibration data:

 roscd kinect2_bridge
 mkdir pdata/<serial>

The 'serial' is the serial number of the Kinect 2 sensor. You can read it in the console output after launching the driver:

 roslaunch kinect2_bridge kinect2_bridge.launch

At startup, you should see a line similar to this one:

 [Freenect2Impl] found valid Kinect v2 @4:3 with serial 500258141742

Copy calibration results from the temporary folder to this folder:

 cp ~/tmp/calib_color.yaml ~/tmp/calib_ir.yaml ~/tmp/calib_pose.yaml ~/workspace/ros/catkin/src/iai_kinect2/kinect2_bridge/data/<serial>/

14. Verify the Calibration:

 rviz rivz

Now in Rviz change the Fixed Frame to:

 kinect2_head_rgb_optical_frame

and change the Topic to:

 /kinect2_head/depth_lowres/points

Now the color data and the depth points (the point cloud) should be aligned. If they are not your intrinsic calibration was not successful and it will need to be redone.