Calibration Failed. OCC ALL calibration cannot work with this camera!

[ravijo]

Dear RealSense Team,

I am using Intel RealSense Viewer v2.44.0 application to do On-Chip Calibration (OCC) on a D415 camera.

---

Required Info
Camera Model	D415
Firmware Version	05.12.12.100
Operating System & Version	Ubuntu 18.04.6 LTS
Kernel Version (Linux Only)	5.4.0-86-generic
Platform	PC
SDK Version	2.44.0-0~realsense0.4389
Language	None
Segment	Robot

Issue Description

While doing OCC, the Intel RealSense Viewer shows following error:

Calibration Failed. OCC ALL calibration cannot work with this camera!

The following error was reported in the dialogue box. This dialog box appears after clicking on the "expand" label while OCC is going on.

Started On-Chip Calibration process
Starting one button calibration
Calibration failed with exception

ERROR: UNKNOWN in rs2_run_on_chip_calibration_cpp(device:0x55a4f7f4d2a0):
Operation timed-out!
Calibration state did not converged in time
Started On-Chip Calibration process
Starting one button calibration
Calibration failed with exception

ERROR: UNKNOWN in rs2_run_on_chip_calibration_cpp(device:0x55a4f7f4d2a0):
Operation timed-out!
Calibration state did not converged in time

I found following doubtful lines in the librealsense.log file:

 10/12 19:32:55,050 DEBUG [140073683199872] (archive.cpp:321) CallbackStarted,Depth,1373,DispatchedAt,1639132375050.225586
 10/12 19:32:55,060 DEBUG [140071785174784] (hw-monitor.cpp:164) hwmon command 0x80( 14 0 0 0 ) failed (response -10= Integrity error)
 10/12 19:32:55,060 DEBUG [140071785174784] (types.cpp:78) recoverable_exception: hwmon command 0x80( 14 0 0 0 ) failed (response -10= Integrity error)
 10/12 19:32:55,060 WARNING [140071785174784] (ds5-auto-calibration.cpp:509) hwmon command 0x80( 14 0 0 0 ) failed (response -10= Integrity error)
 10/12 19:32:55,060 DEBUG [140071650969344] (global_timestamp_reader.cpp:123) calc_value: 73730.139000 -> 1639132375055.111816 with coefs:0.999840, -3.606423, 58382.352000, 1639132359713.386963
 10/12 19:32:55,060 DEBUG [140071650969344] (sensor.cpp:355) FrameAccepted,Depth,Counter,1375,Index,0,BackEndTS,1639132375059.795898,SystemTime,1639132375060.235840 ,diff_ts[Sys-BE],0.439941,TS,1639132375055.111816,TS_Domain,Global Time,last_frame_number,1374,last_timestamp,1639132375044.006348
 10/12 19:32:55,060 DEBUG [140071650969344] (archive.cpp:321) CallbackStarted,Depth,1375,DispatchedAt,1639132375060.286133
 10/12 19:32:55,060 DEBUG [140071650969344] (archive.cpp:321) CallbackStarted,Depth,1375,DispatchedAt,1639132375060.293701

Please see the attached screenshot for better understanding.

I tried following things but no success:

• Switching fast/slow/white wall procedure
• Placed camera 30 cm, 60cm, and 1m far from a white wall
• Used another D415 camera
• Used another USB3 cable
• Restarted the realsense-viewer
• Restarted the PC as well

I tried various things as listed above but the error "Calibration Failed. OCC ALL calibration cannot work with this camera!" still appears.

[MartyG-RealSense]

Hi @ravijo I conducted extensive tests with On-Chip Calibration on two different laptop computers with the same D415 camera. I found that the level of success that was achieved depended on which computer was used, with one PC producing log warnings but otherwise carrying out the calibration normally, and the other PC failing to complete calibration every time.

Could you check please whether you have Infrared stream options available in the Stereo Module section of the Viewer side-panel when using the D415, please? I have observed that On-Chip calibration may not perform normally if there is a glitch that causes no Infrared options to be shown, or only the Infrared option but not Infrared 2.

You could also try calibrating using Intel's Dynamic Calibration RealSense software tool instead of On-Chip calibration. Page 14 onwards of the tool's user guide at the link below provides instructions about Linux installation.

https://www.intel.com/content/www/us/en/support/articles/000026723/emerging-technologies/intel-realsense-technology.html

[ravijo]

Hello @MartyG-RealSense

Thank you very much for quick response.

Could you check please whether you have Infrared stream options available in the Stereo Module section of the Viewer side-panel when using the D415, please?

Do you mean that I need to enable Infrared for OCC procedure? At present, I don't have access to the camera. So I can't check it. Can you please wait for sometime?

I have observed that On-Chip calibration may not perform normally if there is a glitch that causes no Infrared options to be shown, or only the Infrared option but not Infrared 2.

I am not sure but I think I haven't seen "Infrared 2" option inside RealSense Viewer so far. I can find an old screenshot of the RealSense Viewer, in which I used high accuracy preset. Please it it attached below:

Later on, in this setting, I changed the color resolution to 1920x1080 @ 6 FPS.

You could also try calibrating using Intel's Dynamic Calibration RealSense software tool instead of On-Chip calibration.

I was planning to use Dynamic Calibration at the first place. But the documentation says "Dynamic Calibration is optimizing extrinsic parameters, ... ... ... Intrinsic parameters, such as distortion, field of view, principal point, are not dynamically calibrated."

At present, the distortion has 0 values. I feel that using a few 1 or 2 distortion coefficient may improve the overall performance. This is the reason, why I am looking for calibration of intrinsic parameters and given Dynamic Calibration tool may not be helpful.

[MartyG-RealSense]

Normally you do not need to touch the stream enable / disable controls in order to perform On-Chip calibration. The suggestion to check for the availability of the Infrared stream controls is to confirm whether there is a problem with the stream controls on a particular PC that might also affect the calibration procedure. For example, on a PC that I have with missing Infrared controls when the camera is used with that particular computer, the On-Chip calibrator does not offer the ability to select the type of calibration (fast, slow, white wall, etc) and only has a single calibration start button on its interface.

In regard to your kindly provded image, the Infrared on-off controls will only be shown in the side-panel whilst the Stereo Module is disabled (not currently streaming). So that image does not provide information about whether the Infrared streams are available for selection, unfortunately. The Infrared 2 option will also only be available if the camera is on a USB 3 connection, though the '3.2. value beside the camera name in the above image shows that the camera is indeed being used in a USB 3 port and so should have both the Infrared and Infrared 2 stream options available.

It is no problem at all to wait for a period of time until you can check the camera.

It should not be a problem that the Dynamic Calibration tool does not calibrate intrinsics, as it is the extrinsics that will have the most significant effect on the output image. 99% of RealSense users will likely not need to calibrate the intrinsics.

Installing the Dynamic Calibration tool also installs an additional tool called CustomRW. This tool can be used to perform a 'gold reset' of the camera to reset it to its factory-new defaults. If there is a corruption of the 'calibration table' stored inside the camera hardware that may prevent successful calibration then a gold reset can correct the table.

Intel's official online RealSense Store does offer a professional OEM version of the tool with their $1500 USD 'OEM Calibration Target' system, which is aimed at engineering departments and manufacturing facilities. The OEM Dynamic Calibration tool can calibrate both intrinsics and extrinsics. Again though, you should not require intrinsic calibration to obtain a healthy and robust calibration with Dynamic Calibration.

https://store.intelrealsense.com/buy-intel-realsense-d400-cameras-calibration-target.html

It is normal for 400 Series cameras to mostly have a '0' value for their distortion coefficients. It is by intentional design, as described by a RealSense team member at #1430

[ravijo]

Hello @MartyG-RealSense

For example, on a PC that I have with missing Infrared controls when the camera is used with that particular computer, the On-Chip calibrator does not offer the ability to select the type of calibration (fast, slow, white wall, etc) and only has a single calibration start button on its interface.

I confirm that I could see the type of calibration (fast, slow, white wall, etc) and other details during On-Chip Calibration (OCC).

The Infrared 2 option will also only be available if the camera is on a USB 3 connection, though the '3.2. value beside the camera name in the above image shows that the camera is indeed being used in a USB 3 port and so should have both the Infrared and Infrared 2 stream options available.

Ok. I will check the RealSense Viewer on Monday to confirm if Infrared 2 stream is available inside the application. However, I do remember seeing Infrared 2 stream inside ROS Package (but it was disabled by default). Therefore, maybe there is Infrared 2 stream inside the RealSense Viewer too. Please wait for the confirmation.

It should not be a problem that the Dynamic Calibration tool does not calibrate intrinsics, as it is the extrinsics that will have the most significant effect on the output image. 99% of RealSense users will likely not need to calibrate the intrinsics.

Thank you very much. As per your suggestion, I will try Dynamic Calibration tool as well.

[MartyG-RealSense]

Thanks very much for the update. I look forward to your next report. Good luck!

[ravijo]

I confirmed that both Infrared streams are visible in RealSense Viewer. Please see the screenshot below:

However, I still feel that the OCC procedure is highly unpredictable. This time, I used a whiteboard (commonly used in schools for writing/teaching purposes) and found that the rate of failure of OCC is lower than using a white wall. It doesn't mean that OCC always works. It simply means that using a whiteboard increases the chances of success of OCC. While OCC was going on, I noticed the terminal was flooded with neverending error messages, as shown below:

 13/12 12:12:11,742 ERROR [139870169200384] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: Out of frame resources!
 13/12 12:12:11,776 ERROR [139870169200384] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: Out of frame resources!
 13/12 12:12:11,810 ERROR [139870169200384] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: Out of frame resources!
 13/12 12:12:14,807 WARNING [139872103855872] (backend-hid.cpp:1203) Failed to read busnum/devnum. Custom HID Device Path: /sys/bus/platform/drivers/hid_sensor_custom/HID-SENSOR-2000e1.3.auto
 13/12 12:12:14,807 WARNING [139872103855872] (backend-hid.cpp:1203) Failed to read busnum/devnum. Custom HID Device Path: /sys/bus/platform/drivers/hid_sensor_custom/HID-SENSOR-2000e1.2.auto
 13/12 12:12:16,400 WARNING [139869984638720] (ds5-auto-calibration.cpp:509) hwmon command 0x80( 14 0 0 0 ) failed (response -10= Integrity error)
 13/12 12:12:16,600 WARNING [139869984638720] (ds5-auto-calibration.cpp:509) hwmon command 0x80( 14 0 0 0 ) failed (response -10= Integrity error)
 13/12 12:12:16,800 WARNING [139869984638720] (ds5-auto-calibration.cpp:509) hwmon command 0x80( 14 0 0 0 ) failed (response -10= Integrity error)
 13/12 12:12:17,001 WARNING [139869984638720] (ds5-auto-calibration.cpp:509) hwmon command 0x80( 14 0 0 0 ) failed (response -10= Integrity error)

Having a whiteboard everywhere is beyond expectorations and even with a whiteboard, OCC doesn't work 100%. Any idea to make the OCC more robust, please?

[MartyG-RealSense]

If the Dynamic Calibration tool is used then you can use its target-less calibration mode to calibrate the camera by pointing it at a surface that has texture and moving the camera around. Details of this mode are on pages 48-49 of the user guide for the tool.

https://www.intel.com/content/www/us/en/support/articles/000026723/emerging-technologies/intel-realsense-technology.html

In regard to the On-Chip tool, having a good level of lighting in the area that the calibration is being performed in is important
The On-Chip white paper guide also highly recommends setting the Preset of the camera to High Accuracy when using the White Wall setting with the D415.

The guide also recommends having the camera's projector turned On when using White Wall mode with an untextured surface such as your whiteboard but turned Off when using other modes such as fast and slow with a textured surface. The projector status is set with the Emitter Enabled option of the Viewer. Off is set with the Emitter Enabled option's 'Off' menu selection and On is set with 'Laser'.

[MartyG-RealSense]

Hi @ravijo Do you require further assistance with this case, please? Thanks!

[ravijo]

I tried to do OCC, on 3 different surfaces, but all failed.

1. Camera is tiled and looking towards the floor (intentionally chosen a floor which has lots of texture)

2. Camera is looking towards a smooth cement column.

3. Camera is looking towards a smooth wooden texture wall.

Looking forward to hearing from you.

[MartyG-RealSense]

Let's try instead restoring the original factory-new calibration of the camera (as 400 Series cameras are calibrated in the factory during manufacturing) on the assumption that the original calibration will be the best one.

1. Go to the 'More' option in the Viewer and select the Calibration Data option from its drop-down menu to pop up the

2. A 'Camera Calibration' window will pop up that lists the current calibration values of the camera. Left-click on the black box beside the I know what I'm doing option in the corner of the pop-up window to toggle the box color from black (Off) to blue (On). This will unlock a greyed-out unselectable button called Restore Factory.

3. Left-click on the 'Restore Factory' button to reset the camera's calibration to its original factory-default values contained in the flash storage hardware inside the camera. You may see a blue flash as the values are reset - this is totally normal.

[ravijo]

@MartyG-RealSense

Thank you very much for the suggestion.

I am aware of restoring factory calibration parameters. Thank you for telling me again.

The reason, why I am asking you to look at OCC done by me on the floor, cement column, and the wooden wall is, that because it is happening with more than 1 camera.

Doesn't it feel like OCC is not robust enough? I as an end-user simply want to do it with confidence. But I can't!

[MartyG-RealSense]

On-Chip calibration provides a means of rapidly checking the health of the camera calibration and writing an updated calibration to the camera hardware. The Dynamic Calibration tool will provide a robust calibration of the depth sensors and also calibrate the RGB sensor if one is present on that particular RealSense model.

[ravijo]

The Dynamic Calibration tool will provide a robust calibration of the depth sensors and also calibrate the RGB sensor if one is present on that particular RealSense model.

Quoting the information from a previous post, #10075 (comment)

• Dynamic Calibration tool (for extrinsic calibration)
• On-Chip Calibration (for intrinsic calibration)

Therefore, the purpose of both tools is different and both of them should be having equal importance.

[MartyG-RealSense]

The Dynamic Calibration tool was the original calibration tool for the 400 Series cameras when they launched at the end of 2017. The free public version of the tool only calibrates extrinsics, as they have the most impact on the output image. There is an OEM version of the tool that calibrates both intrinsics and extrinsics that is packaged with Intel's $1500 USD OEM Calibration Target system, but 99% of RealSense users will not require this system.

The On-Chip calibration tool was launched later. It was decided by Intel to keep both the On-Chip and Dynamic Calibration tools available rather than retire the older Dynamic Calibration tool. The On-Chip calibration's interface provides the option of calibrating either intrinsics or extrinsics (using 'Intrinsic' and 'Extrinsic' boxes that can be toggled between black and blue), but not both at the same time.

[ravijo]

Thanks for the interesting information.

I was able to use the Dynamic Calibration tool. 💯 Thanks for your kind support ㊗️

However, in this post, I am trying to be confident in performing OCC. For example, in my post above (#10044 (comment)), You can see that I did OCC on 3 different surfaces, but all failed.

Can you please look at it once more?

[MartyG-RealSense]

I ran another series of tests but the results did not change significantly. I also additionally shone a portable light source onto the wall and tested with and without the light and it seemed to make no difference to reliability whether or not a light source was cast onto the wall.

I most consistently achieved a 'Good' calibration result on the wall with the 'Extrinsics' option selected and the 'Slow' calibration type.

[ravijo]

I ran another series of tests but the results did not change significantly.

Can you please share the following info:

• Firmware Version
• SDK Version

I did OCC on 3 different surfaces, (#10044 (comment)) but every time Intel RealSense application showed "Calibration Failed" Error dialogue as shown in the picture below:

During the OCC process, I noticed the terminal was flooded with neverending error messages, as shown below:

 13/12 12:12:11,742 ERROR [139870169200384] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: Out of frame resources!
 13/12 12:12:11,776 ERROR [139870169200384] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: Out of frame resources!
 13/12 12:12:11,810 ERROR [139870169200384] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: Out of frame resources!
 13/12 12:12:14,807 WARNING [139872103855872] (backend-hid.cpp:1203) Failed to read busnum/devnum. Custom HID Device Path: /sys/bus/platform/drivers/hid_sensor_custom/HID-SENSOR-2000e1.3.auto
 13/12 12:12:14,807 WARNING [139872103855872] (backend-hid.cpp:1203) Failed to read busnum/devnum. Custom HID Device Path: /sys/bus/platform/drivers/hid_sensor_custom/HID-SENSOR-2000e1.2.auto
 13/12 12:12:16,400 WARNING [139869984638720] (ds5-auto-calibration.cpp:509) hwmon command 0x80( 14 0 0 0 ) failed (response -10= Integrity error)
 13/12 12:12:16,600 WARNING [139869984638720] (ds5-auto-calibration.cpp:509) hwmon command 0x80( 14 0 0 0 ) failed (response -10= Integrity error)
 13/12 12:12:16,800 WARNING [139869984638720] (ds5-auto-calibration.cpp:509) hwmon command 0x80( 14 0 0 0 ) failed (response -10= Integrity error)
 13/12 12:12:17,001 WARNING [139869984638720] (ds5-auto-calibration.cpp:509) hwmon command 0x80( 14 0 0 0 ) failed (response -10= Integrity error)

[MartyG-RealSense]

I similarly had a cascade of warning messages in the log during testing. As a general rule, the greater the number of orange messages instead of red, the better the chance was of achieving a 'Good' result.

The tests were run in SDK 2.49.0 with a D415 that had firmware version 5.12.14.50

[ravijo]

The tests were run in SDK 2.49.0 with a D415 that had firmware version 5.12.14.50

It seems that you are using the latest versions of SDK and firmware. I am using Ubuntu 18.04 LTS and can not find any available update.

As a general rule, the greater the number of orange messages instead of red, the better the chance was of achieving a 'Good' result.

Noted it down. Thanks a lot. On the other hand, I tried OCC on 3 different surfaces and all returned Calibration Failed. OCC ALL calibration cannot work with this camera!

[MartyG-RealSense]

At the time of writing this, the latest SDK / firmware versions are SDK 2.50.0 and firmware 5.13.0.50.

https://github.com/IntelRealSense/librealsense/releases
https://dev.intelrealsense.com/docs/firmware-releases

[MartyG-RealSense]

Hi @ravijo Do you require further assistance with this case, please? Thanks!

[ravijo]

@MartyG-RealSense

I gave up on OCC as it doesn't work at my side on multiple cameras. I am instead doing Dynamic Calibration and Custom Calibration!

Thanks

[MartyG-RealSense]

Okay, thanks very much @ravijo for the update. I will therefore close this case.

[Boatsure]

Hi @ravijo, does the Dynamic Calibration tool works normal? I have exactly the same issue when using single D435.

And with the same environment setting, a single D435i OCC successfully.

[ravijo]

@surefyyq

I feel that OCC is not robust enough and I still don't have confidence in doing it. An interesting discussion can be found here #10045

On the other hand, the Dynamic Calibration tool works. However, I had difficulty printing it properly on A4 size paper since they have provided US Letter paper. Please see the following issue for the complete discussion #10074

[Boatsure]

Thanks for your share! @ravijo