D435 with new d4 will fuse depth and rgb in black pointcloud and dynamic calibr. tool doesn't work #7300
Comments
|
Hi @FPSychotic Could you try using the CustomRW tool (which you have already used to attempt a gold reset) to run the command Intel.Realsense.CustomRW.exe-r in order to check the camera's calibration data and post the log results in this discussion please? |
|
It sent me the same error than calibration tool I think PS C:\Users\fpsyc> Intel.Realsense.CustomRW.exe -r Device PID: 0B07 Error (9901): calibration tables on device are not supported, Please try again with latest Dynamic Calibrator software. but there is a option on Realsense viewer that allow e get in a .json file calibration data, no idea it is what you need thanks by the help, I'm struggle with this and I cannot afford other camera right now |
|
You are very welcome. An Intel guide-page about this error message suggests updating to the latest version of the Dynamic Calibrator if you have not done so already. If the Calibrator tool does not work after the update, follow this procedure:
|
|
Hi as I wrote Intel.Realsense.CustomRW.exe -r, send the same error than the last version of calibration tool. We are at the same point than the first message |
|
|
|
I got a Firmware backup from january, but the FW backup fail to load from realsense viewer in W10 and Ubuntu 18.04. In W10 the th FW app with .exe wont run, the commands from the tutorial doesn't work. In ubuntu the same coommands works, but it fails too, with this error. search for device with serial number: 832112073321 updating device: failed to locate a device in FW update mode I have a a FW backup from when the camera was working with I guess it contents the original factory calibration, but non Intel tool works, I have all, even a clean w10 ad ubuntu installation, my PC is the refence one, a intel nuc i7 with intel gpu. |
|
The calibration is written to an EEPROM flash memory storage chip inside the camera and not the firmware driver, so using an old firmware file would not restore the original factory calibation. The EEPROM that the calibration data is written to is on the camera depth module board (I confirmed this with the 400 Series data sheet document). The data sheet says that the firmware driver is stored in flash memory on the Vision Processor D4 board. When you say "D4", may I confirm this means that you changed the Vision Processor D4 board please, and not the D430 camera module board with the imaging sensors on it? Looking back at an earlier log that you provided, the camera was being correctly identified as a D435 after your board exchange, suggesting that the camera module board and the firmware are both okay. Re-reading your original comment, it sounds as though the depth, IR and RGB work on their own but when mapping RGB onto depth in 3D point cloud mode you get a black cloud. Is that correct, please? Have you tried a Targetless type calibration with the Dynamic Calibrator tool yet? This method does not calibrate RGB, just depth. I have seen it suggested though that if there is a problem with the calibration tables then one can try a targetless calibration first and then follow it up with a targeted calibration. |
Yes, I changed the Vision processor D4 module board
yes, as it shown in the picture
Not as the command Intel.Realsense.DynamicCalibrator.exe -m 0 that is in the guide for targer-less do nothing, doesn't work. v2.11
Please can you check test if they works (maybe they are broken by the last firmware),Is there some way to check if my eeprom works properly? |
|
The cased USB cameras are not meant to be taken apart, so disassembly may constitute a sales warranty invalidation. There is a debugging function in the SDK called RS2_CAMERA_INFO_CAMERA_LOCKED that reports True if the eeprom is locked. You would have to write a script to generate a readout of the status of the eeprom though. It may be worth going to the Settings interface in the Viewer (left-click the gearwheel icon in the top corner), going to the General tab of the Settings window and setting the Viewer to default settings to eliminate the possibility that the black cloud is not caused by an errant setting. |
|
I'm not asking by any camera warranty, I'm asking to make work the tools available for the users of a product, which doesn't work, or you still didn't show they works.Such tools are needed to get the product fit to propose, the hardware that effectively has warranty, the D4 video processor need that tools working, if not you sold me only a PCB, I didn't pay a PCB. Such tools are in the price that I paid with the module, and they are necessary to make it work. that the tools don't work have nothing in common with my camera manipulation. |
|
the black point cloud is visible in 3 different realsense_viewer installations. Dynamic Calibration tool with pattern gives error (9901) in gui mode in command mode -r and -f wont work, so wont read the table or save it in a file. calibration target-less command do nothing in windows 10, doesn't work, same errors. Just tested in linux, doesn't work |
|
Target-less doesn't work in linux |
|
You can query sensors and their information individually with the sensor-control sample program. https://github.com/IntelRealSense/librealsense/tree/master/examples/sensor-control You can also try to retrieve calibration information with rs-enumate-devices using the command below: rs-enumerate-devices -c https://github.com/IntelRealSense/librealsense/tree/master/tools/enumerate-devices |
** what should I check? dynamic calibration appear as not supported** , just I noted IR streamings won't be shown and the check box option is not in realsense viewer, in ubuntu I can see the IR streams and watch them **
``ubuntu@ubuntu:~$ rs-enumerate-devices Stream Profiles supported by Stereo Module Stream Profiles supported by RGB Camera ubuntu@ubuntu: |
|
On the rs-enumerate-devices attempt, it looks as though that instruction was entered without -c on the end to display the calibration information. Could you try again with rs-enumerate-devices -c please and post here the log of what it says. I tested sensor-control, and the options flow I would recommend using is: Select '0' for the device by index.
Scroll down through the list and find the number for a Depth stream on the list. I chose profile 80 which is 848x480 at 60 Hz on my computer). Type in the number of the profile to check.
When asked to choose a sensor that will be used as the target of extrinsic transformation, select '0' for Stereo Module.
You are asked again to enter a number for a streaming profile. Enter the same number as the last step (e.g 80). The extrinsics for that particular stream mode (848x480 at 60 FPS in my computer's case).
The point of this exercise is to check whether the sensor-control tool can read the calibration details of your camera, whereas the Dynamic Calibrator tool refused to. |
same result than you in control streams color stream do no appear in the options, it is weird for me, maybe is normal **I cannot get intrinsics of rgb either ** |
|
Thank you very much for your tests. It is normal for all of the intrinsic and extrinsic Coefficients to have a value of '0' on the 400 Series cameras, so I do not see a problem there. It does not seem as though the camera is having problems accessing intrinsics and extrinsics. I expected this, since the RGB and depth individually does not seem to have a problem, only when mapping RGB to a point cloud. I just wanted to confirm whether the calibration data could be displayed. Thank you for that. It could be that color sensor details are not listed because the selected device is Stereo Module (0) and the RGB details are under sensor option RGB Camera (1). |
thank to you by the patience and answer me the weekend. does mean a lot i found that the streaming color now appear but it is in black, I think that is the issueWhat would you like to do with the sensor? 0 : Control sensor's options Select an action: 1 ====================================================== Sensor consists of 1 streams:
Please select the desired streaming profile: 61 Streaming profile: Color. Close display window to continue...
|
|
Now is working and I can see the color stream. In ROS it is not publishing aligned depth to color and optical color frames, that have quite sense for me why maybe in Realsense viewer is not able to align color with point cloud, is there some way to test the hardcoded transforms into d435 or if the problem is it cannot align by some reason? maybe it is lost the relation between rgb and stereo sensor, not the calibration. Could be the reason to cannot use CustomRW and Dynamic Calibration tools? |
|
Thank you. The point cloud is not black any more? |
|
yes the pointcloud is black and sometimes white
|
|
the tests are ok, but point cloud is black, and I still cannot use the customWR and dynamic calibration tools, just like start ed or worse because now I know it doesn't work in ROS neither. |
|
I will seek further advice about your point cloud problem and get back to you. |
thanks very much, I understand how so weird is this issue. |
|
Hi again, I have received guidance from Intel about your problem. On the D435, the RGB calibration table is stored on the ASIC board, so if the ASIC board stops working then it is hard to retrieve that information through the USB connection. However, if you have performed a firmware update before, you should have backup firmware stored in your computer at a folder location that is something like: C:\Users<YourUserName>\AppData\Roaming\ 012345678901.20200914_143743.bin If you have performed Dynamic Calibration with the camera before on your computer then there may also be a chance to retrieve the RGB calibration table. Usually under a folder like “C:\CalibrationToolAPI\2.6.8.0\bin\DyCalibResult\802312060126\DC2”, there is a file called “RgbCalibrationTable.xml”. This could be loaded into the camera that has the new D4 board to recover the old RGB calibration table. The Intel.Realsense.CustomRW.exe command can help to load the above XML file. It should be located under a folder like “C:\Program Files\Intel\CalibrationToolAPI\2.6.8.0\bin” |
**ASIC board is the D4 processor, am I right?, when I removed it because the camera didn't boot, I lost the calibration of rgb as the rgb camera is a extra over the module d430 (which keep the calibration data in the camera module). D4 image processor has not as target as spare of d435, it has as target d430 which has calibrated data in eeprom. As D4 has not calibration from factory, I need intrude it at hand, and the pointcloud is black because the software cannot get a calibration from d4 board. I understood right the issue? **
yes, I have an backup with I couln't upload with the recovery tool.I will try again upload it, but the problem is software dont work, reason I openned this case. dynamic cal. tull doesnt work, customRW doesnt work and realsense viewer works but fails to upoad the backup to the camera. As alternative maybe I can extract the .bin firm file and if it content a .json or .xml with the rgb calibration data I can ingress it manually on realsense viewer, with the tool data calibration table, write, it is under calibration on chip option.
I never used windows until this moment to try fix the issue. I wil look by such file in ubuntu, but probaby is not
the tool you refer will not work in windows or ubuntu in my case, this days is 2.11 version,, there is not version 2.06 never more, I tried find 2.06 to try, but I couldnt find it. Maybe it doesnt work because there is not calibration in ASIC, and that is the problem, maybe If I give a file with calibration, it works. After this I will not ask you more and quit you more time, as it is not problem of 4D and I understand the situation. the reason to swap the d4 processor and do not use warranty was the camera was send to me from Intel, and is Intel who have the invoice and who bought the camera. Just recognize your patience and help. I'm sorry if I was a little frustated by the situation of any tool works, but Inow understand that probaby is because the tools are designed for a d435 with calibration data, which is not my case. |
|
Thanks for the detailed update. D4 is the ASIC, yes. I do not have the tools to extract the contents of the firmware backup file that you supplied. I will refer the matter back to Intel for further advice about your file. |
|
Hi again @FPSychotic Intel gave your problem further attention and provided a response. A RealSense team member was able to extract the calibration table from the backup firmware file that you provided. I have attached the file - 832513024585.20200124_232513.bin - as a zipped file at the bottom of this message. Please try copying it into your new ASIC board using this command: Intel.Realsense.CustomRW.exe -r -f 832513024585_CAL_table.xml Let us know what happens, please. The RealSense team member who extracted the backup firmware said that it is likely firmware version 5.12.1 rather than the 5.10.06 quoted earlier in this discussion, as 5.12.1 was shown when they applied it to their camera. |
Hi, thanks by the effort, I copied the file into my user folder root and used the command above, which is read, I think should be -w of write, anyway I used both and same error of always
** using documents directory thinking in avoid permissions issue with -r and -w **
probably is right I don't remember why I thought it was that version, maybe a wrong search in the logs or intel firm website |
|
After try in Ubuntu, same error
|
|
Thanks for the detailed report @FPSychotic I will pass it on to the RealSense team member who handled the calibration extraction. |
|
Hi again @FPSychotic After discussions with Intel, it was concluded that there is nothing else that can be done, unfortunately. Do you have any further questions please? |
|
So calibration tools wont work for me, so finally what is the reason? I cannot calibrate the color camera with the stardar tools. Intel sell D4 ASIC module to work with D430. Can be the D430 calibrated with that standard tools? If I remove the RGB camera, so know I will have a 430, will work the tools? So I remove the rgb camera, I calatibrate it as a d430 or load my previous calibration backup, now I have a stereo camera calibrated for slam, I use the color stream for other uses, as AI, computer vision, video streaming... |
|
A complicating factor in this case is that the new ASIC is not OEM-calibrated, so it is not pairing well with the D430 board inside the D435 that did undergo factory calibration. In theory, if the RGB sensor in the D435 has its cable disconnected then the firmware should identify the camera as being D430 instead of D435. I'm not sure that it is going to change anything but now that you have come this far, you may as well try. |
|
Well I will try and I will tell you the results, maybe helps others |
|
Hi @FPSychotic Do you have an update for us please about your tests? Thanks! |
|
Case closed due to no further comments received. |
|
I'm having exactly the same problem, that the D435 camera can receive both RGB and Stereo streams, but fail to fuse depth and RGB together. I've tried to execute Intel.Realsense.CustomRW.exe -r , and I also received the error: So, after close to four years of effort, the issue remains unsolved? Or do you @MartyG-RealSense have any new approaches to solving the problem? |
|
Hi @qqice If you have access to the RealSense Viewer tool then you could try resetting your camera to its factory-new default calibration using the instructions at #10182 (comment) |
It doesn't work. After I successfully reset the camera to factory-new calibration, the fuse is still in a mess. I think maybe I need access to something like "RotationLeftRGB" to get it fixed. Do you have further information about how to do it? |
|
@qqice Which version of the RealSense SDK / RealSense Viewer are you using, please? If you have firmware 5.16.0.1 installed in your camera then the SDK / Viewer version that should be used with it is 2.55.1. If you are using 2.55.1 and are also using the RealSense Viewer, its 2D mode does not have depth-color alignment. Depth and color can be mapped together though in its 3D pointcloud mode (accessible by clicking on '3D' in the top corner of the Viewer window) by enabling depth first and RGB second. Could you use the instructions at #8110 (comment) to check whether you have a pair of GLSL options called 'Rendering' and 'Processing' under the 'Performance' section of the Settings interface of the RealSense Viewer, please. If you do have these options and they are both enabled - shown as a blue box beside them - then please try disabling Rendering and Processing to see whether depth and RGB can then be successfully mapped together in the Viewer's 3D mode. If RGB is not mapping to depth in 3D mode then an alternative mode of texturing the pointcloud is to enable Depth and Infrared in the 'Stereo Module' section of the side-panel instead of Depth and RGB. Then go to the 'Texture' option at the top of the Viewer window and select 'Infrared 1' from the drop-down to texture the cloud with a black and white RGB-like infrared image instead of color RGB. @FPSychotic was able to texture depth and infrared in the pointcloud mode like this, but depth and RGB resulted in a black pointcloud.
|
Yes, I'm using realsense-viewer with version 2.55.1, and firmware 5.16.0.1. I've learned that depth and color can be mapped together though in its 3D pointcloud mode, but my camera couldn't map them together, only left me something like this:
I've tried to turn on and turn off both of them, but nothing happened.
Yes. I am going through the same situation as @FPSychotic. Depth and infrared fused correctly, but depth and RGB didn't. |
|
@qqice Are you able to test alignment with the RealSense SDK's rs-align depth-color alignment example program to see whether alignment also has a problem there too? If you installed the SDK with tools and examples included then a pre-built executable version of the rs-align program should be in the usr/local/bin folder of Linux. |
Sure. I will post the test pictures below: |
|
@qqice Thank you for your rs-align images. They definitely do not look normal for what would be expected from that program's output. You tried the -r mode of CustomRW to read the camera's calibration values. Have you also tried the -g (Gold Reset) flag that @FPSychotic tried during their own case? Similar to the calibration reset in the Viewer, the Gold Reset function attempts to recover the camera's calibration table.
|
Yes. It does say "Calibration on device successfully reset to default gold factory settings." However, things haven't changed at all. I also tried -w mode, and the error is the same as -r mode. |
|
@qqice Please go back to the 3D mode of the Viewer with both depth and RGB disabled. Then enable depth first and RGB second and see whether the two can align correctly with this method. Enabling in this order is important because if RGB is enabled first and depth second then it can cause the RGB image to disappear and only show the depth image. If this method does not work then it is unfortunately likely that there is nothing further that can be done. |
It still doesn't make my camera work, but thanks for the advice and help anyway! |
| Camera Model | {D435 |
| Firmware Version | 05.12.07.100 but tried the 3 last |
| Operating System & Version | doesnt work in w10,ubuntu 1804,ubuntu arm64 or Android |
| Kernel Version (Linux Only) | (4.16,4.19 or 5,02) |
| Platform | PC/Raspberry Pi 4/ NVIDIA Jetson nx/
| SDK Version | { 2} |
| Language | ros or sdk |
| Segment | } |
Issue Description
My Realsense d435 didn't boot and I bought a new D4, swaped and booted, but now with the new one when it fuses depth with rgb the point cloud is black, will work properly with IR. I thought it is due to calibration, as it got a new d4 module, it flashed ok from the factory one to last 2 FW versions, but same problem.It is recognized as d435 properly and via usb3.2 and I installed the last version of dynamic calibration tool, 2.11 I think, but send error (9901) , it say run Intel.Realsense.CustomRW.exe -g to get factory valors, I do and it say valors restored, after I try again the calibration with same error. I tested different arm boards, PC, linux, w10...
The text was updated successfully, but these errors were encountered: