SR305 depth image incorrect on Jetson nano with BUILD_WITH_CUDA=true #6373
Comments
|
Hi @rkent Is CUDA installed on your Nano please? |
|
Yes CUDA is installed. /usr/local/cuda-10.0 I checked paths. I had the PATH correct but not the LD_LIBRARY_PATH. I fixed that, did a clean recompile, but it did not fix the issue or change its nature. |
|
When you installed using the Debian packages, did you use the JetsonHacks Nano '5 minute installation's guide, please? https://www.jetsonhacks.com/2019/12/22/install-realsense-camera-in-5-minutes-jetson-nano/ This method utilises Intel's dedicated install instructions for Jetson: https://github.com/IntelRealSense/librealsense/blob/master/doc/installation_jetson.md Since the Librealsense Debian packages have CUDA support included in them, this method would seem to be the most straightforward way to achieve a CUDA-enabled Librealsense on Nano if you have not followed this method already. |
|
"achieve a CUDA-enabled Librealsense on Nano" is not really my goal. My goal was to use the SR305 somehow on the Nano, which so far I can only achieve by building myself with CUDA disabled. I am reporting my experiences in case it help others. I don't believe I need CUDA support to accomplish my goals. To answer your question directly, I tried all kinds of things for days to try to get the camera to work, including the various Jetson hacks recommendations. But to demonstrate cleanly the issue, I did a clean test of the jetson hacks install. I reimaged an SD card with a fresh image from Nvidia (jetbot_image_v0p4p0.zip). I then followed the directions in "Install RealSense Camera in 5 minutes – Jetson Nano" namely: $ git clone https://github.com/JetsonHacksNano/installLibrealsense I restart, plugin my SR305 (using the supplied USB-3.0 cable), and run realsense-viewer. Same experience as before. RGB camera works fine. For the depth camera, at first you see nothing. Playing around, post-processing must be turned off to see anything, particularly the threshold filter. This is because, by default, it throws away anything with a reported depth less than 0.1 meters. But once I get a picture, I see that the reported depth is low, like .03 meters. Changing the color pattern to a black-to-white pattern, it is clear that I am seeing an intensity image, not a depth image. I can use the same camera on my x86 Ubuntu computer and it works fine, the reported depths make sense and the image from the depth sensor is clearly a depth image and not an intensity image. I am happy to do other tests if it would be helpful to diagnose this better if it would be helpful to others. But for my own purposes, by building librealsense with CUDA disabled I can get the camera to do what I need it to do. |
|
Thanks so much for your detailed testing and reporting! It's very valuable to have guidance for current and future SR305 Nano users. |
|
I spoke too soon saying '"achieve a CUDA-enabled Librealsense on Nano" is not really my goal'. With CUDA disabled, running the SR305 required 100% of two of the four processors. So I investigated further what the issue is. Turns out that whatever the problem is occurs only in a single file (src/proc/depth-formats-converter.cpp) If I disable the use of CUDA just in that file (by replacing #ifdef RS2_USE_CUDA with #ifdef RS2_USE_CUDA_NOT) then depth images work. Now one CPU at 85% is needed to run the camera, so whatever that change did is not really needed for significant CUDA performance improvements. I've implemented that change in my fork https://github.com/rkent/librealsense which is intended for my personal use. |
|
Thanks very much for the update - I'm glad you were able to develop a solution that works for you! |
|
Do you require any further assistance with this case or can it be closed, please? Thanks! |
|
That depends on the intent of your use of issues. I am accustomed to issues being documentation of known bugs needing fixing. If that is the case, then I have done my best to document a bug, and isolate it to a few lines of code. That bug still remains and affects use of your product. By that definition, this case should not closed. As for my person use, I have developed a workaround that works for me. I have no further personal need of assistance. So you decide according to your policies whether this should be closed. |
|
The effort that you have made in documenting your problem really is appreciated. An element for consideration in whether an issue should be officially classified as a Bug to be investigated is whether it can be reproduced by others. As your report is the only one that I know of in relation to depth image integrity on CUDA on SR305, I will close it for now. The information will still be findable by others in a web search and the matter can be revisited in future if others report the same problem. Thanks again! |
|
Reopened due to a similar case of broken depth with SR305 possibly related to CUDA. https://support.intelrealsense.com/hc/en-us/community/posts/360044304633 |
|
Hi @rkent There was also a bug in CUDA support in the new SDK 2.35.2. Advice about fixing it is provided in the link below. |
|
Re-closed due to no further comments received. The CUDA build fix in the comment above should be incorporated withn the next SDK release (probably 2.36.0). |
|
@MartyG-RealSense : I can confirm that forcing the code to run on CPU fixes the issue but I personally would prefer having the computation done on GPU in a parallel fashion instead of the "for" loop done on CPU. I've looked over the CUDA code (src/cuda/cuda-conversions.cu) and it seems to do the same bit manipulation as the "CPU" version of the code - so it's a "voodoo" kind of issue. (Later edit) Actually is not "voodoo" it is possible a difference between how "CPU" handles bit shift overflows vs "GPU". The "GPU" code can simply saturate the value. I'll dig a bit more but I suspect this to be the issue. |
|
Hi @xavirostudio As you are also experiencing the issue in addition to the previous reports, I have re-opened the issue and will bring it to Intel's attention. Thanks very much! |
|
Adding a note to keep this case open for a further time period. |
3 similar comments
|
Adding a note to keep this case open for a further time period. |
|
Adding a note to keep this case open for a further time period. |
|
Adding a note to keep this case open for a further time period. |
|
As there have been no further reports about this issue in the months since the last comment, it will be closed. |
|
@MartyG-RealSense The problem still exists on jetson nano with jetpack 4.5.1, realsense sr305 and librealsense v2.44.0, albeit workaround provided by the author works. I also fully agree, the discussed issue should not be closed until fixed, as it's clearly a bug. |
|
@MartyG-RealSense I agree with @PQLLUX, this is definitely a bug and this issue should be opened. I researched the topic and found a bug that causes this (in a nutshell - the pointer used later for copying is moved in the cpu implementation, in the CUDA implementation it is not). I did a PR that fixes this problem. |
|
Hi @andrusza2 Thanks very much for creating a PR! I recommend creating a new case on the GitHub that mentions it, and including links to this case here and to your PR. Then that new case can be kept open for as long as your PR is active. Thanks! You can create a new case with the green New Issue button on the front page of this forum. https://github.com/IntelRealSense/librealsense/issues |
|
Hi everyone, The PR fix for the issue of incorrect depth on SR300 with CUDA that @andrusza2 kindly submitted was merged into the development branch of the SDK 9 days ago at the time of writing this. So it will likely be included in the next SDK version after the current 2.45.0, since 2.45.0 was released 12 days ago at the time of writing this (before the fix was merged). |
|
Hi everyone, RealSense SDK 2.47.0 is now available and includes the previously mentioned SR300 fix. https://github.com/IntelRealSense/librealsense/releases/tag/v2.47.0
|
Issue Description
I tried installing anSR305 depth camera on a Jetson Nano. I've used both the binary distribution, as well as compiling from source. In both cases, the depth camera returns incorrect results. Looking at the image on realsense-viewer, it appears to be an actual grayscale image instead of a depth field. The reported depth from depth field, when it should be about .3 meters, reports instead something much smaller, like .02 meters. This same camera works fine on an x86 Ubuntu system.
After playing with the API for a couple of days to see if I could debug the problem, I sort of by accident discovered that the problem goes away if I compile with BUILD_WITH_CUDA=false instead of true.
The text was updated successfully, but these errors were encountered: