Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RGB Camera returns No frames received on Jetson Orin #11716

Closed
paritoshwale opened this issue Apr 20, 2023 · 22 comments
Closed

RGB Camera returns No frames received on Jetson Orin #11716

paritoshwale opened this issue Apr 20, 2023 · 22 comments

Comments

@paritoshwale
Copy link

paritoshwale commented Apr 20, 2023


Required Info
Camera Model D455
Firmware Version 05.14.00.00
Operating System & Version Linux (Ubuntu 14/16/17)
Kernel Version (Linux Only) 5.10.65-tegra
Platform NVIDIA Jetson Orin L4T 34.1.1
SDK Version 2.53.1
Language python
Segment others

Issue Description

I installed the Intel RealSense SDK 2.0 using all the methods individually in this on my Jetson Orin, but unfortunately I get the same error, as follows, for all streams except YUYV. I tried playing around with the Resolution and frame rate settings and still got the same error. I also tried hardware reset from the realsense-viewer but of no vain. Only YUYV stream seems to be working, I also have a jetson Nano, and the camera works fine with the Nano, so I guess the camera hardware can be ruled out.

20/04 16:52:05,805 INFO [281472535599360] (sensor.cpp:38) Frame Callback Color #65 overdue. (FPS: 30, max duration: 32.258064 ms) 20/04 16:52:05,837 ERROR [281472535599360] (synthetic-stream.cpp:48) Exception was thrown during callback: _copyI14rs2_extrinsicsEESt10shared_ptrIT_ES3_EUlPS1_E_ 20/04 16:52:05,868 ERROR [281472535599360] (synthetic-stream.cpp:48) Exception was thrown during callback: _copyI14rs2_extrinsicsEESt10shared_ptrIT_ES3_EUlPS1_E_ 20/04 16:52:05,900 ERROR [281472535599360] (synthetic-stream.cpp:48) Exception was thrown during callback: _copyI14rs2_extrinsicsEESt10shared_ptrIT_ES3_EUlPS1_E_ 20/04 16:52:05,932 ERROR [281472535599360] (synthetic-stream.cpp:48) Exception was thrown during callback: _copyI14rs2_extrinsicsEESt10shared_ptrIT_ES3_EUlPS1_E_ 20/04 16:52:06,002 ERROR [281472535599360] (synthetic-stream.cpp:48) Exception was thrown during callback: _copyI14rs2_extrinsicsEESt10shared_ptrIT_ES3_EUlPS1_E_ 20/04 16:52:06,034 ERROR [281472535599360] (synthetic-stream.cpp:48) Exception was thrown during callback: _copyI14rs2_extrinsicsEESt10shared_ptrIT_ES3_EUlPS1_E_ 20/04 16:52:06,066 ERROR [281472535599360] (synthetic-stream.cpp:48) Exception was thrown during callback: _copyI14rs2_extrinsicsEESt10shared_ptrIT_ES3_EUlPS1_E_ 20/04 16:52:06,098 ERROR [281472535599360] (synthetic-stream.cpp:48) Exception was thrown during callback: _copyI14rs2_extrinsicsEESt10shared_ptrIT_ES3_EUlPS1_E_ 20/04 16:52:06,129 ERROR [281472535599360] (synthetic-stream.cpp:48) Exception was thrown during callback: _copyI14rs2_extrinsicsEESt10shared_ptrIT_ES3_EUlPS1_E_ 20/04 16:52:06,199 ERROR [281472535599360] (synthetic-stream.cpp:48) Exception was thrown during callback: _copyI14rs2_extrinsicsEESt10shared_ptrIT_ES3_EUlPS1_E_ 20/04 16:52:06,199 INFO [281472535599360] (sensor.cpp:38) Frame Callback Color #78 overdue. (FPS: 30, max duration: 32.258064 ms) 20/04 16:52:06,232 ERROR [281472535599360] (synthetic-stream.cpp:48) Exception was thrown during callback: _copyI14rs2_extrinsicsEESt10shared_ptrIT_ES3_EUlPS1_E_ 20/04 16:52:06,264 ERROR [281472535599360] (synthetic-stream.cpp:48) Exception was thrown during callback: _copyI14rs2_extrinsicsEESt10shared_ptrIT_ES3_EUlPS1_E_ 20/04 16:52:06,296 ERROR [281472535599360] (synthetic-stream.cpp:48) Exception was thrown during callback: _copyI14rs2_extrinsicsEESt10shared_ptrIT_ES3_EUlPS1_E_ 20/04 16:52:06,327 ERROR [281472535599360] (synthetic-stream.cpp:48) Exception was thrown during callback: _copyI14rs2_extrinsicsEESt10shared_ptrIT_ES3_EUlPS1_E_ 20/04 16:52:06,359 ERROR [281472535599360] (synthetic-stream.cpp:48) Exception was thrown during callback: _copyI14rs2_extrinsicsEESt10shared_ptrIT_ES3_EUlPS1_E_ 20/04 16:52:06,391 ERROR [281472535599360] (synthetic-stream.cpp:48) Exception was thrown during callback: _copyI14rs2_extrinsicsEESt10shared_ptrIT_ES3_EUlPS1_E_ 20/04 16:52:06,423 ERROR [281472535599360] (synthetic-stream.cpp:48) Exception was thrown during callback: _copyI14rs2_extrinsicsEESt10shared_ptrIT_ES3_EUlPS1_E_ 20/04 16:52:06,455 ERROR [281472535599360] (synthetic-stream.cpp:48) Exception was thrown during callback: _copyI14rs2_extrinsicsEESt10shared_ptrIT_ES3_EUlPS1_E_

@MartyG-RealSense
Copy link
Collaborator

Hi @paritoshwale The methods for a successful installation of the librealsense SDK on Orin are currently to (a) build the SDK from source code but not patch the kernel, or (b) to build from source code with the RSUSB backend installation method so that the kernel is bypassed. #11628 has information about this.

@paritoshwale
Copy link
Author

Hi @MartyG-RealSense, thanks for your quick reply.
I followed the RSUSB backend method and got the above error. Today I tried building the SDK from source without patching the kernel and still could not solve the issue. No RGB frames except YUYV, but a different error this time.
21/04 14:25:00,089 INFO [281472768059648] (rs.cpp:2701) Framebuffer size changed to 1848 x 1016 21/04 14:25:00,089 INFO [281472768059648] (rs.cpp:2701) Window size changed to 1848 x 1016 21/04 14:25:00,089 INFO [281472768059648] (rs.cpp:2701) Scale Factor is now 1 21/04 14:25:00,106 INFO [281472691943680] (backend-v4l2.cpp:656) Enumerating UVC video0 realpath=/sys/devices/platform/3610000.xhci/usb2/2-3/2-3.4/2-3.4:1.0/video4linux/video0 21/04 14:25:00,107 INFO [281472691943680] (backend-v4l2.cpp:656) Enumerating UVC video1 realpath=/sys/devices/platform/3610000.xhci/usb2/2-3/2-3.4/2-3.4:1.0/video4linux/video1 21/04 14:25:00,107 INFO [281472691943680] (backend-v4l2.cpp:656) Enumerating UVC video2 realpath=/sys/devices/platform/3610000.xhci/usb2/2-3/2-3.4/2-3.4:1.0/video4linux/video2 21/04 14:25:00,107 INFO [281472691943680] (backend-v4l2.cpp:656) Enumerating UVC video3 realpath=/sys/devices/platform/3610000.xhci/usb2/2-3/2-3.4/2-3.4:1.0/video4linux/video3 21/04 14:25:00,108 INFO [281472691943680] (backend-v4l2.cpp:656) Enumerating UVC video4 realpath=/sys/devices/platform/3610000.xhci/usb2/2-3/2-3.4/2-3.4:1.3/video4linux/video4 21/04 14:25:00,108 INFO [281472691943680] (backend-v4l2.cpp:656) Enumerating UVC video5 realpath=/sys/devices/platform/3610000.xhci/usb2/2-3/2-3.4/2-3.4:1.3/video4linux/video5 21/04 14:25:00,536 INFO [281472768059648] (backend-v4l2.cpp:656) Enumerating UVC video0 realpath=/sys/devices/platform/3610000.xhci/usb2/2-3/2-3.4/2-3.4:1.0/video4linux/video0 21/04 14:25:00,536 INFO [281472768059648] (backend-v4l2.cpp:656) Enumerating UVC video1 realpath=/sys/devices/platform/3610000.xhci/usb2/2-3/2-3.4/2-3.4:1.0/video4linux/video1 21/04 14:25:00,537 INFO [281472768059648] (backend-v4l2.cpp:656) Enumerating UVC video2 realpath=/sys/devices/platform/3610000.xhci/usb2/2-3/2-3.4/2-3.4:1.0/video4linux/video2 21/04 14:25:00,537 INFO [281472768059648] (backend-v4l2.cpp:656) Enumerating UVC video3 realpath=/sys/devices/platform/3610000.xhci/usb2/2-3/2-3.4/2-3.4:1.0/video4linux/video3 21/04 14:25:00,537 INFO [281472768059648] (backend-v4l2.cpp:656) Enumerating UVC video4 realpath=/sys/devices/platform/3610000.xhci/usb2/2-3/2-3.4/2-3.4:1.3/video4linux/video4 21/04 14:25:00,538 INFO [281472768059648] (backend-v4l2.cpp:656) Enumerating UVC video5 realpath=/sys/devices/platform/3610000.xhci/usb2/2-3/2-3.4/2-3.4:1.3/video4linux/video5 21/04 14:25:00,544 INFO [281472768059648] (context.cpp:382) Found 1 RealSense devices (mask 0xfe) 21/04 14:25:07,370 INFO [281472768059648] (sensor.cpp:1586) Request: RGB8 Color, Resolved to: YUYV Color, 21/04 14:25:07,377 INFO [281472768059648] (backend-v4l2.cpp:1992) Video node was successfully configured to YUYV format, fd 23 21/04 14:25:07,377 INFO [281472768059648] (backend-v4l2.cpp:1994) Trying to configure fourcc YUYV 21/04 14:25:07,377 INFO [281472768059648] (backend-v4l2.cpp:2195) Metadata node was successfully configured to D4XX format, fd 30 21/04 14:25:07,909 ERROR [281472514619648] (synthetic-stream.cpp:48) Exception was thrown during callback: lS0_E_SaIvELN9__gnu_cxx12_Lock_policyE2EE 21/04 14:25:07,909 INFO [281472514619648] (sensor.cpp:38) Frame Callback Color #0 overdue. (FPS: 30, max duration: 32.258064 ms) 21/04 14:25:07,941 ERROR [281472514619648] (synthetic-stream.cpp:48) Exception was thrown during callback: lS0_E_SaIvELN9__gnu_cxx12_Lock_policyE2EE 21/04 14:25:07,974 ERROR [281472514619648] (synthetic-stream.cpp:48) Exception was thrown during callback: lS0_E_SaIvELN9__gnu_cxx12_Lock_policyE2EE 21/04 14:25:08,006 ERROR [281472514619648] (synthetic-stream.cpp:48) Exception was thrown during callback: lS0_E_SaIvELN9__gnu_cxx12_Lock_policyE2EE 21/04 14:25:08,038 ERROR [281472514619648] (synthetic-stream.cpp:48) Exception was thrown during callback: lS0_E_SaIvELN9__gnu_cxx12_Lock_policyE2EE 21/04 14:25:08,071 ERROR [281472514619648] (synthetic-stream.cpp:48) Exception was thrown during callback: lS0_E_SaIvELN9__gnu_cxx12_Lock_policyE2EE 21/04 14:25:08,103 ERROR [281472514619648] (synthetic-stream.cpp:48) Exception was thrown during callback: lS0_E_SaIvELN9__gnu_cxx12_Lock_policyE2EE 21/04 14:25:08,136 ERROR [281472514619648] (synthetic-stream.cpp:48) Exception was thrown during callback: lS0_E_SaIvELN9__gnu_cxx12_Lock_policyE2EE 21/04 14:25:08,168 ERROR [281472514619648] (synthetic-stream.cpp:48) Exception was thrown during callback: lS0_E_SaIvELN9__gnu_cxx12_Lock_policyE2EE 21/04 14:25:08,200 ERROR [281472514619648] (synthetic-stream.cpp:48) Exception was thrown during callback: lS0_E_SaIvELN9__gnu_cxx12_Lock_policyE2EE 21/04 14:25:08,232 ERROR [281472514619648] (synthetic-stream.cpp:48) Exception was thrown during callback: lS0_E_SaIvELN9__gnu_cxx12_Lock_policyE2EE 21/04 14:25:08,265 ERROR [281472514619648] (synthetic-stream.cpp:48) Exception was thrown during callback: lS0_E_SaIvELN9__gnu_cxx12_Lock_policyE2EE 21/04 14:25:08,297 ERROR [281472514619648] (synthetic-stream.cpp:48) Exception was thrown during callback: lS0_E_SaIvELN9__gnu_cxx12_Lock_policyE2EE 21/04 14:25:08,329 ERROR [281472514619648] (synthetic-stream.cpp:48) Exception was thrown during callback: lS0_E_SaIvELN9__gnu_cxx12_Lock_policyE2EE

@MartyG-RealSense
Copy link
Collaborator

A synthetic stream error usually does not have a traceable cause and is equivalent to a "something went wrong" message.

At present, using RSUSB or not performing patching are the only two methods of installation for Orin that are confirmed to be successful and there are not alternative suggestions, unfortunately. The next librealsense version (likely to be numbered 2.54.0) will support JetPack 5.0.2.

Did you remove the original librealsense installation made with the tx2 instructions before attempting a new installation, please?

https://dev.intelrealsense.com/docs/nvidia-jetson-tx2-installation

@paritoshwale
Copy link
Author

Yes, I did remove the original installation using sudo make uninstall. I guess I will have to wait until the next version is released.
Thanks!

@MartyG-RealSense
Copy link
Collaborator

MartyG-RealSense commented Apr 21, 2023

You are very welcome!

The work-in-progress SDK version that will become the next SDK release is always available on the SDK's development GitHub branch. JetPack 5.0.2 support is already added to this development version.

https://github.com/IntelRealSense/librealsense/tree/development

@giladosk
Copy link

giladosk commented Jun 6, 2023

Hi @MartyG-RealSense, are there any news regarding expected release date for the 5.0.2 merging into the master branch? I prefer not to use development branch for production...
We really need this for our Jetsons, as certain models do not allow streaming RGB on 5.0.2 even with the RSUSB method.

Thanks!

@MartyG-RealSense
Copy link
Collaborator

MartyG-RealSense commented Jun 6, 2023

Hi @giladosk I do not have a time estimate for release at this time, unfortunately.

When developing for a production system, using a production firmware driver is likely to be more important than using a non-master SDK branch. The most recent recommended production firmware is 5.13.0.50. Using this will mean using an SDK older than version 2.53.1, such as 2.50.0 or 2.51.1 (both of which are compatible with firmware 5.13.0.50). This is because 2.53.1 requires firmware 5.14.0.0, which is not yet classed as a production firmware.

@giladosk
Copy link

giladosk commented Jun 6, 2023

@MartyG-RealSense We are already using 5.13.0.50, despite the fact the "recommended firmware version" that appears everywhere is 5.14 :)
I tested using both 2.50 and 2.53, with no success. I know how to install (and sometimes patch if needed) it and have done it many times and on many distros. Here I have no success with all the options, so I hope a newer release would be of help.

@MartyG-RealSense
Copy link
Collaborator

@giladosk I have enquired to my Intel RealSense colleagues about whether they have an approximate release time estimate that they can provide.

@MartyG-RealSense
Copy link
Collaborator

My colleagues confirmed that the next release will be in this week. The recommended firmware for that release will be 5.15.0.2.

@MartyG-RealSense
Copy link
Collaborator

The new SDK version 2.54.1 is now available - please see #11884

@giladosk
Copy link

giladosk commented Jun 8, 2023

Thank you @MartyG-RealSense, I shall use it and provide an update!

@giladosk
Copy link

@MartyG-RealSense
Hi, unfortunately, using the newer versions still haven't changed the lack of RGB input.
I have tested the new 2.54.1 both with the older and newer recommended firmware versions.

@MartyG-RealSense
Copy link
Collaborator

@giladosk You said earlier at #11716 (comment) that certain models of Jetson do not stream RGB even on RSUSB. As you mentioned 'certain models', does that mean that there are Jetson models that RGB does work on, please?

@giladosk
Copy link

@giladosk You said earlier at #11716 (comment) that certain models of Jetson do not stream RGB even on RSUSB. As you mentioned 'certain models', does that mean that there are Jetson models that RGB does work on, please?

@MartyG-RealSense Correct. And for them, it works even with the previous librealsense version. I could have suspected it is the vendor to blame, but on previous Jetpack versions they had no issues.

@MartyG-RealSense
Copy link
Collaborator

Are all of the boards using the same Ubuntu version?

@giladosk
Copy link

Are all of the boards using the same Ubuntu version?

The Ubuntu version is the same for all Jetpack 5.0.2, which is 20.04. For this version, the RBG stream does not work for some of the vendors.
For the earlier, Jetpack 4.6, the Ubuntu version is 18.04, and the RGB stream works well for all vendors.

@MartyG-RealSense
Copy link
Collaborator

Yes, JetPack 4.6 is the appropriate JetPack to use with Ubuntu 18.04. Nvidia created JetPack 5 for use with Ubuntu 20.04.

As librealsense does not have Arm64 packages for Ubuntu 20.04, building librealsense from source code with RSUSB backend = true was also an appropriate action.

What was your full CMake buikd instruction on 20.04? In addition to -DFORCE_RSUSB_BACKEND=TRUE, did you also set -DBUILD_GRAPHICAL_EXAMPLES=true ? This instruction does not only build example programs and tools but also includes graphics support in the librealsense build.

@giladosk
Copy link

I did include this flag, as it allows me to fully use the realsense-viewer. I was following the comments you always refer to that include the step-by-step instructions for building on arm:
cmake ../ -DFORCE_RSUSB_BACKEND=ON -DBUILD_PYTHON_BINDINGS:bool=true -DPYTHON_EXECUTABLE=/usr/bin/python3 -DCMAKE_BUILD_TYPE=release -DBUILD_EXAMPLES=true -DBUILD_GRAPHICAL_EXAMPLES=true -DBUILD_WITH_CUDA:bool=true

And it always worked, until this version.

@MartyG-RealSense
Copy link
Collaborator

@giladosk Which Python version are you using, please? 2.54.1, and the earlier development branch that it was based on, added Python 3.11 support but removed support for Python 3.6.

@giladosk
Copy link

@MartyG-RealSense We are using the 20.04-native python version, 3.8.
Importing the pyrealsense2 works, if that's where you're aiming at.

@MartyG-RealSense
Copy link
Collaborator

MartyG-RealSense commented Jun 13, 2023

Thanks, Python 3.8 should be fine as it was 3.6 that had support removed.

It would be helpful to get an update from @paritoshwale the original creator of this issue.

Hi @paritoshwale Have you had the opportunity to test your Jetson Orin and JetPack 5.0.2 with the new 2.54.1 SDK? If you have, did you continue to experience No Frames Received on the RGB stream?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants