Low and unstable Frame Rate with Intel Realsense L515 and Jetson nano

[jacklu333333]

I am using Realsense2 L515 with Jetson nano. However, I experienced a low and unstable frame rate problem

My camera configuration only enable color, depth, aligned function with 1280x720 30 fps, 640x480 30 fps, 1280x720 30 fps, respectively.

The fps for me to get is quite unstable sometimes it will fail to publish any messages and average fps which I can get is around 10.

I rebuild my library locally with
OpenCV 4.0
Realsense 2.2.24
Librealsense 2.44
to use the GPU, however, the GPU remains 0% with low fps.

I suspect it was the problem of power issue, so I provide the external power through the other socket, which is for sync. But the problem remains.

The USB cable and the camera are not broken because I experiment with the laptop and get the ideal fps 30.

It will not be the computation limitation problem because I have experiment solely with realsense ros running. Also I got the 30 fps onece with Jetson nao once.

[jacklu333333]

Any update?

[jacklu333333]

Hi, I used this
JetsonHacksNano/installROS
JetsonHacksNano/installLibrealsense
JetsonHacksNano/installRealSenseROS

and follow the instruction
However, I am using L515 and melodic ros, in the above default version is 2.2.11. Hence I have to set the version to
Realsense 2.2.24
Librealsense 2.44

But the camera becomes really weird, if I enable aligned, depth, and color image at the same time. Only the depth is working, neither aligned and the color is working.

And if I block the depth sensor physically, the color image will be available. But it will be quite unable and the image blink(miss frame)

Hope you can reproduce the same error.

[jacklu333333]

I manage to get it to work with Realsense 2.3.0
However, I cannot enable aligned and pointcloud at the same time. '
If I use both aligned and pointcloud
depth 30 fps
color 0 fps
aligned 0 fps
pointcloud 0 fps

If I enable only aligned
depth 30 fps
color 24 fps
aligned 12 fps

If I enable only pointcloud
depth 30 fps
color 0 fps
pointcloud 0 fps

The low frame and unable problem remain.

[RealSenseSupport]

@jacklu333333

It seems to be a system level bandwidth issue. I'm trying to understand, if you use a system that is higher power, higher performance system, you're able to get proper fps with the configuration that you need?

[jacklu333333]

If it is the case, then it is really weird.
Since I also test on my laptop with core i5 gen 10, Nvidia mx350, and USB 3.0 port, connecting with the original USB cable, the FPS drop is still around 10%. I wouldn't say it is a high-end computer, but it should be sufficient.

[jacklu333333]

Is there any recommendation for the single-board-processor(SBC) to support the high bandwidth for L515?

[MartyG-RealSense]

Hi @jacklu333333 Do you require further assistance with this case, please? Thanks!

[jacklu333333]

It seems like this is mostly fixed by building in Release mode. Sorry to bother you. (Will re-open if it becomes an issue again.)

It seems it was not the reason in my case, since I install it directly from the ubuntu ROS1 source. But I will test it with manual build to see whether it can resolve the problem or not.

Thank you for your reply.
Best Regards,
Jack Lu

[MartyG-RealSense]

Thanks very much, @jacklu333333 - I look forward to your next update after a manual build test. Good luck!

[MartyG-RealSense]

Hi @jacklu333333 Do you have an update about this case that you can provide, please? Thanks!

[jacklu333333]

Hi, sorry for the late reply. I am having some version compatible issues recreating the environment in Jetson nano.

However, I do test up my computer with docker using ROS connecting through USB 3.0 with the hardware configuration of the following:

CPU: AMD Ryzen 7 3700X 8-Core @ 16x 3.6GHz
GPU: NVIDIA GeForce GTX 1660
RAM: 32050MiB

I follow the instruction to make the package with the cmd
catkin_make -DCATKIN_ENABLE_TESTING=False -DCMAKE_BUILD_TYPE=Release
to ensure it is release version with the realsense-ros version 2.2.24

I enable aligned, pointcloud, accelerometer, and gyroscope. With the following resolution configuration.

depth : 640*480 30fps
color : 1280*720 30fps
accelerometer : 100hz
gyroscope :  100hz

Initially, I got perfectly 30 fps if I only subscribe to the pointcloud. However, once I subscribe to any other information, I can see a significant frame drop depending on how much information I subscribe.

@jacklu333333

It seems to be a system level bandwidth issue. I'm trying to understand, if you use a system that is higher power, higher performance system, you're able to get proper fps with the configuration that you need?

I start to wonder that maybe it is indeed the bandwidth problem. But in that case, it means even with so many features that L515 has, its own communication doesn't support its whole features at the same time.

I will provide an update later for the Jetson nano when the environment is setuped.

[MartyG-RealSense]

Hi @jacklu333333 Regarding the suggestion above from RealSenseSupport about power, Intel's official Jetson instructions have a strong recommendation for Nano boards that the barrel jack connector is enabled to provide extra power.

https://github.com/IntelRealSense/librealsense/blob/master/doc/installation_jetson.md

[jacklu333333]

Yes, I did power through barrel jack with the adapter of 5V 3A power supply. Previously, I do suspect it is the problem with power but I have no way to test it. However, since I already provide the highest power Jetson nano can handle, I assume it is not the reason.

Hi @jacklu333333 Regarding the suggestion above from RealSenseSupport about power, Intel's official Jetson instructions have a strong recommendation for Nano boards that the barrel jack connector is enabled to provide extra power.

https://github.com/IntelRealSense/librealsense/blob/master/doc/installation_jetson.md

[MartyG-RealSense]

Which firmware driver version do you currently have installed in your camera?

[jacklu333333]

Hi,
The firmware version is Device FW version: 01.05.05.00
I am now able to recreate the Jetsonnano environment.
The result remains the same even with catkin_make -DCATKIN_ENABLE_TESTING=False -DCMAKE_BUILD_TYPE=Release command to make the release version.

The following document is my launch file and screen log respectively hope this can help isolating the problem.
Screen Log
Launch File

[MartyG-RealSense]

Thank you very much for the information and logs. When you are publishing the accel and gyro, does performance improve if you change 'unite_imu_method' in your launch file to use copy instead of linear_interpolation

<arg name="unite_imu_method" default="copy"/>

[jacklu333333]

The problem remains with this configuration.

Thank you very much for the information and logs. When you are publishing the accel and gyro, does performance improve if you change 'unite_imu_method' in your launch file to use copy instead of linear_interpolation

<arg name="unite_imu_method" default="copy"/>

[MartyG-RealSense]

If you are using Docker as mentioned in #1850 (comment) and experience lag if you subscribe to more than the pointcloud, do you also experience lag if using ROS without Docker?

[jacklu333333]

It was not lag, it was data loss.

If you are using Docker as mentioned in #1850 (comment) and experience lag if you subscribe to more than the pointcloud, do you also experience lag if using ROS without Docker?

I did experiment upon three different environment

1. Nvidia-docker on PC
2. Laptop
3. Jetson nano

The hardware configuration's superiority is in this order, so is the performance. All three of them have a certain level of frame loss depending upon how much data is published and suscribe. But even with the best one (PC) I still can not acquire 100% data. However, the major issue is that I need to work it with SBC (Jetson nano) so that it can integrate with robots.

[MartyG-RealSense]

Other RealSense users with Jetson Nano reported problems with the control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11 error in your launch log in #1164

One user in that discussion whose FPS was about 7 also thought it may be power related. They added in #1164 (comment) that they had fewest problems when installing on Nano using the 'binary' method provided by the JetsonHacks website. They may be referring to the guide in the link below.

https://www.jetsonhacks.com/2019/10/25/realsense-ros-wrapper-jetson-nano/

[MartyG-RealSense]

Hi @jacklu333333 Do you require further assistance with this case, please? Thanks!

[jacklu333333]

Thank you for your help.
I just test on the binary again. However, the issue remains. I have no choice but to suspect that it is simply the product is not well-design. It is the hardware problem, not the software problem.

In here, they do mention the problem might have something to do with the orientation of the USB cable.

All in all, thank you anyway, let's close this thread since there is no other possibility.

[MartyG-RealSense]

You are very welcome, @jacklu333333 - thanks very much for the update.