L515 Unstable Frame Rate

[FeiSong123]

• Before opening a new issue, we wanted to provide you with some useful suggestions (Click "Preview" above for a better view):

  • Consider checking out SDK examples.
  • Have you looked in our documentations?
  • Is you question a frequently asked one?
  • Try searching our GitHub Issues (open and closed) for a similar issue.

• All users are welcomed to report bugs, ask questions, suggest or request enhancements and generally feel free to open new issue, even if they haven't followed any of the suggestions above :)

---

Required Info
Camera Model	L515
Firmware Version	01.05.08.01
Operating System & Version	Linux
Kernel Version (Linux Only)	5.4.0-122-generic
Platform	PC
SDK Version	2.50.0
Language	C++
Segment	Robot

Issue Description

<Describe your issue / question / feature request / etc..>
L515 wait_for_frames() sometimes (52ms-60ms) sometimes under 10ms, cannot reach the preset 30 frames.
When run on same code with other camera D435/D455 (60FPS), it's fine.

here is the time cout by L515:

wait time:0.014355
update time:0.100019
time:0.340201

-------------------程序总时间:1.52427

wait time:61.1373
update time:61.2394
time:61.4984

-------------------程序总时间:63.3608

wait time:0.429982
update time:0.499923
time:0.700047

-------------------程序总时间:1.8975

wait time:61.1578
update time:61.258
time:61.5415

-------------------程序总时间:63.0588

wait time:2.03914
update time:2.14394
time:2.45241

-------------------程序总时间:4.00483

wait time:59.0686
update time:59.1937
time:59.4571

-------------------程序总时间:60.8233

wait time:3.79388
update time:3.87624
time:4.12769

-------------------程序总时间:5.37105

wait time:58.0602
update time:58.1426
time:58.3969

-------------------程序总时间:60.457

Here is my code:

TickMeter tk;
tk.start();
  frameSet = pipe.wait_for_frames();											//等待相机数据	
tk.stop();
cout << "wait time:" << tk.getTimeMilli() << endl;
tk.start();

  rs2::video_frame colorFrame = frameSet.get_color_frame();					//获取彩色帧
  rs2::depth_frame alignedDepthFrame = frameSet.get_depth_frame();			//获取深度帧

  srcImage = cv::Mat(cv::Size(IMAGE_WIDTH, IMAGE_HEIGHT), CV_8UC3, (void*)colorFrame.get_data(), cv::Mat::AUTO_STEP);				//将彩色图像数据存储在Mat矩阵中
  depthImage = cv::Mat(cv::Size(DEPTH_WIDTH, DEPTH_HEIGHT), CV_16UC1, (void*)alignedDepthFrame.get_data(), cv::Mat::AUTO_STEP);	//将深度图像数据存储在Mat矩阵中
memcpy(data, alignedDepthFrame.get_data(), DEPTH_WIDTH * DEPTH_HEIGHT * 2);

#ifdef PCL
rs2Points = rs2Cloud.calculate(alignedDepthFrame);							
rs2Cloud.map_to(colorFrame);
cloudByRS2 = PointsToPointCloud(rs2Points, colorFrame);
#endif

tk.stop();
cout << "update time:" << tk.getTimeMilli() << endl;

[MartyG-RealSense]

Hi @FeiSong123 Could you test please whether performance improves if you unplug the micro-sized end of the USB cable from the base of the L515 camera, turn the connector around the other way and re-insert it into the base of the camera (USB Type-C cables are two way insertion at the micro-sized end).

[FeiSong123]

I repeated the operation several times and found that improved to 30ms-40ms for once only.

[MartyG-RealSense]

Next, please try disabling the auto-exposure priority RGB option. When auto-exposure is enabled and auto-exposure priority is disabled, this should force the RealSense SDK to try to maintain depth and color FPS at the same rate instead of allowing it to vary.

The C++ code below can be inserted in the line after the second tk.start instruction and queries the color sensor (which has index number '1', whilst the depth sensor is index number '0').

auto sensor = pipe_profile.get_device().query_sensors()[1];
sensor.set_option(RS2_OPTION_AUTO_EXPOSURE_PRIORITY, 0);

[FeiSong123]

I added it to the initialization configuration, but it still unstable (30ms-60ms), sometimes under 10ms

rs2::pipeline_profile selection = pipe.start(cfg);
cout << "start cfg Ok ..\n";

auto colorSensors = selection.get_device().query_sensors()[1];
colorSensors.set_option(RS2_OPTION_AUTO_EXPOSURE_PRIORITY, 0);
// colorSensors.set_option(RS2_OPTION_ENABLE_AUTO_EXPOSURE, 0); //zero is turn off
// colorSensors.set_option(RS2_OPTION_EXPOSURE, 650);
// colorSensors.set_option(RS2_OPTION_GAIN, 300); //设置曝光时间
// colorSensors.set_option(RS2_OPTION_WHITE_BALANCE, 4500); //设置白平衡
cout << "set exposure OK ..\n";

[MartyG-RealSense]

If you have disabled auto-exposure and are setting a manual exposure value in a certain range then it can cause the FPS to lag. This effect is described at #1957 (comment)

It may therefore be worth commenting out the auto-exposure disable and manual exposure = 650 lines in order to confirm whether or not the manual exposure is the cause of the FPS drop.

The constant FPS rate provided by disabling auto-exposure priority will also not work if auto-exposure is disabled.

[FeiSong123]

I used auto exposure and turned off auto exposure priority as described at #1957 (comment) and FPS is now stable.
Thanks for your help very much!

[MartyG-RealSense]

That's great, @FeiSong123 - thanks very much for the update :)

[FeiSong123]

Hi, I want to know how to set RS2_OPTION_AUTO_EXPOSURE_MODE, but I couldn't find the corresponding usage in the repository.

[FeiSong123]

When I set up my camera as follows, I encountered a strange problem:

    auto colorSensors = profiles.get_device().query_sensors()[1];
colorSensors.set_option(RS2_OPTION_AUTO_EXPOSURE_PRIORITY, 0);

Below are four consecutive images taken by the camera, which appear to suddenly increase and then decrease in light intensity:

Then I wondered if there was a problem with the auto exposure settings, so I went to check the auto exposure related settings.

[MartyG-RealSense]

There are some options in the RealSense SDK that have never been used and so have no scripting references available for them. The best reference available for RS2_OPTION_AUTO_EXPOSURE_MODE is a Python one at #5602 (comment) that describes the values for this setting as:

0: Static
1: Anti-Flicker
2: Hybrid

A check of the L515's supported options using the RealSense SDK's rs-sensor-control tool showed that the option was not supported on L515.