RSD455 can not get 90FPS at 480*270 resolution #7749

FrankCreen · 2020-11-09T16:35:58Z

Test Program

// include the librealsense C++ header file
#include <librealsense2/rs.hpp>

#include <opencv2/opencv.hpp>

#include <chrono>

using namespace std;
using namespace cv;


int main()
try
{
    //Contruct a pipeline which abstracts the device
    rs2::pipeline pipe;

    //Create a configuration for configuring the pipeline with a non default profile
    rs2::config cfg;
    //Add desired streams to configuration
    cfg.enable_stream(RS2_STREAM_COLOR, 480, 270, RS2_FORMAT_BGR8, 90);

    //Instruct pipeline to start streaming with the requested configuration
    pipe.start(cfg);

    // Display in a GUI
    const auto window_name = "Display Video";
    namedWindow(window_name, WINDOW_AUTOSIZE);


   

    //用于计时
    //show time
    auto start = chrono::system_clock::now();
    int frame = 0;

    while (waitKey(1) < 0 && getWindowProperty(window_name, WND_PROP_AUTOSIZE) >= 0)
    {
        rs2::frameset data = pipe.wait_for_frames(); // Wait for next set of frames from the camera

        //获取彩色图像帧
        rs2::frame color = data.get_color_frame();

        //保存彩色图像
        Mat image(Size(480, 270), CV_8UC3, (void *)color.get_data(), Mat::AUTO_STEP);


        // Update the window with new data
        imshow(window_name, image);

        frame++;
        //one loop end
        auto end = chrono::system_clock::now();
        auto duration = chrono::duration_cast<chrono::microseconds>(end - start);
        double timecost = double(duration.count()) * chrono::microseconds::period::num / chrono::microseconds::period::den;

        cout << frame / timecost << " FPS" << endl;

    }


    return 0;
}
catch (Exception ex)
{
    cout << "Error!!!" << endl;
    cerr << ex.msg << endl;
}

Output

55.5214 FPS
55.5205 FPS
55.5054 FPS
55.5079 FPS
55.4945 FPS
55.4764 FPS
55.4838 FPS
55.4904 FPS
55.491 FPS
55.4976 FPS
55.5032 FPS
55.5051 FPS
55.5176 FPS
55.532 FPS
55.5319 FPS

The text was updated successfully, but these errors were encountered:

MartyG-RealSense · 2020-11-09T17:51:26Z

Hi @FrankCreen I ran a test at 480x270 RGB in the RealSense Viewer program at 90 FPS in RGB8 format with RGB Auto-Exposure disabled and Auto-Exposure Priority enabled and achieved 62 FPS.

Having RGB Auto-Exposure enabled and Auto-Exposure Priority enabled provided 29 FPS.

Running the test with RGB Auto-Exposure enabled and Auto-Exposure Priority disabled provided 90 FPS. This is because having AE enabled and AE Priority disabled forces the FPS to try to stay at a constant rate.

If your project requires Auto-Exposure disabled then you can also fix the FPS to a constant rate by setting manual exposure to a certain range.

#1957

If you decide to go for the method of Auto-Exposure Enabled and Auto-Exposure Prority disabled then the link below has C++ code for disabling AE Priority:

#5290

FrankCreen · 2020-11-10T07:52:31Z

Hi @FrankCreen I ran a test at 480x270 RGB in the RealSense Viewer program at 90 FPS in RGB8 format with RGB Auto-Exposure disabled and Auto-Exposure Priority enabled and achieved 62 FPS.

Having RGB Auto-Exposure enabled and Auto-Exposure Priority enabled provided 29 FPS.

Running the test with RGB Auto-Exposure enabled and Auto-Exposure Priority disabled provided 90 FPS. This is because having AE enabled and AE Priority disabled forces the FPS to try to stay at a constant rate.

If your project requires Auto-Exposure disabled then you can also fix the FPS to a constant rate by setting manual exposure to a certain range.

#1957

If you decide to go for the method of Auto-Exposure Enabled and Auto-Exposure Prority disabled then the link below has C++ code for disabling AE Priority:

#5290

Thanks for your detailed reply,I got some progress but not complete. I add the "Auto_Exposure " and "Auto_Exposure_Priority" setting option after the "pipe.start" , the output even far away from 90FPS.

Test Program with exposure setting

// include the librealsense C++ header file
#include <librealsense2/rs.hpp>

#include <opencv2/opencv.hpp>

#include <chrono>

using namespace std;
using namespace cv;

int main()
try
{
    //Contruct a pipeline which abstracts the device
    rs2::pipeline pipe;

    //Create a configuration for configuring the pipeline with a non default profile
    rs2::config cfg;
    //Add desired streams to configuration
    cfg.enable_stream(RS2_STREAM_COLOR, 480, 270, RS2_FORMAT_BGR8, 90);

    //Instruct pipeline to start streaming with the requested configuration
    pipe.start(cfg);

    //Sensor Settings
    auto sensor1 = pipe.get_active_profile().get_device().query_sensors()[0];
    sensor1.set_option(RS2_OPTION_ENABLE_AUTO_EXPOSURE, 1);
    auto sensor2 = pipe.get_active_profile().get_device().query_sensors()[1];
    sensor2.set_option(RS2_OPTION_AUTO_EXPOSURE_PRIORITY, 0);

    // Display in a GUI
    const auto window_name = "Display Video";
    namedWindow(window_name, WINDOW_AUTOSIZE);

    //用于计时
    //show time
    auto start = chrono::system_clock::now();
    int frame = 0;

    while (waitKey(1) < 0 && getWindowProperty(window_name, WND_PROP_AUTOSIZE) >= 0)
    {
        rs2::frameset data = pipe.wait_for_frames(); // Wait for next set of frames from the camera

        //获取彩色图像帧
        rs2::frame color = data.get_color_frame();

        //保存彩色图像
        Mat image(Size(480, 270), CV_8UC3, (void *)color.get_data(), Mat::AUTO_STEP);

        // Update the window with new data
        imshow(window_name, image);

        frame++;
        //one loop end
        auto end = chrono::system_clock::now();
        auto duration = chrono::duration_cast<chrono::microseconds>(end - start);
        double timecost = double(duration.count()) * chrono::microseconds::period::num / chrono::microseconds::period::den;

        cout << frame / timecost << " FPS" << endl;
    }

    return 0;
}
catch (Exception ex)
{
    cout << "Error!!!" << endl;
    cerr << ex.msg << endl;
}

Output

64.4443 FPS
64.4459 FPS
64.4472 FPS
64.4435 FPS
64.443 FPS
64.4466 FPS
64.4482 FPS
64.4479 FPS
64.4422 FPS
64.4419 FPS
64.4394 FPS
64.4388 FPS
64.4406 FPS

So,what I missed ?

#7661

MartyG-RealSense · 2020-11-10T08:17:31Z

Depth and RGB have separate Auto Exposure, and it is the RGB Auto Exposure that should be enabled. Can you check whether you are setting Auto Exposure on the RGB sensor and not the Depth one please?

It looks as though you should be using [1] as the index for query_sensors on the auto sensor1 line instead of [0] in order to set RGB exposure instead of depth exposure.

//Sensor Settings
    auto sensor1 = pipe.get_active_profile().get_device().query_sensors()[1];
    sensor1.set_option(RS2_OPTION_ENABLE_AUTO_EXPOSURE, 1);
    auto sensor2 = pipe.get_active_profile().get_device().query_sensors()[1];
    sensor2.set_option(RS2_OPTION_AUTO_EXPOSURE_PRIORITY, 0);

FrankCreen · 2020-11-10T08:34:59Z

Depth and RGB have separate Auto Exposure, and it is the RGB Auto Exposure that should be enabled. Can you check whether you are setting Auto Exposure on the RGB sensor and not the Depth one please?

It looks as though you should be using [1] as the index for query_sensors on the auto sensor1 line instead of [0] in order to set RGB exposure instead of depth exposure.
//Sensor Settings
    auto sensor1 = pipe.get_active_profile().get_device().query_sensors()[1];
    sensor1.set_option(RS2_OPTION_ENABLE_AUTO_EXPOSURE, 1);
    auto sensor2 = pipe.get_active_profile().get_device().query_sensors()[1];
    sensor2.set_option(RS2_OPTION_AUTO_EXPOSURE_PRIORITY, 0);

Haha,not change more@_@

66.2738 FPS
66.2836 FPS
66.2801 FPS
66.2939 FPS
66.3056 FPS
66.3154 FPS
66.333 FPS
66.3363 FPS
66.3113 FPS
66.2843 FPS
66.2761 FPS
66.2796 FPS
66.2944 FPS
66.2772 FPS
66.2535 FPS

There's no official tutorials about this. I found these code on the internet which work with D435. Does we have to add some other magic code to work with D455?

MartyG-RealSense · 2020-11-10T08:54:34Z

You should not need special code for the script to work with D455.

The sensor code looks okay to me. Could you verify please in the RealSense Viewer that you can achieve 90 FPS with RGB AE enabled and AE Priority disabled? If it can, then I would recommend investigating whether the frame / timecost formula for calculating the FPS printout is providing the correct result.

The information overlay that provides the live FPS information can be toggled with the 'i' option on the top of the stream's panel.

FrankCreen · 2020-11-12T03:16:42Z

You should not need special code for the script to work with D455.

The sensor code looks okay to me. Could you verify please in the RealSense Viewer that you can achieve 90 FPS with RGB AE enabled and AE Priority disabled? If it can, then I would recommend investigating whether the frame / timecost formula for calculating the FPS printout is providing the correct result.

The information overlay that provides the live FPS information can be toggled with the 'i' option on the top of the stream's panel.

Hi Marty! I want to ask the difference between the "Hardware FPS" and "Viewer FPS".Futhermore，the "Actual fps" in the below. Thank you very much!

MartyG-RealSense · 2020-11-12T10:09:03Z

The model-views.cpp SDK file provides the following explanation:

Hardware FPS captures the number of frames per second produced by the device. It is possible and likely that not all of these frames will make it to the application.

Viewer FPS captures how many frames the application manages to render. Frame drops can occur for variety of reasons.

https://github.com/IntelRealSense/librealsense/blob/master/common/model-views.cpp#L2655

So the camera hardware may be successfully capturing at 90 FPS but there is some factor that is holding the FPS back when the data is rendered in the Viewer.

In my own tests in the Viewer, I got around 62 FPS when set at 90 if I had RGB Auto-Exposure disabled and Auto-Exposure Priority enabled.

Otherwise, if I had RGB Auto-Exposure enabled and Auto-Exposure Priority disabled then I got 90 FPS.

FrankCreen · 2020-11-12T16:01:49Z

The model-views.cpp SDK file provides the following explanation:

Hardware FPS captures the number of frames per second produced by the device. It is possible and likely that not all of these frames will make it to the application.

Viewer FPS captures how many frames the application manages to render. Frame drops can occur for variety of reasons.

https://github.com/IntelRealSense/librealsense/blob/master/common/model-views.cpp#L2655

So the camera hardware may be successfully capturing at 90 FPS but there is some factor that is holding the FPS back when the data is rendered in the Viewer.

In my own tests in the Viewer, I got around 62 FPS when set at 90 if I had RGB Auto-Exposure disabled and Auto-Exposure Priority enabled.

Otherwise, if I had RGB Auto-Exposure enabled and Auto-Exposure Priority disabled then I got 90 FPS.

Thanks! I solved my problem with a NVIDIA 1070ti GPU that supports opencv4 computation accleration. ^_^

MartyG-RealSense · 2020-11-12T17:54:02Z

Excellent news - thanks very much for the update!

MartyG-RealSense · 2020-11-19T11:12:46Z

Case closed due to a successful outcome and no further comments received.

MartyG-RealSense added the d455 label Nov 9, 2020

MartyG-RealSense mentioned this issue Nov 13, 2020

D455 can not connected on the Jetson Xaiver #7766

Closed

MartyG-RealSense closed this as completed Nov 19, 2020

MartyG-RealSense mentioned this issue Aug 23, 2021

The captured video did not reach the specified frame rate #9659

Closed

MartyG-RealSense mentioned this issue Mar 17, 2022

Number of frames in a .bag file, and timestamp of each frame #10317

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

RSD455 can not get 90FPS at 480*270 resolution #7749

RSD455 can not get 90FPS at 480*270 resolution #7749

FrankCreen commented Nov 9, 2020 •

edited

Loading

MartyG-RealSense commented Nov 9, 2020 •

edited

Loading

FrankCreen commented Nov 10, 2020

MartyG-RealSense commented Nov 10, 2020 •

edited

Loading

FrankCreen commented Nov 10, 2020

MartyG-RealSense commented Nov 10, 2020 •

edited

Loading

FrankCreen commented Nov 12, 2020 •

edited

Loading

MartyG-RealSense commented Nov 12, 2020 •

edited

Loading

FrankCreen commented Nov 12, 2020

MartyG-RealSense commented Nov 12, 2020

MartyG-RealSense commented Nov 19, 2020

RSD455 can not get 90FPS at 480*270 resolution #7749

RSD455 can not get 90FPS at 480*270 resolution #7749

Comments

FrankCreen commented Nov 9, 2020 • edited Loading

Test Program

Output

MartyG-RealSense commented Nov 9, 2020 • edited Loading

FrankCreen commented Nov 10, 2020

Test Program with exposure setting

Output

MartyG-RealSense commented Nov 10, 2020 • edited Loading

FrankCreen commented Nov 10, 2020

MartyG-RealSense commented Nov 10, 2020 • edited Loading

FrankCreen commented Nov 12, 2020 • edited Loading

MartyG-RealSense commented Nov 12, 2020 • edited Loading

FrankCreen commented Nov 12, 2020

MartyG-RealSense commented Nov 12, 2020

MartyG-RealSense commented Nov 19, 2020

FrankCreen commented Nov 9, 2020 •

edited

Loading

MartyG-RealSense commented Nov 9, 2020 •

edited

Loading

MartyG-RealSense commented Nov 10, 2020 •

edited

Loading

MartyG-RealSense commented Nov 10, 2020 •

edited

Loading

FrankCreen commented Nov 12, 2020 •

edited

Loading

MartyG-RealSense commented Nov 12, 2020 •

edited

Loading