Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🐛 v4l: fix perf issue introduce by disabled arena buffer #129

Merged
merged 1 commit into from
Jun 15, 2023

Conversation

OlivierLDff
Copy link

@OlivierLDff OlivierLDff commented Jun 15, 2023

fix #128

  • Calling stream.next push internal buffers to v4l then call stream.start. By default with new buffer count is 4.
  • Add Raspberry Pi support #121 directly call stream.start, without setting any buffer. This result in v4l only having 1 buffer (I guess), so because there is no double buffer (or more), 1 frame out of two is skipped.

I used the following python script to benchmark expected camera behavior:

import cv2


# Create a VideoCapture object
cap = cv2.VideoCapture(0)

# Check if camera opened successfully
if cap.isOpened() == False:
    print("Unable to read camera feed")

# Define the codec and create VideoWriter object.The output is stored in 'outpy.avi' file.
window_name = "Image"
cv2.namedWindow(window_name, cv2.WINDOW_NORMAL)

fps_count = 0
last_fps_time_updated = 0

while True:
    ret, frame = cap.read()

    # print frame.shape
    print(f"Frame: {frame.shape}")

    if ret == True:
        # Display the resulting frame
        cv2.imshow(window_name, frame)

        fps_count += 1
        current_time = cv2.getTickCount()
        if (current_time - last_fps_time_updated) > cv2.getTickFrequency():
            print("FPS: {}".format(fps_count))
            fps_count = 0
            last_fps_time_updated = current_time

        # Press Q on keyboard to stop recording
        if cv2.waitKey(1) & 0xFF == ord("q"):
            break

    # Break the loop
    else:
        break

And the following rust code:

                let mut camera = Camera::new(
                    index.clone(),
                    RequestedFormat::new::<RgbFormat>(RequestedFormatType::Closest(
                        CameraFormat::new(
                            Resolution {
                                width_x: 320,
                                height_y: 240,
                            },
                            FrameFormat::MJPEG,
                            30,
                        ),
                    )),
                )
                .unwrap();

                println!("format = {:?}", camera.camera_format());
                let mut last_fps_updated = std::time::Instant::now();
                let mut frame_count = 0;
                camera.open_stream().unwrap();
                loop {
                    let frame = camera.frame().unwrap();
                    println!(
                        "Captured frame of size {}, with size {:?}, format {:?}",
                        frame.buffer().len(),
                        frame.resolution(),
                        frame.source_frame_format()
                    );
                    let decoded = frame.decode_image::<RgbFormat>().unwrap();
                    println!(
                        "DecodedFrame of {}, resolution {:?}x{:?}",
                        decoded.len(),
                        decoded.width(),
                        decoded.height()
                    );
                    frame_count += 1;

                    let now = std::time::Instant::now();
                    if now - last_fps_updated > std::time::Duration::from_secs(1) {
                        println!("FPS: {}", frame_count);

                        frame_count = 0;
                        last_fps_updated = now;
                    }
                }

The performance regression can easily be observed in v4l examples too by adding stream.start:

https://github.com/raymanfx/libv4l-rs/blob/867d9b122fde603ff7f1bfcdf19de48845c9e1f7/examples/stream_capture_mmap.rs#L26-L29

- Calling `stream.next` push internal buffers to v4l then call `stream.start`. By default with `new` buffer count is 4.
- l1npengtul#121 directly call `stream.start`, without setting any buffer. This result in v4l only having 1 buffer (I guess), so because there is no double buffer (or more), 1 frame out of two is skipped.
@OlivierLDff
Copy link
Author

I've correctly rebased on 0.10 to fix conflict introduced by my other PR.

@l1npengtul
Copy link
Owner

Can you undo the rebase? Apologies, I was fixing the PR in my IDE, 0.10 is dead and is in "unless-this-kills-your-system-no-more-updates" mode

@OlivierLDff
Copy link
Author

I'm not sure to understand, this PR was already targetting 0.10 branch, like my other PR. There were a conflict due to the lock introduced, I needed to fix for the merge to be possible.
I guess this is quite an important bug fix, having frame rate divided by 2 doesn't kill the system, but make the library unusable on linux. 13 fps instead of 25 for a camera isn't acceptable

@OlivierLDff
Copy link
Author

I would gladly cherry pick my multiples commits later and create other PR for the senpai branch, but it doesn't compile at the moment so I can't check if this works.

@l1npengtul
Copy link
Owner

Ah. Apologies. Yeah, I'll make sure to cherry pick this and #127 for later.

@l1npengtul l1npengtul merged commit ea97124 into l1npengtul:0.10 Jun 15, 2023
@l1npengtul l1npengtul added the cherry-pick Cherry pick into release ver 0.3 label Jun 15, 2023
@OlivierLDff
Copy link
Author

Yes, I will do as soon as senpai branch will be compiling.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cherry-pick Cherry pick into release ver 0.3
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants