Add hardware accelerated scaling when using ffmpeg hwaccel presets

[NickM-27]

In my testing, using hardware accelerated scaling brought CPU usage down considerably when using the non-native resolution for detect.

[netlify]

✅ Deploy Preview for frigate-docs canceled.

Name	Link
🔨 Latest commit	6a4c23b
🔍 Latest deploy log	https://app.netlify.com/sites/frigate-docs/deploys/63af0c4982d6c00009b97185

[NickM-27]

This is an example of the drop in ffmpeg process CPU usage after enabling this for my 4 streams (3 that are resized and 1 that isn't)

[banthungprong]

Do I have to change the config files for this? When I test the image I get the following type of errors for all cams:

frigate | 2022-12-31 11:32:20.550734105 [2022-12-31 11:32:20] ffmpeg.links.detect ERROR : pipe:: Invalid argument
frigate | 2022-12-31 11:32:20.558463857 [2022-12-31 11:32:20] watchdog.vorne ERROR : Ffmpeg process crashed unexpectedly for vorne.
frigate | 2022-12-31 11:32:20.559598447 [2022-12-31 11:32:20] watchdog.vorne ERROR : The following ffmpeg logs include the last 100 lines prior to exit.
frigate | 2022-12-31 11:32:20.559606134 [2022-12-31 11:32:20] ffmpeg.vorne.detect ERROR : [NULL @ 0x55f19a5204c0] Unable to find a suitable output format for 'pipe:'

[NickM-27]

What does your config look like?

[banthungprong]

ffmpeg:
  global_args: -hide_banner -loglevel error
  hwaccel_args: -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p

  output_args:
    record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an

  links:
    enabled: True
    ffmpeg:
      inputs:
        - path: rtsp://.../Streaming/Channels/202
          roles:
            - detect
        - path: rtsp://.../Streaming/Channels/201
          roles:
            - record
            - restream
    restream:
      enabled: True

[NickM-27]

In general I'd highly suggest using an ffmpeg preset instead of manual args. But I'll check that out and see what's going on with a config like that

[banthungprong]

Will do... Can I find a list of presets in the docs or do in the sources (assuming that this feature is a new one)?

[deviant77]

Will do... Can I find a list of presets in the docs or do in the sources (assuming that this feature is a new one)?

https://github.com/blakeblackshear/frigate/blob/dev/frigate/ffmpeg_presets.py

[NickM-27]

https://deploy-preview-4055--frigate-docs.netlify.app/configuration/ffmpeg_presets

[mattthewclayton]

Hi @NickM-27 - I think I have found an issue with this PR but just wanted to check with you first before I submit a fix:

In the following snippet "detect_args" are never added to "ffmpeg_output_args" and so a lot of my bespoke options under self.ffmpeg.output_args.detect are just thrown away.

    def _get_ffmpeg_cmd(self, ffmpeg_input: CameraInput):
        ffmpeg_output_args = []
        if "detect" in ffmpeg_input.roles:
            detect_args = get_ffmpeg_arg_list(self.ffmpeg.output_args.detect)
            scale_detect_args = parse_preset_hardware_acceleration_scale(
                ffmpeg_input.hwaccel_args or self.ffmpeg.hwaccel_args,
                detect_args,
                self.detect.fps,
                self.detect.width,
                self.detect.height,
            )

            ffmpeg_output_args = scale_detect_args + ffmpeg_output_args + ["pipe:"]

The relevant snippet from my config.yaml:

    ffmpeg:
      hwaccel_args: preset-vaapi
      output_args:
        # Also save detect stream for longer term storage.
        detect: -f segment -segment_time 900 -segment_atclocktime 1 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an /media/frigate/full_recordings/front/front-%Y-%m-%d_%H_%M_%S.mp4 -f rawvideo -pix_fmt yuv420p
      inputs:
        - path: http://192.168.0.21/flv?port=1935&app=bcs&stream=channel0_main.bcs&user=admin&password=redacted
          input_args: preset-http-reolink
          roles:
            - record
        - path: http://192.168.0.21/flv?port=1935&app=bcs&stream=channel0_ext.bcs&user=admin&password=redacted
          input_args: preset-http-reolink
          roles:
            - detect

My fix would simply to just change :

 ffmpeg_output_args = scale_detect_args + ffmpeg_output_args + ["pipe:"]

to

 ffmpeg_output_args = detect_args + scale_detect_args + ffmpeg_output_args + ["pipe:"]

This works for me, but I am not sure if you didn't include detect_args for a good reason?

I also appreciate my usecase is a little advanced. I set Frigate to just save just recordings with events and then I use these detect_args to permanently record everything in 15min length files elsewhere. I do this for good reasons (at least for me).

[NickM-27]

@mattthewclayton yes your detect args are very odd as you are trying to record with detect.

not including the detect args is on purpose because the detect args should not be used in conjunction with the ffmpeg presets given that the output is different depending on the hwaccel type

[mattthewclayton]

@NickM-27 - yes good point about using detect to record, it's because I only wanted to keep long term recordings using the lower quality sub stream but when there is an event I want frigate to capture the high quality stream. I am not sure there is currently a way to achieve this using standard configuration (happy to be told I'm wrong).

I also wanted to limit parallel connections to the camera. I guess I could try and use the new restream functionality and run this long term low quality capture stuff completely separately to frigate.

I won't look at "fixing" this issue - as it's more of a side effect of my odd config. Thanks for your input.

[NickM-27]

@mattthewclayton I did some tests and I have found that it doesn't actually matter if the default detect output args are included in those cases. I am going to put up a PR but just wanted to comment 👍