[Config Support]: Bounding boxes not showing on review or clips

[jaaacck]

Describe the problem you are having

When reviewing clips are you able to see the bounding box around the detected objects? Sometimes I can't find what has been detected so would be good to see what Frigates thinks is an object.

If I go to "https://nvr.domain.com/api/garden/any/clip.mp4?bbox=1" I can successfully see the clip playing but there is no box around the detected person:

On the review page I also can't see any bounding boxes around the object I think Frigate has detected:

If I run the camera via the API (https://nvr.domain.com/api/garden?bbox=1) I can see the bounding boxes, but I want to see it against the previous detections.

I've had a read of the full config example here (https://docs.frigate.video/configuration/reference) but I can't seem to work out how to get these bounding boxes on.

Version

0.14.1-f4f3cfa

Frigate config file

mqtt:
  host: mosquitto

ffmpeg:
  hwaccel_args: preset-nvidia-h264

go2rtc:
  streams:
    garden:
      - rtsp://<username>:<password>@192.168.1.66/Streaming/Channels/101
    side:
      - rtsp://username>:<password>@192.168.1.71/Streaming/Channels/101

  webrtc:
    candidates:
      - 192.168.1.101:8555
      - stun:8555

# Global tracking
objects:
  track:
    - person
    - dog
    - cat
    - car

# GPU based detection
detectors:
  tensorrt:
    type: tensorrt
    device: 0 #This is the default, select the first GPU

model:
  path: /config/model_cache/tensorrt/yolov7-tiny-416.trt
  input_tensor: nchw
  input_pixel_format: rgb
  width: 416
  height: 416

cameras:

  garden:
    ffmpeg:
      inputs:
        - path: rtsp://localhost:8554/garden
          input_args: preset-rtsp-restream
          roles:
            - record
            # - audio
            - detect
          hwaccel_args: preset-nvidia-h264
    detect:
      enabled: true
      width: 1280
      height: 720
      fps: 10
    objects:
      track:
        - person
        - cat
        - dog
        - car
    record:
      enabled: true
      retain:
        days: 7
        mode: all
      events:
        pre_capture: 5
        post_capture: 5
        retain:
          default: 42
          mode: active_objects
    motion:
      mask:
        - 0,0.104,0.314,0.099,0.312,0,0,0
        - 0.605,0,0.727,0,0.721,0.071,0.618,0.069
    snapshots:
      enabled: true
      bounding_box: true
    live:
      stream_name: garden

    zones:
      Main_Road:
        coordinates: 
          0.255,0.165,0.257,0.219,0.277,0.351,0.146,0.393,0.097,0.503,0.099,0.731,0,0.819,0.002,0.569,0.001,0.464
        loitering_time: 0
        inertia: 3
      Outside_house_road:
        coordinates: 0.156,0.397,0.686,0.288,0.767,0.446,0.072,0.558
        loitering_time: 0
      House_Area:
        coordinates: 
          0.095,1,0.106,0.551,0.774,0.453,0.678,0.255,0.88,0.261,1,0.406,0.998,0.647,0.996,0.905,0.954,0.997
        inertia: 3
        loitering_time: 0
    review:
      alerts:
        required_zones:
          - Outside_house_road
          - House_Area
          - Main_Road

  side:
    ffmpeg:
      inputs:
        - path: rtsp://localhost:8554/side
          input_args: preset-rtsp-restream
          roles:
            - record
            # - audio
            - detect
          hwaccel_args: preset-nvidia-h264
    detect:
      enabled: true
      width: 1280
      height: 720
      fps: 10
    objects:
      track:
        - person
        - cat
        - dog
        - car
    record:
      enabled: true
      retain:
        days: 7
        mode: all
      events:
        pre_capture: 5
        post_capture: 5
        retain:
          default: 42
          mode: active_objects
    snapshots:
      enabled: true
      bounding_box: true
    live:
      stream_name: side

version: 0.14

Relevant Frigate log output

None

Relevant go2rtc log output

None

Frigate stats

cameras:
  garden:
    audio_dBFS: 0
    audio_rms: 0
    camera_fps: 10.1
    capture_pid: 436
    detection_enabled: true
    detection_fps: 0.1
    ffmpeg_pid: 447
    pid: 421
    process_fps: 10.1
    skipped_fps: 0
  side:
    audio_dBFS: 0
    audio_rms: 0
    camera_fps: 10
    capture_pid: 441
    detection_enabled: true
    detection_fps: 0
    ffmpeg_pid: 450
    pid: 428
    process_fps: 10.1
    skipped_fps: 0
cpu_usages:
  '1':
    cmdline: /package/admin/s6/command/s6-svscan -d4 -- /run/service
    cpu: '0.0'
    cpu_average: '0'
    mem: '0.0'
  '15':
    cmdline: s6-supervise s6-linux-init-shutdownd
    cpu: '0.0'
    cpu_average: '0'
    mem: '0.0'
  '16':
    cmdline: >-
      /package/admin/s6-linux-init/command/s6-linux-init-shutdownd -c
      /run/s6/basedir -g 3000 -C -B
    cpu: '0.0'
    cpu_average: '0'
    mem: '0.0'
  '24':
    cmdline: s6-supervise s6rc-oneshot-runner
    cpu: '0.0'
    cpu_average: '0'
    mem: '0.0'
  '25':
    cmdline: s6-supervise s6rc-fdholder
    cpu: '0.0'
    cpu_average: '0'
    mem: '0.0'
  '26':
    cmdline: s6-supervise certsync
    cpu: '0.0'
    cpu_average: '0'
    mem: '0.0'
  '27':
    cmdline: s6-supervise certsync-log
    cpu: '0.0'
    cpu_average: '0'
    mem: '0.0'
  '28':
    cmdline: s6-supervise frigate
    cpu: '0.0'
    cpu_average: '0'
    mem: '0.0'
  '29':
    cmdline: s6-supervise frigate-log
    cpu: '0.0'
    cpu_average: '0'
    mem: '0.0'
  '30':
    cmdline: s6-supervise go2rtc
    cpu: '0.0'
    cpu_average: '0'
    mem: '0.0'
  '31':
    cmdline: s6-supervise go2rtc-healthcheck
    cpu: '0.0'
    cpu_average: '0'
    mem: '0.0'
  '32':
    cmdline: s6-supervise go2rtc-log
    cpu: '0.0'
    cpu_average: '0'
    mem: '0.0'
  '33':
    cmdline: s6-supervise nginx
    cpu: '0.0'
    cpu_average: '0'
    mem: '0.0'
  '34':
    cmdline: s6-supervise nginx-log
    cpu: '0.0'
    cpu_average: '0'
    mem: '0.0'
  '45':
    cmdline: >-
      /package/admin/s6/command/s6-ipcserverd -1 --
      /package/admin/s6/command/s6-ipcserver-access -v0 -E -l0 -i data/rules --
      /package/admin/s6/command/s6-sudod -t 30000 --
      /package/admin/s6-rc/command/s6-rc-oneshot-run -l ../.. --
    cpu: '0.0'
    cpu_average: '0'
    mem: '0.0'
  '46':
    cmdline: /package/admin/s6-2.11.3.2/command/s6-fdholderd -1 -i data/rules
    cpu: '0.0'
    cpu_average: '0'
    mem: '0.0'
  '93':
    cmdline: s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/nginx
    cpu: '0.0'
    cpu_average: '0'
    mem: '0.0'
  '94':
    cmdline: s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/go2rtc
    cpu: '0.0'
    cpu_average: '0'
    mem: '0.0'
  '95':
    cmdline: s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/frigate
    cpu: '0.0'
    cpu_average: '0'
    mem: '0.0'
  '97':
    cmdline: s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/certsync
    cpu: '0.0'
    cpu_average: '0'
    mem: '0.0'
  '105':
    cmdline: /usr/local/go2rtc/bin/go2rtc -config=/dev/shm/go2rtc.yaml
    cpu: '4.8'
    cpu_average: '5'
    mem: '0.1'
  '113':
    cmdline: bash ./run.user go2rtc-healthcheck
    cpu: '0.0'
    cpu_average: '0'
    mem: '0.0'
  '114':
    cmdline: python3 -u -m frigate
    cpu: '3.0'
    cpu_average: '2'
    mem: '1.9'
  '135':
    cmdline: 'nginx: master process nginx'
    cpu: '0.0'
    cpu_average: '0'
    mem: '0.0'
  '178':
    cmdline: 'nginx: worker process'
    cpu: '0.0'
    cpu_average: '0'
    mem: '0.1'
  '179':
    cmdline: 'nginx: worker process'
    cpu: '0.0'
    cpu_average: '0'
    mem: '0.1'
  '180':
    cmdline: 'nginx: worker process'
    cpu: '0.0'
    cpu_average: '0'
    mem: '0.1'
  '181':
    cmdline: 'nginx: worker process'
    cpu: '0.1'
    cpu_average: '0'
    mem: '0.1'
  '182':
    cmdline: 'nginx: cache manager process'
    cpu: '0.0'
    cpu_average: '0'
    mem: '0.0'
  '344':
    cmdline: bash ./run.user certsync
    cpu: '0.0'
    cpu_average: '0'
    mem: '0.0'
  '373':
    cmdline: 'frigate.logger       '
    cpu: '0.0'
    cpu_average: '0'
    mem: '0.5'
  '375':
    cmdline: frigate.recording_manager
    cpu: '1.3'
    cpu_average: '1'
    mem: '0.7'
  '384':
    cmdline: frigate.review_segment_manager
    cpu: '0.6'
    cpu_average: '0'
    mem: '0.6'
  '403':
    cmdline: >-
      /usr/bin/python3 -c from multiprocessing.resource_tracker import
      main;main(59)
    cpu: '0.3'
    cpu_average: '0'
    mem: '0.0'
  '404':
    cmdline: frigate.detector.tensorrt
    cpu: '0.6'
    cpu_average: '1'
    mem: '1.2'
  '406':
    cmdline: 'frigate.output       '
    cpu: '3.3'
    cpu_average: '3'
    mem: '0.6'
  '421':
    cmdline: 'frigate.process:garden'
    cpu: '4.5'
    cpu_average: '7'
    mem: '0.7'
  '428':
    cmdline: 'frigate.process:side '
    cpu: '3.5'
    cpu_average: '7'
    mem: '0.7'
  '436':
    cmdline: 'frigate.capture:garden'
    cpu: '4.9'
    cpu_average: '5'
    mem: '0.6'
  '441':
    cmdline: 'frigate.capture:side '
    cpu: '5.3'
    cpu_average: '5'
    mem: '0.6'
  '447':
    cmdline: >-
      ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel cuda
      -hwaccel_output_format cuda -user_agent FFmpeg Frigate/0.14.1-f4f3cfa
      -rtsp_transport tcp -timeout 5000000 -i rtsp://localhost:8554/garden -f
      segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime
      1 -c copy -an /tmp/cache/garden@%Y%m%d%H%M%S%z.mp4 -r 10 -vf
      fps=10,scale_cuda=w=1280:h=720:format=nv12,hwdownload,format=nv12,format=yuv420p
      -threads 2 -f rawvideo -pix_fmt yuv420p pipe:
    cpu: '8.7'
    cpu_average: '8'
    mem: '0.7'
  '450':
    cmdline: >-
      ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel cuda
      -hwaccel_output_format cuda -user_agent FFmpeg Frigate/0.14.1-f4f3cfa
      -rtsp_transport tcp -timeout 5000000 -i rtsp://localhost:8554/side -f
      segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime
      1 -c copy -an /tmp/cache/side@%Y%m%d%H%M%S%z.mp4 -r 10 -vf
      fps=10,scale_cuda=w=1280:h=720:format=nv12,hwdownload,format=nv12,format=yuv420p
      -threads 2 -f rawvideo -pix_fmt yuv420p pipe:
    cpu: '8.1'
    cpu_average: '8'
    mem: '0.7'
  '469':
    cmdline: >-
      ffmpeg -threads 1 -f rawvideo -pix_fmt yuv420p -video_size 1280x720 -i
      pipe: -threads 1 -f mpegts -s 1280x720 -codec:v mpeg1video -q 8 -bf 0
      pipe:
    cpu: '0.0'
    cpu_average: '0'
    mem: '0.1'
  '475':
    cmdline: >-
      ffmpeg -threads 1 -f rawvideo -pix_fmt yuv420p -video_size 1280x720 -i
      pipe: -threads 1 -f mpegts -s 1280x720 -codec:v mpeg1video -q 8 -bf 0
      pipe:
    cpu: '0.0'
    cpu_average: '0'
    mem: '0.1'
  '480':
    cmdline: >-
      ffmpeg -threads 1 -f rawvideo -pix_fmt yuv420p -video_size 1280x720 -i
      pipe: -threads 1 -f mpegts -s 1280x720 -codec:v mpeg1video -q 8 -bf 0
      pipe:
    cpu: '0.0'
    cpu_average: '0'
    mem: '0.0'
  '3172':
    cmdline: >-
      /usr/bin/python3 -c from multiprocessing.resource_tracker import
      main;main(48)
    cpu: '0.0'
    cpu_average: '0'
    mem: '0.0'
  '27131':
    cmdline: sleep 60
    cpu: '0.0'
    cpu_average: '0'
    mem: '0.0'
  '27250':
    cmdline: sleep 30s
    cpu: '0.0'
    cpu_average: '0'
    mem: '0.0'
  frigate.full_system:
    cpu: '10.9'
    mem: '35.8'
detection_fps: 0.1
detectors:
  tensorrt:
    detection_start: 0
    inference_speed: 5.46
    pid: 404
gpu_usages:
  NVIDIA GeForce GTX 1060 6GB:
    dec: 10.0%
    enc: 0.0%
    gpu: 1.0%
    mem: 12.58%
processes:
  go2rtc:
    pid: 105
  logger:
    pid: 373
  recording:
    pid: 375
  review_segment:
    pid: 384
service:
  last_updated: 1732096726
  latest_version: 0.14.1
  storage:
    /dev/shm:
      free: 122.5
      mount_type: tmpfs
      total: 128
      used: 5.5
    /media/frigate/clips:
      free: 263776.3
      mount_type: fuse.shfs
      total: 8820444.9
      used: 8554803.6
    /media/frigate/recordings:
      free: 263776.3
      mount_type: fuse.shfs
      total: 8820444.9
      used: 8554803.6
    /tmp/cache:
      free: 940.6
      mount_type: tmpfs
      total: 953.7
      used: 13.1
  temperatures: {}
  uptime: 164173
  version: 0.14.1-f4f3cfa

Operating system

UNRAID

Install method

Docker Compose

docker-compose file or Docker CLI command

version: "3.9"
services:
  frigate:
    container_name: frigate
    privileged: true
    restart: unless-stopped
    image: "ghcr.io/blakeblackshear/frigate:0.14.1-tensorrt"
    shm_size: "128mb"
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /mnt/user/appdata/frigate/config:/config
      - /mnt/user/security_cams:/media/frigate
      - type: tmpfs
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    ports:
      - "5000:5000"
      - "1935:1935" # RTMP feeds
      - "8554:8554" # Restreams
    environment:
      FRIGATE_RTSP_PASSWORD: "<password>"
      YOLO_MODELS: "yolov7-tiny-416"
    networks:
      - nginx_web
      - backend
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

  mosquitto:
    container_name: mosquitto
    restart: unless-stopped
    image: eclipse-mosquitto
    ports:
      - "1883:1883" #default mqtt port
      - "9001:9001" #default mqtt port for websockets
    volumes:
      - /mnt/user/appdata/mosquitto/config:/mosquitto/config
      - /mnt/user/appdata/mosquitto/data:/mosquitto/data
      - /mnt/user/appdata/mosquitto/log:/mosquitto/log
    networks:
      - backend


networks:
  backend:
  nginx_web:
    external: true

Object Detector

TensorRT

Screenshots of the Frigate UI's System metrics pages

Any other information that may be helpful

No response

[hawkeye217]

Adding ?bbox=1 to the clip.mp4 endpoint is not valid. Frigate does not re-annotate recording segments through the API.

Frigate does provide an mjpeg stream for debugging where you can add bounding boxes and see them in real-time: https://docs.frigate.video/integrations/api/#get-apicamera_name

You can also get the latest image from your camera as a jpeg annotated with bounding boxes: https://docs.frigate.video/integrations/api/#get-apicamera_namelatestjpgh300

To see bounding boxes for previous detections, you'll need to use snapshots (https://docs.frigate.video/integrations/api/#get-apieventsidsnapshotjpg) or use the Object Lifecycle feature, available in Frigate 0.15.0, currently in beta.

[jaaacck]

I've moved to the beta and it has the feature I wanted as you suggested