RTSP stream failing to record due to missing IDR or non-IDR NALU

[gkubes]

Which version are you using?

v1.8.4

Which operating system are you using?

• Linux amd64 standard
• Linux amd64 Docker
• Linux arm64 standard
• Linux arm64 Docker
• Linux arm7 standard
• Linux arm7 Docker
• Linux arm6 standard
• Linux arm6 Docker
• Windows amd64 standard
• Windows amd64 Docker (WSL backend)
• macOS amd64 standard
• macOS amd64 Docker
• Other (please describe)

Describe the issue

Unable to record RTSP stream to disk in mp4 format. Continually see errors indicating the video stream does not have an IDR or non-IDR NALU and then proceeds to immediately create and close a segment. As a result, the segments on disk are < 1sec, only show a black images with the the correct audio.

I am able to view the same camera RTSP stream using a WebRTC client.

Describe how to replicate the issue

1. start the server
2. proxy path with rtsp stream using h264 yuv420p video and pcm_alaw audio. I'm using Tapo C120 camera.
3. enable record in fmp4 format.

Did you attach the server logs?

yes

mediamtx.log

Did you attach a network dump?

yes

capture-pcap.zip

[gkubes]

Diving head first into the H.264 spec and analyzing my pcap file.

It appears that my camera is sending Fragmentation Units A (NALU #28) as opposed to the expected IDR (NALU Type #5) or non-IDR (NALU Type #1) as implemented by dts_extractor.go. These FU-A allows fragmenting a NAL unit into several RTP packets. I see the completion end marker of the FU-A followed by Supplemental Enhancement Information (SEI) before finally getting the expected non-IDR NALU Types.

[gkubes]

I built out mediamtx from source and confirmed that dts_extrator.go is returning the above error in the case when the Access Unit contains a single SEI NALU.

The question becomes: should the SEI be incorporated into dts_extractor algorithm or should it be skipped and simply return the previous DTS (the same value can be handled given bluenviron/mediacommon#119)?

I found the following documentation from the h.264 spec and it seems like skipping the NALUwould be sufficient and have also confirmed that my video is then recorded.

If the NAL unit has no timing properties of its own (e.g., parameter set and SEI NAL units), the RTP timestamp is set to the RTP timestamp of the primary coded picture of the access unit in which the NAL unit is included, according to Section 7.4.1.2 of [1].

https://datatracker.ietf.org/doc/html/rfc6184#ref-1

[aler9]

Fixed by bluenviron/mediacommon#142