Start streaming parts before first completed segment in HLS Low Latency

[Jordy84]

In #79 I disabled the randomAccess check to start streaming HLS low latency parts before the first segment was completed. This enabled us to display the video feed much faster as our video source GOP size is 20seconds.

The PR was rejected because it was creating segments without a starting IDR frame.

This PR solves the same problem in a (hopefully) supported manner.

[codecov]

Codecov Report

Merging #92 (0744515) into main (d7d2f4b) will decrease coverage by 0.27%.
Report is 1 commits behind head on main.
The diff coverage is 71.68%.

❗ Current head 0744515 differs from pull request most recent head b2fdf4c. Consider uploading reports for the commit b2fdf4c to get more accurate results

@@            Coverage Diff             @@
##             main      #92      +/-   ##
==========================================
- Coverage   68.92%   68.65%   -0.27%     
==========================================
  Files          42       42              
  Lines        3652     3717      +65     
==========================================
+ Hits         2517     2552      +35     
- Misses        954      981      +27     
- Partials      181      184       +3     

Files Changed	Coverage Δ
muxer_segment.go	62.50% <0.00%> (-8.93%)	⬇️
muxer_segment_mpegts.go	80.89% <0.00%> (-1.86%)	⬇️
pkg/storage/file_disk.go	76.00% <0.00%> (-12.38%)	⬇️
pkg/storage/file_ram.go	68.96% <0.00%> (-21.95%)	⬇️
muxer.go	36.82% <66.66%> (+0.93%)	⬆️
muxer_segment_fmp4.go	71.07% <84.21%> (-1.10%)	⬇️
muxer_server.go	70.68% <86.66%> (+0.13%)	⬆️
muxer_segmenter_fmp4.go	70.63% <100.00%> (+0.78%)	⬆️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[aler9]

Hello, by looking at the code, you want to make the playlist available as soon as a certain number of parts is available, is that correct?

This approach may work, but did you test it with iOS?

[Jordy84]

Hi @aler9, that is correct. When we release the playlist as soon as a certain number of parts are available, we are able to quickly show the stream even with a GOP size of 20seconds.

I tried my best testing this in the iOS simulator on my Mac using a downloaded video published by ffmpeg. The behaviour is the same as with the main branch only faster. But using both the main branch as my branch the simulator doesn't show video but it does request the correct files.

Maybe you have some tests you normally do in iOS?

[Jordy84]

Hi @aler9,

Is there something I can do to help getting this PR tested/completed? Maybe you know of some tests I can do or write?

Thanks!

[aler9]

Hello, as previously written, i have to test this on iOS.

[jordybsecurite]

Ok thanks! Will wait for your results. Hopefully it all works :)

[aler9]

While this approach works with hls.js, it doesn't work on iOS, that requires at least a complete segment before serving the stream.

There's an alternative way to make video appear immediately (without even waiting DurationRequiredPartCount, that would become redundant), that i had in mind for some time, it's just a little difficult to implement, and consists in in replacing gaps with video files that contain black frames, encoded with the same codec of the stream. It would allow immediate playback on any device.

Basically you have to store into the server blank frames encoded with AV1, VP9, H264, H265, generate a 1-second long MP4 segment that contains frames correspondent to the current codec, and serve it instead of gaps. In this way, a playlist is available immediately in full.

I don't know if or when this will be implemented, a PR would be welcome.

[Jordy84]

That's really unfortunate.

About the proposed solution, do you mean instead of sending a playlist with 7 non-existent gap.mp4 files, we should immediately send actual blank mp4 files with a length of 1 sec? Seems doable!

Is it an option to add this PR under a feature flag (enable it using the mediamtx.yaml file) while we develop the other proposed solution?

Our application really needs the faster startup time right now and we would rather keep using the official images instead of creating and building our own forks. Of course we will commit to developing the more compatible solution

[sourcez]

Hi, we experience similar problems so this feature would be a great asset. Is there any progress on this topic ?