Add optional track synchronization in cmaf muxer #101
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Qizot
force-pushed
the
cmaf-muxer-tracks-synchronization
branch
from
5d98c7a
to
02ec1f4
Compare
mat-hek
requested changes
Feb 12, 2024
mat-hek
reviewed
Feb 16, 2024
mat-hek
approved these changes
Feb 19, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Introduction
This PR introduces a new mechanism to a CMAF muxer allowing to synchronize audio/video tracks but without muxing them together.
Problem
Protocols such as HLS/DASH allow audio/video tracks to be served separately but this requires them to have the same segment boundaries.
With the previous muxer's implementation to serve 2 separate tracks one needed 2 separate muxer instances. This solution had a problem, fluctuating video stream could cause desynchronization with audio track (segments could be created faster/slower causing a sequence numbers mismatch) as the synchronization was done purely through a target segment duration (unlike when muxing together, where the video is taken into account when assembling audio segment part).
Solution
This PR replaces the default static
:outputpad with a dynamic one and introduces a:trackspad option that tells which input pads should be mixed together on the output. Specifying 2 different output pads will result in 2 separate tracks that should be synchronized with each other.