How to create a custom MediaSource?

[stevendesu]

[REQUIRED] Searched documentation and issues

I've looked at a few similar questions, but none had a resolution that I feel will work for me:

• Creating Custom Media Source, to Mimic the performance of AdMediaSource with Custom Ad Servers #3967 - creating an AdsLoader was recommended for handling custom ad logic
• Create MediaSource from FileDescriptor #3757 - no solution was given, although the issue seems to be related to playing a supported file type, just in a non-supported way
• Build MediaSource from InputStream (without any Uri) #4149 - because the issue revolved around playing a single media item with a custom binary format, creating a custom Extractor was recommended

Along with the issues linked above, I've check the ExoPlayer documentation on MediaSources and asked on StackOverflow. I've also experimented a bit in Android Studio, attempting to solve the problem myself through trial and error and probing the relevant objects. In this way I found that addEventListener() will be called, followed by prepareSource(), and then thousands of calls are made to maybeThrowSourceInfoRefreshError()

[REQUIRED] Question

Our company serves an XML file describing the content to be played (example: https://vcloud.blueframetech.com/api/broadcast/vmap/79999)

This XML file, among other things, gives a link to a network ID (a short 4-6 second video advertising the owner of the content), a link to the content itself (always HLS, sometimes live and sometimes archived), and a list of ad tags for playing pre-roll and mid-roll ad breaks.

We provide a web-based player and Android and iOS apps to our clients that can parse this XML file and properly play the content (example: http://njcaatv.com/landing/index).

Historically we were very hands-on with the implementation of these players. If someone wanted a custom app with their own graphics, layout, and features - we would create the app for them. If someone wanted a custom website to search through our broadcasts and play them, we would create it for them. This year we are working on converting our players into stand-alone libraries so that our clients can make their own apps and their own websites.

For the web player this is already complete. Using VideoJS we've created a single JavaScript file that can be imported into a website. You can then pass one of our XML file URLs to this player as a video source and it will download the XML, parse out all necessary information, and properly play the network ID, content, and ads.

I'm now working on the Android player, and wish to base it on ExoPlayer. To have similar ease-of-use to our web player, I had hoped to treat our XML files as a new type of media source. This way someone can simply call .prepare(url) and .setPlayWhenReady(true) on a BFPlayer instance (where BFPlayer extends ExoPlayer) and it will parse out all necessary information and properly play the network ID, content, and ads.

I believe the proper way to implement this in ExoPlayer is using a custom MediaSource. Just as an M3U8 contains no video data but provides links to the TS files (which do contain video data), our XML file will contain no video data but provide links (sometimes to MP4 files in the case of the network ID, sometimes to M3U8 files). Therefore this feels like it's at the same level of abstraction as the HLSMediaSource, SsMediaSource, and DashMediaSource.

If I were to build a custom MediaSource for this purpose, how would I go about it? What is expected of each method, what events do I need to fire, and how do I integrate with the DataSource and Renderer to ultimately display everything properly?

As a side note: when ads are playing, I will need to continue to download and parse the content in the background to extract metadata. This metadata is used to signal our player to perform certain actions synchronized with the content, including ending ad breaks early and resuming content. On the web player this is fairly trivial because ads are played in a separate <video> instance, running in an <iframe>, overlaid on top of the content <video> player. However when reading #3967 it was mentioned that the content and ads should share a player. In this case, is there a way to ensure that we continue to download content and monitor it for metadata?

[tonihei]

Thanks for doing a detailed research first. Based on your problem description, it seems you need indeed a custom MediaSource.

There are 2 main approaches to writing a MediaSource:

1. Extending BaseMediaSource - This is for 'leaf' media source which handle everything themselves (including reading the data, setting up sample streams, etc.). For example HlsMediaSource or ProgressiveMediaSource.
2. Extending CompositeMediaSource<T> - This is for 'wrapper' media sources which essentially forward everything to other media sources. For example, ConcatenatingMediaSource or AdsMediaSource.

If I understand your expected content correctly, you would eventually like to use HlsMediaSource for the main content and (most likely) ProgressiveMediaSource for the intro and the ad breaks. That means the second option (CompositeMediaSource) is probably the best option for your case.

There are also two options how this composition may look like:

1. You can wrap a ConcatenatingMediaSource and update its playlist with your custom content as needed. That's probably easier to set-up, but you can't use ExoPlayer's ads structure to make things like player.isAd() work correctly, or to allow to skip failing ads, etc.
2. You can also wrap the MediaSources for your media parts directly. This gives you more flexibility, but required slightly more effort to set up.

Either way, you need to override the following methods:

• prepareSourceInternal = Called to prepare the media source. Make sure to call the super method as well. This is the point where you should start loading your XML config file on a background thread. The best way is to use Loader and ParsingLoadable in the same way as e.g. the SmoothStreaming manifest is loaded in SsMediaSource.
  • When using the Loader as suggested, you also need to implement the Loader.Callback interface and listen to updates (completed, failed, cancelled). Again, see SsMediaSource for an example.
  • After the XML load completed successfully, you need to set-up the media structure based on this XML 'manifest' using either of the two options above. To add a wrapped source to the CompositeMediaSource, call prepareChildSource(id, mediaSource) with some arbitrary id. This id also has a custom type parameter, so you can really use anything that is suitable to identify the sources. If you use the ConcatenatingMediaSource approach, you may as well use null of type T=void as you only have one child media source anyway.
• onChildSourceInfoRefreshed = This method is called once a child source is ready. Each child source tells you its Timeline (i.e its media structure). If you have multiple child sources, it's probably best to wait for all child sources before telling the player how the media looks like, but that's up to you. Once you are ready to publish the media structure, you need to create a Timeline object yourself describing the media. See the various examples in our code for how this may look like. Again, ConcatenatingMediaSource makes this easier as you just need to forward the already concatenated Timeline. Finally, tell the player of your Timeline by calling refreshSourceInfo(timeline, manifest).
• maybeThrowSourceInfoRefreshError = This method is repeatedly called (yes, even thousands of times) while the media source has not published its first Timeline. You should throw any pending exceptions in here. For example, call Loader.maybeThrowError to throw load errors from your XML load, but also call the super method, which automatically throws all child source related errors.
• releaseSourceInternal = Please release your Loader class in here and also call the super method to automatically release any child media sources.
• createPeriod = Forward this call to the relevant child media source. The id parameter tells you which part of your media this is about as it contains a reference to the ids in the Timeline structure.
• releasePeriod = The same, just make sure to use the right child media source. For the ConcatenatingMediaSource variant, this is again simple, because you only have this one child.
• getWindowIndexForChildWindowIndex / getMediaPeriodIdForChildMediaPeriodId / getMediaTimeForChildMediaTime = This is for correct event reporting. It gets a window index/media period/media time in one of the child Timeline objects and asks you to translate that to your published Timeline. This is not needed for the ConcatenatingMediaSource approach as it's already handled in there. Also, only override the methods which actually change something.

If you want to refresh your XML from time to time to get updates, just restart the Loader approach any time you like. Also note that the onChildSourceInfoRefreshed method may be called any time when a child sources changes (e.g. for live stream updates) and you should update your Timeline accordingly each time.

when ads are playing, I will need to continue to download and parse the content in the background to extract metadata.

Do you mean, you need to actually download the media data to get these updates? The usual approach would be to signal that out-of-band, e.g. through an update in your XML file. ExoPlayer does not load two streams in parallel. It only loads the next stream required for playback and then starts preloading the following stream to get smooth transitions.

[tonihei]

Closing, because the question was answered in detail.