[Proposal] MediaElement

[brminnick]

MediaElement

• Proposed
• Prototype: Not Started
• Implementation: Not Started
  • iOS Support
  • Android Support
  • macOS Support
  • Windows Support
• Unit Tests: Not Started
• Sample: Not Started
• Documentation: Not Started

Summary

MediaElement is a view for playing video and audio. Media that's supported by the underlying platform can be played from the following sources:

• The web, using a URI (HTTP or HTTPS).
• A resource embedded in the platform application, using the ms-appx:/// URI scheme.
• Files that come from the app's local and temporary data folders, using the ms-appdata:/// URI scheme.
  The device's library.

Detailed Design

MediaElement.shared.cs

public class MediaElement : View, IMediaElementController
{
  public static readonly BindableProperty AspectProperty;
  public static readonly BindableProperty AutoPlayProperty;
  public static readonly BindableProperty BufferingProgressProperty;
  public static readonly BindableProperty CurrentStateProperty;
  public static readonly BindableProperty DurationProperty;
  public static readonly BindableProperty IsLoopingProperty;
  public static readonly BindableProperty KeepScreenOnProperty;
  public static readonly BindableProperty PositionProperty;
  public static readonly BindableProperty ShowsPlaybackControlsProperty;
  public static readonly BindableProperty SourceProperty;
  public static readonly BindableProperty VideoHeightProperty;
  public static readonly BindableProperty VideoWidthProperty;
  public static readonly BindableProperty VolumeProperty;
  public static readonly BindableProperty SpeedProperty;
  
  public double BufferingProgress { get; }
  public bool CanSeek { get; }
  public MediaElementState CurrentState { get; }
  public TimeSpan? Duration { get; }
  public int VideoHeight { get; }
  public int VideoWidth  { get;}
  
  public Aspect Aspect { get; set; }
  public bool AutoPlay { get; set; }
  public bool IsLooping { get; set; }
  public bool KeepScreenOn { get; set; }
  public bool ShowsPlaybackControls { get; set; }
  public TimeSpan Position { get; set; }
  [Forms.TypeConverter(typeof(MediaSourceConverter))]
  public MediaSource? Source { get; set; }
  public double Volume { get; set; }
  public double Speed { get; set; }
  
  public event EventHandler? MediaEnded;
  public event EventHandler? MediaFailed;
  public event EventHandler? MediaOpened;
  public event EventHandler? SeekCompleted;
  
  public void Play() ;
  public void Pause();
  public void Stop();
}

Usage Syntax

[jfversluis]

The MediaElement needs some love, I think what is there now is pretty stable, lots of room for improvement though. But I think a control to play media should actually be part of .NET MAUI in-the-box. While it's not that, I think we should definitely have it in the Toolkit to provide something, hopefully build it out and move it over to .NET MAUI as a mature control.

[pierre01]

MediaElement is essential to many applications and should be added to the Maui.Essentials library the same as haptic feedback is.

[jfversluis]

The big difference is that Essentials does not have any UI elements. It will be implemented here as it is in the Toolkit now and hopefully we can transition it into .NET MAUI itself.

[peterfoot]

I had thought about whether it could be ported to MAUI and if so what changes, if any, could be made. One thing which did occur - should it follow the UWP model which has seen the MediaElement deprecated for a MediaPlayer and MediaPlayerElement. The latter being the UI control and the former having all the mechanics to playback media. It would mean more flexibility for audio only and not needing to add a UI control in those cases. I think the underlying platform APIs in iOS and Android would also suit this model...

[peterfoot]

And I agree with @jfversluis it would be awesome for this control to transition into MAUI itself.

[jfversluis]

@peterfoot if you're up for it, it's yours for the taking here :)

I think timing-wise we won't be able to get this into .NET MAUI straight-away, so let's get it in here first and then we can start making a case to move it for .NET 7.

[emceelovin]

A/V is paramount IMO. This should've been on the timeline from day one facepalm. I'm all for whatever will allow basic playback, but the proposal doesn't mention anything about a Stream as one the source options, and that's the most important IMO. I'm working on a tiny IMediaPlayer interface in my MAUI project to back with native implementations, but maybe I can help on this? I would like to see this ASAP also.

[brminnick]

Thanks @emceelovin! We would love your help with MediaElement 💯

[emceelovin]

@brminnick will do anything I can!

[brminnick]

Adding the blocked label for now.

We agree that MediaElement should support streaming. For Android, supporting streaming requires the use of Google's Exo Player, an open-source library made by Google that enables HLS and DASH on Android.

Currently, there is no net6.0-android binding of Google's ExoPlayer. This is blocking us from adding MediaElement.

I've spoken with David Ortinau and Jon Dick on the .NET MAUI Engineering team about this gap in our feature offering. They are currently evaluating the feasibility of creating an official Microsoft-created ExoPlayer binding for net6.0-android.

For Xamarin.Forms developers, the current recommendation is to use the community-created Xam.Plugins.Android.ExoPlayer NuGet Package by Martijn van Dijk. I use Martijn's ExoPlayer NuGet package in my app that I've published to the App Stores, GitTrends, and it works really well to stream video from Azure Media Services! Here's the ExoPlayer source code in GitTrends for those curious: https://github.com/brminnick/GitTrends/blob/main/GitTrends.Android/CustomRenderers/VideoPlayerViewCustomRenderer.cs

[VladislavAntonyuk]

dotnet/maui#6764

[sturlath]

Please also consider DRM (with/without token) ans AES token encryption for streaming.

[jabak]

Please also consider DRM (with/without token) ans AES token encryption for streaming.

I agree. I would consider this to be part of basic audio playback.

Playing/looping audio from a memstream (c# memstream etc.), an url (https) or a file (beep.wav - beep.mp3) including DRM content is what is needed for common cross platform apps.

[mhrastegari]

Yeah you are right :( I'm really sad right now about how our conversation got there, I was just in too much stress and i should have think more about what i going to comment here.
I'm so sorry dear Gerald you're amazing. ❤️

[jfversluis]

Don't worry! No harm was done, we're still friends and I understand it comes from a good place. But also want to make sure that you have expectations we can't live up to.

Thanks for saying this though, much appreciated!

[Flaechekueste50]

Hi! I just tried out the MediaElement in my project (an audio streaming radio app) and I've got following Exception which I really don't understand at all:

Are you able to tell me what this Exception means and how I can solve the problem?

[jfversluis]

@Flaechekueste50 as above, not really. It's being actively developed, no support is given at this time. If you can give me a source URL that will stay online for a while I can add it to #717 and test it out to see what we can do to make it work.

[aawaheed]

Will there be any support for streaming over UDP?

[mhrastegari]

Hey guys these 3 days we were discovered a serious memory leak in MCT MediaElement in our project and we fixed it in Android and Windows(not tested on macOS and iOS), and just wanna return the favor, specially to @jfversluis :)
So how can i help and contribute?

[pictos]

@mhrastegari thanks for that <3

First you should work against this branch, when you've everything working just open a PR targeting this branch

[mhrastegari]

@pictos Done #781 .

[jfversluis]

Appreciate it @mhrastegari thank you so much!

[PaulVrugt]

Is my assumption correct that this will support a Stream as source? If so, this will finally fix the issue that we have had for ages that we need to be able to pass a custom header to a uri, since the video files we are playing are coming from an api that needs authentication via the authorization header.

[jfversluis]

@PaulVrugt not sure if that will be possible with the first version. What would passing headers through a stream look like? Do you have some pseudo code or a more thorough explanation of what you want to do?

[vynxc]

@PaulVrugt not sure if that will be possible with the first version. What would passing headers through a stream look like? Do you have some pseudo code or a more thorough explanation of what you want to do?

[vynxc]

@PaulVrugt not sure if that will be possible with the first version. What would passing headers through a stream look like? Do you have some pseudo code or a more thorough explanation of what you want to do?

this is hard coded but im sure you get the idea 😁

[ne0rrmatrix]

I have been testing against the nuget package. It has been a great learning experience for me. This is fun! I am working on developing an app to play podcasts from a set of rss feeds from twit.tv.

It is working in Windows and Android. Still a lot for me to do to have it ready for testing though. I am planning on looking into helping out here and fixing the issues I am having with maui. I am new to developing and have never contributed to an open source project. I am hoping to make this my first.

I am going to go and work on fix to a very minor issue with media element. It might take a week or so to see if I have any hope to help with this. The first thing I want to fix is a problem where if you don't specify in media:MediaElement a WidthRequest and HeightRequest that fits within the android screen the ShowPlayBackControls is missing. Also the video does not show. If you do specify the width and height it works. Has horribly stuttering with the video I am playing. It is streaming over the internet. Not going to try and fix that first lol. I am going to work on something small like the UI issue. Wish me luck!

[mhrastegari]

Hey @ne0rrmatrix i believe this can help you for your fullscreen issue.

davidbritch/dotnet-maui-videoplayer#9 (comment)

[JoacimWall]

Hi try to find the source code for the Media Element to test I I can add support for subtiles on Android but can't find the source. Is not the MediaElement open source?

//Thanks Joacim

[bijington]

Hi try to find the source code for the Media Element to test I I can add support for subtiles on Android but can't find the source. Is not the MediaElement open source?

//Thanks Joacim

The code hasn't been merged into main yet. The work is still ongoing under the PR #667

[JoacimWall]

Hi try to find the source code for the Media Element to test I I can add support for subtiles on Android but can't find the source. Is not the MediaElement open source?

What I need is this on the ExoPlayer
this.playerView.SetShowSubtitleButton(true)
Should this be able to solve by a handler from the maui app?
Now has added a property to the MediaPlayer that is only for Android. I saw that ware other property's that was only for a specific plattform.

//Thanks Joacim

The code hasn't been merged into main yet. The work is still ongoing under the PR #667

Thanks
Next question, Now when I have fixed the support for subtiles on android how do I submit this. I can't not figure out if there is any way to create a Pull Request on an already existing PR(667).

//Thanks Joacim

[jfversluis]

@JoacimWall let's wait until MediaPlayer has been merged. That PR is way too big already, I don't want to delay it any further by doing a PR against the PR.

If you really need it right now then you should be able to indeed do it with a custom mapper entry. Something like this:

MediaPlayerHandler.PropertyMapper.AppendToMapping("subtitleMapper", (handler, player) =>
{
#if ANDROID
    handler.PlatformView.GetFirstChildOfType<StyledPlayerView>()?.SetShowSubtitleButton(true);
#endif
});

[JoacimWall]

@JoacimWall let's wait until MediaPlayer has been merged. That PR is way too big already, I don't want to delay it any further by doing a PR against the PR.

If you really need it right now then you should be able to indeed do it with a custom mapper entry. Something like this:

MediaPlayerHandler.PropertyMapper.AppendToMapping("subtitleMapper", (handler, player) =>
{
#if ANDROID
    handler.PlatformView.GetFirstChildOfType<StyledPlayerView>()?.SetShowSubtitleButton(true);
#endif
});

Ok no problem. Thanks for the handler code.
//Joacim

[ne0rrmatrix]

With stuff still in early alpha on this build what sort of things can we do that will be helpful? I am learning about maui, mvvm as quick as I can. I am going through and learning about maui and trying to figure out all the ins and outs so I can contribute by fixing bugs myself. I want to help and provide solutions to issues not just submit bugs. I have all the time in the world and want to contribute. I have learned a lot already. But I am in the process of trying to figure out how to understand how the community toolkit and this project fit together with dotnet maui. I plan to keep figuring it out over the next month.

I have learned how to use the community toolkit to do almost all of things I want. But I need to learn the maui long hand way to contribute and make my life easier I think? Most of the issues I have had with maui so far I have found to be user errors on my part on how I implement something. I am this week going to spend my time going over and doing everything the long way so I can learn the structure of how maui is done without the toolkit. This is in the hopes of making it easier to learn how to contribute to toolkit and understand the structure of the code. I have been going through the toolkit and looking at the code.

[ghost]

Reopening Proposal.

Only Proposals moved to the Closed Project Column and Completed Project Column can be closed.

[Meysamsh1986]

Hello,
I have a problem with CommunityToolkit.Maui.MediaElement
Closed Caption menu (subtitle for videos) is working fine in windows platform, but caption menu does not show in android platform,
How can I fix this issue ?