ScreenCaptureRequested draft

specs/ScreenCaptureRequested

@@ -0,0 +1,394 @@
+# Background
+
+JavaScript's Screen Capture API `GetDisplayMedia` allows developers to get a video stream of a user's tabs, windows, or desktop. This API is available in WebView2, but the current default UI has some problems that we need to fix to make sure that hybrid apps using WebView2 have a more seamless web/native experience. This includes removing the tab column in the UI, replacing default strings and icons that do not match in WV2, and potentially having the ability to customize the UI itself. 
+
+These apps also expect that the screen capture dialog has an event before the UI is shown to give the host app an opportunity to block or allow UI from showing at all.
+
+In this document we describe the updated API. We'd appreciate your feedback.
+
+# Description
+
+We propose introducing the `ScreenCaptureRequested` event. This event will be raised whenever the WebView2 and/or iframe corresponding to the CoreWebView2Frame or any of its descendant iframes requests permission to use the Screen Capture API before the UI is shown. 
+
+To maintain backwards compatibility, by default we plan to raise
+`ScreenCaptureRequested` on both `CoreWebView2Frame` and `CoreWebView2`. The
+`CoreWebView2Frame` event handlers will be invoked first,
+before the `CoreWebView2` event handlers. If `Handled` is set true as part of
+the `CoreWebView2Frame` event handlers, then the `ScreenCaptureRequested` event
+will not be raised on the `CoreWebView2`, and its event handlers will not be
+invoked.
+
+In the case of a nested iframe requesting permission, we will raise the event
+off of the top level iframe.
+
+# Examples
+## C++: Registering Screen Requested Handler on CoreWebView2
+``` cpp
+wil::com_ptr<ICoreWebView2> m_webviewEventSource;
+EventRegistrationToken m_screenCaptureRequestedToken = {};
+
+m_webviewEventSource->add_ScreenCaptureRequested(
+    Callback<ICoreWebView2ScreenCaptureRequestedEventHandler>(
+        [this](ICoreWebView2* sender, ICoreWebView2ScreenCaptureRequestedEventArgs* args)
+          -> HRESULT 
+            { 
+                // Get Frame Info
+                wil::com_ptr<ICoreWebView2FrameInfo> frameInfo;
+                CHECK_FAILURE(args->get_FrameInfo(&frameInfo));
+
+                // Frame Name
+                wil::unique_cotaskmem_string frameName;
+                CHECK_FAILURE(frameInfo->get_Name(&nameRaw));
+
+                // Frame Source
+                wil::unique_cotaskmem_string frameSource;
+                CHECK_FAILURE(frameInfo->get_Source(&frameSource));
+
+                // Frame ID
+                wil::com_ptr<ICoreWebView2StagingFrameInfo> frameInfoStaging;
+                CHECK_FAILURE(frameInfo->QueryInterface(IID_PPV_ARGS(&frameInfoStaging)));
+                UINT32 frameId = 0;
+                frameInfoStaging->get_FrameId(&frameId);
+
+                // If the host app wants to cancel the request
+                CHECK_FAILURE(args->put_Cancel(TRUE));
+
+                return S_OK;
+            })
+            .Get(),
+  &m_screenCaptureRequestedToken);
+
+```
+## C++: Registering Screen Requested Handler on CoreWebView2Frame
+``` cpp
+wil::com_ptr<ICoreWebView2> m_webview;
+auto webview4 = m_webview.try_query<ICoreWebView2_4>();
+if (webview4) 
+{
+    EventRegistrationToken m_frameCreatedToken = {};
+    EventRegistrationToken m_screenCaptureRequestedToken = {};
+
+    CHECK_FAILURE(webview4->add_FrameCreated(
+        Callback<ICoreWebView2FrameCreatedEventHandler>(
+            [this](ICoreWebView2* sender, ICoreWebView2FrameCreatedEventArgs* args) 
+              -> HRESULT
+            {
+                wil::com_ptr<ICoreWebView2Frame> webviewFrame;
+                CHECK_FAILURE(args->get_Frame(&webviewFrame));
+
+                auto webviewFrame3 = webviewFrame.try_query<ICoreWebView2Frame3>();
+                if (webviewFrame3)
+                {
+                    CHECK_FAILURE(webviewFrame3->add_ScreenCaptureRequested(
+                        Callback<ICoreWebView3FrameScreenCaptureRequestedEventHandler>(
+                            [this](ICoreWebView3Frame* sender,
+                               ICoreWebView2ScreenCaptureRequestedEventArgs* args) -> HRESULT
+                            {
+
+                                    // Get Frame Info
+                                    wil::com_ptr<ICoreWebView2FrameInfo> frameInfo;
+                                    CHECK_FAILURE(args->get_FrameInfo(&frameInfo));
+
+                                    // Frame Name
+                                    wil::unique_cotaskmem_string frameName;
+                                    CHECK_FAILURE(frameInfo->get_Name(&frameName));
+
+                                    // Frame Source
+                                    wil::unique_cotaskmem_string frameSource;
+                                    CHECK_FAILURE(frameInfo->get_Source(&frameSource));
+
+                                    // Frame ID
+                                    wil::com_ptr<ICoreWebView2StagingFrameInfo> frameInfoStaging;
+                                    CHECK_FAILURE(frameInfo->QueryInterface(IID_PPV_ARGS(&frameInfoStaging)));
+                                    UINT32 frameId = 0;
+                                    frameInfoStaging->get_FrameId(&frameId);
+
+                                    // If the host app wants to cancel the request
+                                    CHECK_FAILURE(args->put_Cancel(TRUE));
+
+                                    // Let CoreWebView2 handler know the event is already handled
+                                    PutHandled(args);
+                                    return S_OK;
+                            })
+                            .Get(),
+                        &m_screenCaptureRequestedToken));
+                }
+                return S_OK;
+        }).Get(),
+        &m_FrameCreatedToken));
+}
+
+static void PutHandled(ICoreWebView2ScreenCaptureRequestedEventArgs* args)
+{
+    // In the case of an iframe requesting screen capture permission, the default behavior is
+    // to first raise the ScreenCaptureRequested event off of the CoreWebView2Frame
+    // and invoke it's handlers, and then raise the event off the CoreWebView2
+    // and invoke it's handlers. However, If we set Handled to true on the
+    // CoreWebView2Frame event handler, then we will not raise the
+    // ScreenCaptureRequested event off the CoreWebView2.
+    if (args)
+    {
+        CHECK_FAILURE(args->put_Handled(true));
+    }
+}
+```
+
+## C#: Registering Screen Capture Requested Handler
+```c#
+private WebView2 webView;
+webView.CoreWebView2.ScreenCaptureRequested += (sender, screenCaptureArgs) =>
+{   
+    // Get Frame Info
+    wil::com_ptr<ICoreWebView2FrameInfo> frameInfo;
+    frameInfo = screenCaptureArgs.frameInfo
+
+    // Frame Name
+    string frameName;
+    frameName = frameInfo.Name;
+
+    // Frame Source
+    string frameSource;
+    frameSource = frameInfo.Source;
+
+    // Frame ID
+    UInt32 frameId = 0;
+    frameId = frameInfo.FrameId;
+
+    // If the host app wants to cancel the request from a specific frame
+    if ( frameId == 1 || frameSource == "https://developer.microsoft.com/en-us/microsoft-edge/webview2/"
+            || frameName == "BlockScreenCapture")
+    {
+        screenCaptureArgs.Cancel = true;
+    }
+}
+```
+
+
+## C#: Registering IFrame Permission Requested Handler
+```c#
+private WebView2 webView;
+
+m_webview.CoreWebView2.FrameCreated += (sender, frameCreatedArgs) =>
+{
+    // Checking for runtime support of CoreWebView2Frame.ScreenCaptureRequested
+    try
+    {
+        frameCreatedArgs.Frame.ScreenCaptureRequested += (frameSender, screenCaptureArgs) =>
+        {
+            // Get Frame Info
+            wil::com_ptr<ICoreWebView2FrameInfo> frameInfo;
+            frameInfo = screenCaptureArgs.frameInfo
+
+            // Frame Name
+            string frameName;
+            frameName = frameInfo.Name;
+
+            // Frame Source
+            string frameSource;
+            frameSource = frameInfo.Source;
+
+            // Frame ID
+            UInt32 frameId = 0;
+            frameId = frameInfo.FrameId;
+
+            // If the host app wants to cancel the request from a specific frame
+            if ( frameId == 1 || frameSource == "https://developer.microsoft.com/en-us/microsoft-edge/webview2/"
+                    || frameName == "BlockScreenCapture")
+            {
+                screenCaptureArgs.Cancel = true;
+            }
+
+            // Let CoreWebView2 handler know the event is already handled
+            PutHandled(screenCaptureArgs);
+        };
+    }
+    catch (NotImplementedException exception)
+    {
+        // If the runtime support is not there we probably want this
+        // to be a no-op.
+    }
+};
+
+
+
+void PutHandled(CoreWebView2ScreenCaptureEventArgs args)
+{
+    // In the case of an iframe requesting permission to use Screen Capture, the default behavior is
+    // to first raise the ScreenCaptureRequested event off of the CoreWebView2Frame
+    // and invoke it's handlers, and then raise the event off the CoreWebView2
+    // and invoke it's handlers. However, If we set Handled to true on the
+    // CoreWebView2Frame event handler, then we will not raise the
+    // ScreenCaptureRequested event off the CoreWebView2.
+    // 
+    // NotImplementedException could be thrown if underlying runtime did not
+    // implement Handled. However, we only run this code after checking if
+    // CoreWebView2Frame.ScreenCaptureRequested exists, and both exist together,
+    // so it would not be a problem.
+    args.Handled = true;
+}
+```
+
+# API Details
+## C++
+```
+interface ICoreWebView2_20;
+interface ICoreWebView2ScreenCaptureRequestedEventArgs;
+interface ICoreWebView2ScreenCaptureRequestedEventHandler;
+
+interface ICoreWebView2Frame3;
+interface ICoreWebView2FrameScreenCaptureRequestedEventHandler;
+interface ICoreWebView2FrameScreenCaptureRequestedEventArgs;
+
+/// This interface is an extension of `ICoreWebView2` that supports the ScreenCaptureRequested event. 
+// MSOWNERS: [email protected] 
+[uuid(accc0e97-fa2d-4a8d-ad61-cc9ae57a1825), object, pointer_default(unique)] 
+interface ICoreWebView2_20 : IUnknown { 
+  /// Add an event handler for the `ScreenCaptureRequested` event. 
+  /// `ScreenCaptureRequested` event is raised when the Screen Capture API is requested by the user using getDisplayMedia(). 
+  HRESULT add_ScreenCaptureRequested( 
+      [in] ICoreWebView2ScreenCaptureRequestedEventHandler* eventHandler, 
+      [out] EventRegistrationToken* token); 
+  /// Remove an event handler previously added with `add_ScreenCaptureRequested`. 
+  // MSOWNERS: [email protected] 
+  HRESULT remove_ScreenCaptureRequested( 
+      [in] EventRegistrationToken token); 
+}   
+/// Receives `ScreenCaptureRequested` events. 
+// MSOWNERS: [email protected] 
+[uuid(9b5bbea1-4a58-4567-8b42-8781d3986cb4), object, pointer_default(unique)] 
+interface ICoreWebView2ScreenCaptureRequestedEventHandler : IUnknown { 
+  /// Called to provide the event args when a screen capture is requested on 
+  /// a WebView element. 
+  HRESULT Invoke( 
+      [in] ICoreWebView2* sender, 
+      [in] ICoreWebView2ScreenCaptureRequestedEventArgs* args); 
+} 
+/// Event args for the `ScreenCaptureRequested` event. 
+// MSOWNERS: [email protected] 
+[uuid(a1d309ee-c03f-11eb-8529-0242ac130003), object, pointer_default(unique)] 
+interface ICoreWebView2ScreenCaptureRequestedEventArgs : IUnknown { 
+  /// The associated frame information that requests the screen capture  
+  /// permission. This can be used to grab the frame source, name, frameId,  
+  /// and parent frame information. 
+  [propget] HRESULT FrameInfo([out, retval] ICoreWebView2FrameInfo**  
+     frameInfo); 
+  /// By default, both the `ScreenCaptureRequested` event handlers on the 
+  /// `CoreWebView2Frame` and the `CoreWebView2` will be invoked, with the 
+  /// `CoreWebView2Frame` event handlers invoked first. The host may 
+  /// set this flag to `TRUE` within the `CoreWebView2Frame` event handlers 
+  /// to prevent the remaining `CoreWebView2` event handlers from being  
+  /// invoked. 
+  /// 
+  /// If a deferral is taken on the event args, then you must synchronously 
+  /// set `Handled` to TRUE prior to taking your deferral to prevent the 
+  /// `CoreWebView2`s event handlers from being invoked. 
+  [propget] HRESULT Handled([out, retval] BOOL* handled); 
+  /// Sets the `Handled` property. 
+  [propput] HRESULT Handled([in] BOOL handled); 
+  /// The host may set this flag to cancel the screen capture. If canceled, 
+  /// the screen capture UI is not displayed regardless of the 
+  /// `Handled` property. 
+  [propget] HRESULT Cancel([out, retval] BOOL* cancel); 
+  /// Sets the `Cancel` property. 
+  [propput] HRESULT Cancel([in] BOOL cancel); 
+  /// Returns an `ICoreWebView2Deferral` object. Use this deferral to 
+  /// defer the decision to show the Screen Capture UI. 
+  /// 
+  /// Returns an `ICoreWebView2Deferral` object.    
+  HRESULT GetDeferral([out, retval] ICoreWebView2Deferral** deferral); 
+} 
+/// This is an extension of the ICoreWebView2Frame interface that supports ScreenCaptureRequested 
+// MSOWNERS: [email protected] 
+[uuid(12885cda-9caa-4793-9c38-f15827dbab1f), object, pointer_default(unique)] 
+interface ICoreWebView2Frame3 : IUnknown { 
+  /// Add an event handler for the `ScreenCaptureRequested` event. 
+  /// `ScreenCaptureRequested is raised when content in an iframe any of its 
+  /// descendant iframes requests permission to use the Screen Capture  
+  /// API from getDisplayMedia() 
+  /// 
+  /// This relates to the `ScreenCaptureRequested` event on the  
+  /// CoreWebView2`. 
+  /// Both these events will be raised in the case of an iframe requesting 
+  /// permission. The `CoreWebView2Frame`'s event handlers will be invoked 
+  /// before the event handlers on the `CoreWebView2`. If the `Handled`  
+  /// property of the `ScreenCaptureRequestedEventArgs` is set to TRUE  
+  /// within the`CoreWebView2Frame` event handler, then the event will not  
+  /// be raised on the `CoreWebView2`, and its event handlers will not be   
+  /// invoked. 
+  /// 
+  /// In the case of nested iframes, the 'ScreenCaptureRequested' event will 
+  /// be raised from the top level iframe. 
+  /// 
+  /// If a deferral is not taken on the event args, the subsequent scripts  
+  /// re blocked until the event handler returns.  If a deferral is taken,  
+  /// the scripts are blocked until the deferral is completed. 
+  HRESULT add_ScreenCaptureRequested( 
+      [in] ICoreWebView2FrameScreenCaptureRequestedEventHandler* handler, 
+      [out] EventRegistrationToken* token); 
+
+  /// Remove an event handler previously added with  
+  /// `add_ScreenCaptureRequested` 
+  HRESULT remove_ScreenCaptureRequested( 
+      [in] EventRegistrationToken token); 
+} 
+
+/// Receives `ScreenCaptureRequested` events for iframes. 
+// MSOWNERS: [email protected] 
+[uuid(c07ac75c-2105-4bb8-9c57-21b6ed8fb381), object, pointer_default(unique)] 
+interface ICoreWebView2FrameScreenCaptureRequestedEventHandler : IUnknown { 
+  /// Provides the event args for the corresponding event. 
+  HRESULT Invoke( 
+      [in] ICoreWebView2Frame* sender, 
+      [in] ICoreWebView2ScreenCaptureRequestedEventArgs * args); 
+} 
+
+```
+
+## C#
+```c#
+namespace Microsoft.Web.WebView2.Core
+{
+    runtimeclass CoreWebView2ScreenCaptureRequestedEventArgs
+    {
+
+        [interface_name("Microsoft.Web.WebView2.Core.ICoreWebView2ScreenCaptureRequestedEventArgs")]
+        {
+            CoreWebView2FrameInfo FrameInfo { get; };
+            Boolean Cancel { get; set; };
+            Boolean Handled { get; set; };
+        }
+
+    }
+
+    runtimeclass CoreWebView2
+    {
+        // ...
+        event Windows.Foundation.TypedEventHandler<CoreWebView2, CoreWebView2ScreenCaptureRequestedEventArgs> ScreenCaptureRequested;
+    };
+
+    runtimeclass CoreWebView2Frame
+    {
+        // ...
+        // ICoreWebView2Frame3 members
+        [doc_string("ScreenCaptureRequested is raised when content in an iframe or any of its descendant iframes requests permission to use screen capture.\nIf a deferral is not taken on the event args, the subsequent scripts are blocked until the event handler returns. If a deferral is taken, the scripts are blocked until the deferral is completed.")]
+        event Windows.Foundation.TypedEventHandler<CoreWebView2Frame, CoreWebView2ScreenCaptureRequestedEventArgs> ScreenCaptureRequested;
+    }
+
+}
+```
+
+# UI Changes
+
+Having the column for specific tabs/WV2s doesn’t make sense in the vast majority of cases, so we will remove that column entirely. Apps that want to offer the ability to select a specific Tab/WV2 will need to use the full API when we have it available to construct their own UI. 
+
+
+Next, the URL of the WV2 is used in a handful of locations that we should replace by default: 
+
+“Choose what to share with <url>” in the main dialog. 
+
+“<url> is sharing a window” in the sharing bar when sharing a window. 
+
+All of these should be replaced with “this app”. 
+
+
+When the sharing bar is open, the icon for it is a WV2 icon, not the host app’s icon. We should use the host app’s icon (or no icon?)