Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[browser][MT] Multithreading and JavaScript async interop in .NET 9 #85592

Closed
26 of 31 tasks
Tracked by #68162
lambdageek opened this issue May 1, 2023 · 40 comments
Closed
26 of 31 tasks
Tracked by #68162

[browser][MT] Multithreading and JavaScript async interop in .NET 9 #85592

lambdageek opened this issue May 1, 2023 · 40 comments
Assignees
Labels
arch-wasm WebAssembly architecture area-VM-threading-mono os-browser Browser variant of arch-wasm tracking This issue is tracking the completion of other related issues.
Milestone

Comments

@lambdageek
Copy link
Member

lambdageek commented May 1, 2023

Tracking issue for further work on JS interop and multithreading.

Constituent part of #76956

Goals

  • CPU intensive workloads on dotnet thread pool
  • Allow user to start new managed threads using new Thread and join it.
  • Add new C# API for creating dedicated web workers with JS interop. Allow JS async/promises via external event loop.
  • enable blocking Task.Wait and lock() like APIs from C# user code on all threads or non-JS/UI threads
    • Current public API throws PNSE for it
    • This is core part on MT value proposition.
    • If people want to use existing MT code-bases, most of the time, the code is full of locks.
    • People want to use existing desktop/server multi-threaded code as is.
  • allow HTTP and WS C# APIs to be used from any thread despite underlying JS object affinity
  • JSImport/JSExport interop in maximum possible extent.
  • don't change/break single threaded build. †

Lower priority goals

  • try to make it debugging friendly
  • sync C# to async JS
    • dynamic creation of new pthread
    • implement crypto via subtle browser API
    • allow MonoVM to lazily download DLLs from the server, instead of during startup.
    • implement synchronous APIs of the HTTP and WS clients. At the moment they throw PNSE.
  • sync JS to async JS to sync C#
    • allow calls to synchronous JSExport from UI thread (callback)
  • don't prevent future marshaling of JS transferable objects, like streams and canvas.
  • offload CPU intensive part of WASM startup to WebWorker, so that the pre-rendered (blazor) UI could stay responsive during Mono VM startup.

Non-goals

  • interact with JS state on WebWorker of managed threads other than UI thread or dedicated JSWebWorker

Design discussion and experiment

Depending on selected design

Bugs

Memory growth, alignment

Broken tests

Nice to have

Progress

Future

@lambdageek lambdageek added tracking This issue is tracking the completion of other related issues. area-System.Runtime.InteropServices.JavaScript labels May 1, 2023
@lambdageek lambdageek added this to the 8.0.0 milestone May 1, 2023
@lambdageek lambdageek self-assigned this May 1, 2023
@lambdageek
Copy link
Member Author

/cc @lewing @pavelsavara

@ghost
Copy link

ghost commented May 2, 2023

Tagging subscribers to 'arch-wasm': @lewing
See info in area-owners.md if you want to be subscribed.

Issue Details

Tracking issue for further work on JS interop and multithreading.

Constituent part of #76956

Scenarios

  • [blazor] Single threaded Blazor WebAssembly code can turn on threading and continue working. Main thread interop keeps working. "Hidden" async interop in the HTTP stack and in aspnetcore keeps working.
  • [pool-promise] Multi-threaded WebAssembly code can do async JS interop from threadpool threads using promises [wasm-mt] Land the webworker JSImport support #84489
  • [pool-timeout] Multi-threaded WebAssembly code can do async JS interop from threadpool threads using timeouts and event listeners
  • [thread-eventloop] Multi-threaded WebAssembly code can start new threads using new Thread and use async interop

Work Items

[blazor]

  • Async runtime code that uses ConfigureAwait(false) behaves correctly when using async JS interop (e.g. HTTP stack)
  • The aspnetcore BeginInvokeDotNet/EndInvokeDotNetAfterTask APIs work correctly in multithreaded apps (ContinueWith(..., TaskScheduler.Default)).

[pool-promise]

[pool-timeout]

[thread-eventloop]

  • [API proposal] new Thread has a way to spin up "external eventloop" threads
  • [API proposal] Give .NET code a way to keep an external eventloop thread alive ("keepalive token")
  • JS API provide a way for JS code to keep a .NET external eventloop thread alive
  • awaiting unsettled JS interop promises keeps external eventloop threads alive
  • reachable JS objects that reference C# objects keep an external eventloop thread alive
  • SynchronizationContext for external eventloop threads that allows posting work to the thread from other C# threads (ie extend JSSynchronizationContext to non-main threads)
Author: lambdageek
Assignees: lambdageek
Labels:

arch-wasm, tracking, area-System.Runtime.InteropServices.JavaScript

Milestone: 8.0.0

@pavelsavara
Copy link
Member

Blazor's requirements are:

  1. Only one work item runs at at time on this sync context
  2. You can post to it from any thread
  3. Once on the sync context, JS interop must work (so presumably means we can be sure we're now on the main thread)

@lewing lewing modified the milestones: 8.0.0, 9.0.0 Jul 26, 2023
@pavelsavara pavelsavara changed the title [wasm-mt] Multithreading and JavaScript async interop in .NET 8 [wasm-mt] Multithreading and JavaScript async interop in .NET 9 Sep 27, 2023
@pavelsavara pavelsavara changed the title [wasm-mt] Multithreading and JavaScript async interop in .NET 9 [browser][MT] Multithreading and JavaScript async interop in .NET 9 Nov 9, 2023
@pavelsavara pavelsavara self-assigned this Nov 9, 2023
@Rippletank
Copy link

I'm using the latest .net8.0 RC2 release in a multithreaded WASM app using UNO and Skiasharp. I am getting the "Please use dedicated worker...etc" from the AssertWebWorkerContext() method as expected.

The work around is sending JSInterop calls to the main thread, which works but results in some animation stuttering particularly with large db/cryptography jobs.

I would like to try out the WebWorker class but it does not appear to be included in the Microsoft.NETCore.App\8.0.0-rc.2.23479.6\System.Runtime.InteropServices.JavaScript.dll which the application is using and because of the internal methods I see how to implement it outside of the dotnet runtime.

Is this just not available in released packages yet or is there some way to enable it?

I notice there is a sample that uses it but doesn't this need compiling the runtime from source?

@pavelsavara
Copy link
Member

I'm using the latest .net8.0 RC2 release in a multithreaded WASM app using UNO and Skiasharp.

@Rippletank Threads are not supported in .Net 8. As you can see above, there are many reasons why we are not ready for it.

@jtorjo
Copy link

jtorjo commented Nov 9, 2023

@pavelsavara Thanks for the update. I'm curious if we'll ever have this, because if I remember correctly, this was .net7 --> .net8, and now it's .net8 --> .net9

@Rippletank
Copy link

@Rippletank Threads are not supported in .Net 8. As you can see above, there are many reasons why we are not ready for it.

Ok, I see. It is confusing because multithreading is enabled with Uno web assembly apps and it works within the app itself but obviously doesn't work with the interop.

I'm curious, is ok to use it, apart from interop issues, or is it generally problematic at the moment? For things like http etc.

@pavelsavara
Copy link
Member

I'm curious, is ok to use it, apart from interop issues, or is it generally problematic at the moment? For things like http etc.

MT is problematic in Net8 in my experience, that's why it's experimental. I suggest you ask Uno how to turn it of.

@pavelsavara Thanks for the update. I'm curious if we'll ever have this, because if I remember correctly, this was .net7 --> .net8, and now it's .net8 --> .net9

Yes those are not easy problems. We are not out of the woods yet for Net9 either. We don't want to ship product which could randomly deadlock. Wish us luck.

@Matheos96
Copy link

@pavelsavara I'm definitely wishing you luck. But it seems we won't have it for another 2+ years, if I understand you correctly.

I understand that you are waiting with some specific use-case for this ? Could you please share more details why/how you would benefit from threads in browser ? Also anyone else reading this, I would like to hear about your use-cases.

Here is our use-case:

We have a Client-side only Blazor WASM app. Our main component is a PCB 3D Board Viewer. We load the data from our database but the data itself is not that complex (source from CAD formats). It does not technically contain the 3D data we need to render. So we build the 3D models on demand, based on our own Data model which is being deserialized and interpreted, all client side. The amount of details on such a board is immense, meaning we will have to do A LOT of calculations. For big boards we may have waiting times for more than 4 minutes, simply waiting for the deserialization (not too bad) and generation of 3D vertices, uvs and so on. During this time, the browser is pretty much completely frozen, we have some Task.Delay(1)'s here and there to update a progress bar in between but often we get the "not responding" message anyway. This is obviously a horrible user experience. Our dream is to be able to offload this generation to separate threads, for example separate board layers don't depend on each other and can be built completely in parallel. Ideally we want to show something really quickly, and then in the background keep calculating stuff that become available once done. This is obviously nothing but a dream at this point, still being restricted to the UI thread (and potentially web workers which we will look into). The whole thing does not get any better from the fact that we depend on A LOT of JS interop too... But that is another story.

So that is our use-case. One could ask why we don't do server-side rendering, but the fact is that we don't really have a deployed production "web" version yet. Currently we are providing this as an extension of our legacy phat client (embedding the SPA in a CEF Sharp window). One of our selling points is also the fact that we "make everything happen" in the browser, without the need for a backend. We also offer a "from file" option which keeps all data in the browser, meaning we definately have to do the calculations in the wasm app.

@curiousdannii
Copy link

curiousdannii commented Jan 22, 2024

Copying my use case from #68162:

I have an app I'm trying to port, which calls a blocking function in a non-managed dll:

[DllImport("Glk")]
internal static extern void glk_select(ref Event ev);

To port it to WASM/JS it instead needs to call an async JS function (ignore that it no longer takes any arguments):

[JSImport("glk_select", "main.js")]
internal static partial Task glk_select();

I've made an interface that lets me link to either the DLL or the JSImport, but I just need some way to block while I wait for the promise to resolve. And that's pretty much it! I don't really care how this is accomplished. While the details of which threads run where will matter for other people and their use cases, for me it's just an implementation detail. An elegant solution is more what I'm after.

(I also tried making the app async, but it's a large messy legacy app (28k+ LOC) and I would end up needing convert nearly everything to async functions, so threads seems a better option.) (And async functions not having ref or out arguments and VB.net not having tuple destructuring makes it even grosser to try to convert.)

@iSeiryu
Copy link

iSeiryu commented Jan 23, 2024

Copy-pasting use-cases from here: https://www.reddit.com/r/Blazor/comments/199o6e6/why_is_multithreading_such_a_requested_feature/

  • You want to spawn up background threads to perform tasks that aren't directly related to UI things that only occasionally come back to UI

  • You have things that can run in the background to load into a static property to show when ready and you don't want to tie up UI thread time on processing it - you expect the UI to just show it when ready

  • You have some legacy CS code you want to include in the app - that's the whole point right, you share code but oops this code has a sync method and your UI is now fucked

  • You hope to dear god you can have something like promises that you get in JS

  • Prevent all runtime exceptions relating to threading. I sure had them early days before I fully realised what I had bought into, back to the legacy code integration thing

  • Just why don't we have multithreading in what is otherwise a multi-threaded environment ... except for Blazor

@pavelsavara
Copy link
Member

pavelsavara commented Jan 23, 2024

You hope to dear god you can have something like promises that you get in JS

I'm not sure I follow, could you please elaborate @iSeiryu ?
You can marshal Task/Promise with JSImport.

@iSeiryu
Copy link

iSeiryu commented Jan 23, 2024

@pavelsavara those are not my words. That's just a list of use-cases I came across yesterday. Btw, someone else in that Reddit thread asked a similar question.

@dennis-garavsky
Copy link

dennis-garavsky commented Jan 26, 2024

enable blocking Task.Wait and lock() like APIs from C# user code on all threads

We will also benefit from this enhancement at DevExpress in certain products, and our customers will for sure benefit in their projects (while migrating shared code to .NET 8 and Blazor).

For instance, in our application framework DevExpress XAF (powered by ASP.NET Core Blazor), we researched ways to plug in a middle tier service into our existing code base with Blazor WebAssembly and EF Core. The following code throws "Cannot wait on monitors on this runtime" with Blazor .NET 8 as it did 5 years ago when we started supporting Blazor in XAF and experimented with WebAssembly back then (dropped WebAssembly due to low performance and the lack of multi-threading support). Today we still rely on Blazor Server in XAF Blazor, but want to support WebAssembly render modes in the future.

public class WebApiSecuredDataServerClient
        protected override TResult Invoke<TResult>(string action) {
            return StaSafeHelper.Invoke(() => InvokeAsync<TResult>(action, CancellationToken.None).ConfigureAwait(false).GetAwaiter().GetResult());
        }

NOTE: our Blazor UI Components already fully support both WebAssembly and Server modes from November 2023.

As other readers also noted, we also want to reuse a large amount of our existing framework core code in Blazor WebAssembly, as we successfully did in Blazor Server, WinForms and WebForms for over 15 years now. Converting "nearly everything to async functions" in the framework core code is NOT realistic due to the huge breaking changes for other customers and rewrite costs, of course (it may be easier to drop everything and start over with a new product then).

Thank you for your consideration.

@pavelsavara
Copy link
Member

enable blocking Task.Wait and lock() like APIs from C# user code on all threads

We are still re-considering this one. See my other comment.

Converting "nearly everything to async functions" in the framework core code is NOT realistic

We understand that ^^.

Alternatives are

  • if you could offload such code out of UI thread with moderate effort, this is best.
  • multi-threading runtime will not allow any managed code in physical UI thread and run in in WebWorker (deputy). It will marshal all JS interop across physical threads. This is still work in progress POC, we hope to know more soon.
  • allow you to take the risk (via feature flag) and allow you to spin-wait in UI thread and face deadlocks. We will not support this for production.

@curiousdannii
Copy link

The mono runtime is compiled using Emscripten. Is this also the case with AOT compilation? If that's the case, any chance that we could run ASYNCIFY like we would for any other Emscripten app? That could be another way of converting sync functions into async.

@glutio
Copy link

glutio commented Apr 2, 2024

Trying .net9 preview 2, wasmbrowser template, the all synchronous call .net->js->.net hangs the browser window. Seems like a regression from .net8

@pavelsavara
Copy link
Member

Trying .net9 preview 2, wasmbrowser template, the all synchronous call .net->js->.net hangs the browser window. Seems like a regression from .net8

This is by design on multi-threaded build. The reason is that managed code is not running on UI thread, but your JS is running there. The managed thread is blocked waiting for the first call to return from JS while you try to send it another message with managed call.

Nested synchronous JS interop calls would not be supported in MT at all. It will throw PNSE in next previews.
Non-nested synchronous JS interop would be possible in .Net->JS direction.
Non-nested synchronous JS->.Net direction only behind a configuration flag and could lead to deadlocks in some scenarios.

Here is work in progress PR on the topic. #99833

Single-threaded build would not change in this regard.

@maxkatz6
Copy link
Contributor

@pavelsavara would it be possible to call a sync .NET exported function (or a JSImport callback) without blocking JS thread? I.e. discarding a result. Making .NET function async is a decent workaround here, but I don't see why it should be forced for "fire and forget" scenarios. Thanks.

@maxkatz6
Copy link
Contributor

maxkatz6 commented Apr 23, 2024

Also, will new behavior be toggleable? It's not yet clear if we can support new threading model in .NET 9 WASM, since it has to work with sync requestAnimationFrame callbacks, webGL and Skia on top of Emscripten, something that yet needs to be tested.
But .NET 8 WASM threading model might be sufficient for most Avalonia apps (although we would love to make new one supported too).

Or even better, could it be possibly to initialize .NET thread on a physical UI thread? In our case it would work as a render thread with Skia and WebGL. While the rest of the app is managed on a normal deputy thread.

@kekekeks
Copy link

kekekeks commented Apr 23, 2024

I believe that the lack of access to the "true" UI thread would completely break WebGL interop.
WebGL doesn't have a dedicated present/swapBuffers call and instead just presents render results once the user code exits the current event callback. If JS API calls are getting enqueued from a worker thread, that would mean that webgl content will be "presented" on every single call, thus making any kind of rendering impossible.

Another problem would be WebGL usage with SkiaSharp: Skia uses WebGL internally and expects those calls to happen on the "true" UI thread rather than on a web worker.

@pavelsavara
Copy link
Member

pavelsavara commented Apr 23, 2024

"fire and forget" scenarios.

I created DiscardNoWait for it and it's applied to void methods.
But it's not on public API, yet.
It executes asynchronously similar to methods returning Task, but it doesn't need to marshal the Task.
If you search this repo you can see it used in some tests.

You can help me by pushing it thru API review process (and have discussion about it's name on that PR).

/// <summary>
/// Dispatches the call asynchronously and doesn't wait for result.
/// </summary>
/// <returns>The marshaler metadata.</returns>
public static JSMarshalerType DiscardNoWait { get; } = new JSMarshalerType(new JSFunctionBinding.JSBindingType
{
Type = MarshalerType.DiscardNoWait
});

Could it be possibly to initialize .NET thread on a physical UI thread?

We will not support running managed code on UI thread, I'm confident now.
It leads to too many deadlocks which are not possible to predict or even test against.

At the moment the UI thread is still attached to Mono VM, but that's going to change as soon as possible.

Also, will new behavior be toggleable?

There is toggle for allowing synchronous JSExport (with all the consequences of spin-blocking UI thread)
We just have to make it into MSBuild property
It's quite easy to cause deadlock with it, but not so bad as running managed code on UI thread.

But .NET 8 WASM threading model might be sufficient for most Avalonia apps

Net8 WASM threading is very broken, my 2c: don't do it to your users.

I believe that the lack of access to the "true" UI thread would completely break WebGL interop.

I would like to learn more, how to make more native scenarios possible with MT.
This is not right place for detailed discussion, so I made #101421 for it

@maxkatz6
Copy link
Contributor

You can help me by pushing it thru API review process

How can I help with the review process? Naming looks fine, easier to understand than "OneWay".

@pavelsavara
Copy link
Member

You can help me by pushing it thru API review process

How can I help with the review process? Naming looks fine, easier to understand than "OneWay".

You can follow https://github.com/dotnet/runtime/blob/main/docs/project/api-review-process.md
Create issue with proposal coach it thru the API review meeting.
Also create PR which removes the API from

<Suppression>
<DiagnosticId>CP0001</DiagnosticId>
<Target>T:System.Runtime.InteropServices.JavaScript.JSType.DiscardNoWait</Target>
<Left>ref/net9.0/System.Runtime.InteropServices.JavaScript.dll</Left>
<Right>runtimes/browser/lib/net9.0/System.Runtime.InteropServices.JavaScript.dll</Right>
</Suppression>

Which will make it public.

Examples are https://github.com/dotnet/runtime/issues?q=is%3Aissue+label%3Aapi-approved+is%3Aclosed

@maxkatz6

@kekekeks
Copy link

Will async callbacks be only supported for JSExport? We don't use those and are using [JSMarshalAs(JSType.Function)].

If we try to change JS->.NET callbacks to return tasks, the SDK complains about 18>TimerHelper.cs(14,113): Error SYSLIB1072 : The type 'System.Func<System.Threading.Tasks.Task>' is not supported by source-generated JavaScript interop. The generated source will not handle marshalling of parameter 'callback'. For more information see https://aka.ms/dotnet-wasm-jsinterop (https://learn.microsoft.com/dotnet/fundamentals/syslib-diagnostics/syslib1072)

What is the new intended way to pass delegates to JS side?

@pavelsavara
Copy link
Member

The type 'System.Func<System.Threading.Tasks.Task>' is not supported

That's a (known) gap. Could you please open separate issue for it and ping me there ? @kekekeks

@pavelsavara
Copy link
Member

pavelsavara commented May 14, 2024

I'm closing this as complete for the scope of Net9 for the dotnet runtime.

Summary

The difference between Net8 and Net9 threading is that it actually works now. 😅
All the managed code is running in the web workers and that makes it possible to use blocking .Wait on all managed threads.
Limitation (by design) is that you can only call asynchronous C# methods as JSExport from the UI thread (main JS with DOM).
All JavaScript interop via JSImport is dispatched on to UI thread via low level async message.
Another limitation by design is that you can't make nested synchronous callback from inside synchronous JSImport call.
HTTP and WebSocket connections are made from UI thread on your behalf.

Status is still: experimental

Blazor

For Blazor WASM, the multi-threading feature was cut for Net9.
The biggest gaps in Blazor are described in dotnet/aspnetcore#54365

Demo

I updated the Raytracer demo https://pavelsavara.github.io/dotnet-wasm-raytracer/

Performance

We didn't optimize for multi-threaded performance yet.
As compared to single-threaded dotnet on WASM, it will be slightly slower per core, but you can use many cores now.
It should also allow you to offload CPU intensive work from the main thread and keep the DOM responsive.

Future work

There is draft of JSWebWorker API which we will explore in the future. It can allow you to interact with another JS than the UI thread.
We didn't implement any of the sync-over-async scenarios in this release.

If there are bugs or missing features, please create new issue on runtime repo and ping me, we will triage and prioritize.

@curiousdannii
Copy link

@pavelsavara Is the working threading in the Net9 preview 3 from April 11? Or can we expect it in the next preview?

@pavelsavara
Copy link
Member

The demo above is on Preview 3.
Net9 Preview 3 has most of it and Preview 4 has fixes and some cleanup, I expect that we will improve quality further.
If you run into issues you can always try nightly build and see if it was already fixed.

@kekekeks
Copy link

Is any kind of debugging supposed to work with MT mode? I can't get vscode nor /_framework/debug to stop on breakpoints.

@pavelsavara
Copy link
Member

Is any kind of debugging supposed to work with MT mode? I can't get vscode nor /_framework/debug to stop on breakpoints.

There is open issue for that #81282 and few more for broken tests

@github-actions github-actions bot locked and limited conversation to collaborators Jun 15, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
arch-wasm WebAssembly architecture area-VM-threading-mono os-browser Browser variant of arch-wasm tracking This issue is tracking the completion of other related issues.
Projects
None yet
Development

No branches or pull requests