Mono does not implement coreclr_create_delegate hosting API or ComponentActivator #59815
Comments
dotnet-issue-labeler
bot
added
the
untriaged
|
I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label. |
|
@jkoritzinsky This CompnentActivator ? What's DNNE - I'm not familiar with the acronym. |
lambdageek
removed
the
untriaged
|
Yes, that is the ComponentActivator I'm referring to. DNNE is DotNetNativeExports, an project that @AaronRobinsonMSFT started that makes exposing managed code as native exports much simpler (and has significantly streamlined our ability to test the DllImportGenerator source generator). |
|
@jkoritzinsky What's a good way to get an overview of how the DllImportGenerator testing is designed? You're telling me some pieces and I'm having a hard time putting the whole thing together in my head. @steveisok I think we need to have a discussion about how this testing will look on mobile. |
@lambdageek The design is pretty simple. The tests are XUnit based and have a series of P/Invoke signatures. The DNNE package generates native platform library that represents specific exports from the managed assembly. When the test P/Invokes into the generated native library it does two things: (1) verifies the runtime is loaded and (2) finds the associated managed function that was indicated to be exported. Loading the runtime is straight forward enough and mono supports that without issue. However, discovery of the managed function by returning a function pointer to that function isn't. That is typically done via DNNE's implementation can be found at https://github.com/AaronRobinsonMSFT/DNNE/blob/master/src/platform/platform.c. |
ok, I think I understand so far. In C# you write a P/Invoke declaration. It's body is some boilerplate that ultimately calls back to C#. DNNE generates that boilerplate. Then from a C# xunit test you invoke the P/Invoke, that calls into the generated native library which invokes a managed method and the managed method records the test success in some way?
I'm unclear on what (1) means. Isn't this axiomatically true: the runtime is loaded because the test is running. I'm slightly unclear on what (2) means: essentially you want to get back to managed land from native. One way to have done that was to use something like GetFunctionPointerForDelegate which would require passing the function pointer from managed to native. But an alternate approach is to call The bigger issue is that Mono does not have corehost (and hence the corefxr entrypoints) on mobile. But it doesn't seem like that's an essential part of this scheme - you could use the |
|
DNNE itself is typically used for providing native exports of managed assemblies. The generated native library can be consumed naturally from a native application and the fact that the underlying implementation is managed is an implementation detail that is hidden. The usage of DNNE in this case is to avoid having any explicit native assets to test the DllImport source generator. Testing the DllImport source generator then becomes entirely managed with the the thin native veneer useful to properly consume the test target via P/Invoke. For (1), yes in this scenario the runtime will already be initialized and that is handled opaquely by
That isn't the alternative approach that is how DNNE works - imagine a world where a managed assembly wants to provide the implementation to native and only project a native API. The way to do that is via DNNE and a small bit of C code that hides the runtime activation and managed to function pointer conversion away. |
|
Thanks for the clarifications @jkoritzinsky @AaronRobinsonMSFT.
Ok, great. So I understood what's going on ;-)
I didn't mean that it's an alternative for DNNE. Just trying to make sure I understand what the tests are meant to do. So I think in summary to run on mobile where dynamic linking is allowed we would need to:
For static linking (ie ios device builds; and webassembly) we'd need to do... something else - probably a different version of DNNE's |
|
@lambdageek I don't fully understand why all these steps are needed in this case. Is it because of mono's hosting semantics? I was assuming we could treat DNNE generated native libraries as if they were any other native binary – P/Invoke into them and be none the wiser? Step (4) is all that I was expecting as actual work. |
@AaronRobinsonMSFT My assumption is: we want to run the dll import generator tests on Android,ios and wasm, not just on Linux & mac desktops with Mono instead of CoreCLR as the underlying runtime. If we only want to run on desktops then (4) is sufficient.
I'd say it's the hosting model, rather than something specific to mono. There's no nethost, hostfxr or hostpolicy on mobile and any code that depends on them will not work.
The DNNE generated native libraries are probably ok (assuming we compile them with the right toolchain). But they depend on nethost, hostfxr and hostpolicy, right? So the functions that We haven't tried the desktop versions of My guess is that the most straightforward way would be to build a mobile |
The scope was what I was missing here. Thanks for the details. |
|
+1 for corehost support on mobile (simulators for the starters). and +2 for a true static linking support (e.g. |
@am11 I'm curious why you say that. What is the usecase? From my perspective all of the logic of "find the right runtime to run this managed code" is already mostly irrelevant in self-contained desktop scenarios, and is completely useless on mobile - you run what's in the app bundle - there are no choices. |
@AaronRobinsonMSFT @lambdageek I can have someone on my team look to provide the mobile libraries. Safe to assume you would like this sooner rather than later? |
Mainly non-production/xamarin; I was coming from:
Yup, there are still some implementation details even in self-contained e.g. how it uses user configuration json files, finds the hook in (coreclr) runtime and exports symbols for sos/diagnostics etc. I think making the same (familiar) hosts work on various platforms will open avenues for broader scenarios. |
…to shared CoreLib (#59962) * [CoreLib] Move ComponentActivator to the shared area * [mono] Implement coreclr_create_delegate hosting API Related to #59815 * Mark IsolatedComponentLoadContext as unsupported on mobile and browser Should make ComponentActivator work without them (depends on AssemblyDependencyResolver, which depends on hostpolicy or hostpolicy pinvoke redirects) * Also unsupported on MacCatalyst
|
Ok, mono on desktop platforms should support this now. (lightly verified by trying a DNNE sample with Mono https://github.com/lambdageek/dnne_monovm_sample) Mobile requires a bunch of host work. A new thing I found out is that ComponentActivator (or rather IsolatedLoadContext) depends on AssemblyDependencyResolver, which in turn depends on hostpolicy being on the device (or hostpolicy pinvoke overrides as is done for coreclr single file). See #59961 for related work |
|
@lambdageek I'm trying this out with the DllImportGenerator.Tests test suite and I'm seeing the following error messages (and crashes resulting from double faults/SIGABRTs/etc): They don't seem to consistently happen due to a specific test, so I'm not sure exactly what's going on. |
|
@jkoritzinsky are there stack traces in the logs? How do I run the DllImportGenerator.Tests suite - I think I will need to repro locally.
|
|
Here's the output from Mono's dump (this is in a release build): Here's what I did to repro:
That should show you the failures, as well as give you repro instructions to directly run the test process without the |
|
@jkoritzinsky thanks I'll take a look. Best guess so far is that |
|
Yea so it's failing in the call to [UnmanagedCallersOnly(EntryPoint = "hresult_out_ushort_string")]
public static int ReturnAsOutString(int hr, ushort** ret)
{
string str = hr.ToString();
*ret = (ushort*)Marshal.StringToCoTaskMemUni(str); /// this is Interop.Sys.Malloc
return hr;
}because evidently we're already in GC Safe mode. Which presumably means we got a function pointer to the managed method ( Separately the interpreter has some warnings about mismatching stack types but I can't tell yet if that's related to any failure or if it's benign. |
That's with the JIT (and presumably AOT will be ok, too, given the issue). But the interpreter is very unhappy. |
|
Ah ok, in the interpreter, |
|
With #60200, the DllImportGenerator.Tests suite passes for me on desktop (macOS) MonoVM with both the JIT or the interpreter. |
…ointer for functions marked with UnmanagedCallersOnlyAttribute, return a pointer to the n2m wrapper for calls to GetFunctionPointer Contributes to dotnet#59815
Now that dotnet#59815 and dotnet#60200 are merged, the DllImportGenerator tests pass on Mono in desktop form factors (where hostfxr and hostpolicy exist).
Update the hosting API has been added to Mono. This issue is mainly open now to track the remaining work needed in order to run the DllImportGenerator tests on mobile platforms (or indeed other apps that may want the desktop hosting API on mobile). See #59815 (comment) for one path forward.
Mono does not implement the
coreclr_create_delegatehosting API method nor the ComponentActivator class used by the nethost library. This is blocking us from running the DllImportGenerator.Tests suite, since we use DNNE to enable easier managed->native testing in the libraries suite by implementing the "native" side primarily in managed code.The text was updated successfully, but these errors were encountered: