Expose managed Main with NativeAOT when compiling the application as a static library

[ivanpovazan]

Description

In order to integrate Xamarin and NativeAOT when targeting iOS platforms the following scenario is used:

• Xamarin acts as a hosting native application which is statically linked with NativeAOT runtime, AOTed managed application and all the required libraries.
• Xamarin's startup sequence initialises NativeAOT runtime, the managed ObjCRuntime and invokes the managed Main method to start the application.

However, if the application is compiled as a static library with the ILCompiler, the managed Main method is not exposed to the native world.

To workaround this limitation, one could change the source code of the application and add a method with UnmanagedCallersOnlyAttribute that wraps the Main, but if we consider the experience of the end-user, there should be an automatic way of injecting the required code which would make the transition to NativeAOT seamless.

[ghost]

Tagging subscribers to 'os-ios': @steveisok, @akoeplinger
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

In order to integrate Xamarin and NativeAOT when targeting iOS platforms the following scenario is used:

• Xamarin acts as a hosting native application which is statically linked with NativeAOT runtime, AOTed managed application and all the required libraries.
• Xamarin's startup sequence initialises NativeAOT runtime, the managed ObjCRuntime and invokes the managed Main method to start the application.

However, if the application is compiled as a static library with the ILCompiler, the managed Main method is not exposed to the native world.

To workaround this limitation, one could change the source code of the application and add a method with UnmanagedCallersOnlyAttribute that wraps the Main, but if we consider the experience of the end-user, there should be an automatic way of injecting the required code which would make the transition to NativeAOT seamless.

Author:	ivanpovazan
Assignees:	-
Labels:	feature-request, os-ios, area-NativeAOT-coreclr
Milestone:	8.0.0

[am11]

if the application is compiled as a static library with the ILCompiler

I haven't tested it, but I think it should possible today. Which symbols to keep in static library scenario is up to the final linkage which in our case is left to the end-user (and we don't provide supporting script / build steps today: #70277). With -p:StaticExecutable=true, we have control over which symbols to keep in the final linkage of executable, which can be modified.

[ivanpovazan]

@am11 I was more referring to how ILCompiler compiles a native library vs an executable and the symbols it exposes via corresponding *RootProviders:

• if it compiles an assembly as a native lib it will expose: __managed__Startup
  

  runtime/src/coreclr/tools/aot/ILCompiler/Program.cs

  Line 221 in 9b76c28

  compilationRoots.Add(new NativeLibraryInitializerRootProvider(typeSystemContext.GeneratedAssembly, CreateInitializerList(typeSystemContext)));

• on the other hand if it compiles an assembly as an executable it will expose: __managed__Main
  

  runtime/src/coreclr/tools/aot/ILCompiler/Program.cs

  Line 227 in 9b76c28

  compilationRoots.Add(new MainMethodRootProvider(entrypointModule, CreateInitializerList(typeSystemContext)));

That being said, an executable compiled as a native library will not have its managed Main method exposed (and no way to pass parameters to it).

[jkotas]

There are number of things that happen around Main. For example, the arguments passed to Main are stored so that they can be also retrieved via Environment.GetCommandLineArgs() API, the runtime waits for other foreground to finish before proceeding to exit, the latched exit code is used as the exit code if the Main method does not return it, etc.

I assume that you want some or all of these to be included in the Main method wrapper. Is that right?

We should agree on the details like this before you proceed with implementation, so that it does not have to be reworked.

[AustinWise]

In case it's helpful, the approach I took for my proof-of-concept was to generate both the library and main method entry points in ILC. I then used the library version of bootstrapper to enable UnamangedCallersOnly functions to be called before the managed main method. This enables you to provide your own native main function, but otherwise compile and run like a normal nativeaot executable:

AustinWise@810f978

The managed main in this proof of concept still contains all those features Jan mentions, so I'm not sure its what you want.

[ivanpovazan]

@jkotas from the mentioned set of features around managed Main, what does ILC support by exposing __managed__Main?

---

@AustinWise thank you, that looks like something we would want.
I was wondering would it make sense to always expose __managed__Main when: -p:OutputType=Exe -p:NativeLib=Static

[jkotas]

@jkotas from the mentioned set of features around managed Main, what does ILC support by exposing __managed__Main?

All of them and more.

The IL code for __managed__Main is generated here:

runtime/src/coreclr/tools/aot/ILCompiler.Compiler/IL/Stubs/StartupCode/StartupCodeMainMethod.cs

Line 62 in ab2d195

public override MethodIL EmitIL()

.

I do not think you want the full existing __managed__Main for your purpose. For example, I do not think you want to run the library initializers. They are run by the library initialization. Running them twice is not a good idea.

[MichalStrehovsky]

I agree with what Jan wrote, we don't want to run both the library startup code and EXE startup code at the same time - they would run the same things twice. However, we could go with something similar to what @AustinWise suggested in #81097 (comment). (We just need to address the double execution.)

On a high level:

The NativeLibraryStartupMethod (the library case) will:

• Initialize CoreLib and friends (internal implementation detail not to worry about - just need to make sure nothing else runs before these - managed code cannot even do a new before these run)
• Run module initializers (this is potentially user code)

On the other hand, StartupCodeMainMethod (the exe case) will:

• Initialize CoreLib and friends (same as above)
• Store command line args so that Environment.GetCommandLineArgs can access them
• Store entrypoint assembly so that Assembly.GetEntryAssembly works.
• Initialize COM apartment state (not relevant outside Windows)
• Run module initializers (same as above)
• Call user code Main
• Store return value of Main in Environment class (if it's non-void)
• Shut down CoreLib (wait for foreground threads, call user code in AppContext.OnProcessExit, etc.)
• Exit with the exit code latched in the Environment class

One simple solution based on @AustinWise commit would be to add a bool flag to StartupCodeMainMethod class indicating whether to generate CoreLib initialization and module constructors. Then the workflow would be:

• NativeLibraryStartupMethod-generated code does its thing as the first thing when the module is loaded, as part of static initialization. NOTE: it will run user code in module constructors, and these module constructors won't see correct Environment.GetCommandLineArgs and Assembly.GetEntryAssembly if they ask.
• At some point the outside host calls __managed_main and passes command line arguments. StartupCodeMainMethod-generated code will run. This generated code will do everything as usual, but the corelib-initialization and module constructor running would be skipped.

If this is not desirable, we'll need to come up with some other ordering. I think it mainly depends on when do we want to run module initializers.

[ivanpovazan]

Thank you for the detailed explanation.

One simple solution based on @AustinWise commit would be to add a bool flag to StartupCodeMainMethod class indicating whether to generate CoreLib initialization and module constructors.

This approach sounds good to me. @rolfbjarne what do you think?

[rolfbjarne]

Thank you for the detailed explanation.

One simple solution based on @AustinWise commit would be to add a bool flag to StartupCodeMainMethod class indicating whether to generate CoreLib initialization and module constructors.

This approach sounds good to me. @rolfbjarne what do you think?

Yes, I think that sounds good too.

[ivanpovazan]

@AustinWise, would you be interested in taking over this issue?

[AustinWise]

Sure, I can create a PR.

[ivanpovazan]

thanks a lot! I have assigned the issue to you, and let me know if you need any help, I would gladly assist if needed.

[AustinWise]

@ivanpovazan

I created #81873 with my changes. I was not sure how this should fit into the larger build system, so I made the ILC parts flexible.

[ivanpovazan]

Closed via: #81873
Thank you @AustinWise!