Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Symbols required for AOT are private in libmonosgen-2.0.dylib #55000

Closed
rolfbjarne opened this issue Jul 1, 2021 · 18 comments · Fixed by #55224, #55344, #56144 or #56545
Closed

Symbols required for AOT are private in libmonosgen-2.0.dylib #55000

rolfbjarne opened this issue Jul 1, 2021 · 18 comments · Fixed by #55224, #55344, #56144 or #56545
Assignees
Milestone

Comments

@rolfbjarne
Copy link
Member

Description

Mono's AOT compiler generates references to numerous functions in lib mono, but those functions are not public in libmonosgen-2.0.dylib, only libmonosgen-2.0.a.

Example for mono_array_new_1:

$ nm microsoft.netcore.app.runtime.mono.iossimulator-x64/6.0.0-preview.7.21329.1/runtimes/iossimulator-x64/native/libmonosgen-2.0.dylib | grep mono_array_new_1 
00000000001a2b40 t _mono_array_new_1

$ nm microsoft.netcore.app.runtime.mono.iossimulator-x64/6.0.0-preview.7.21329.1/runtimes/iossimulator-x64/native/libmonosgen-2.0.a | grep mono_array_new_1
                 U _mono_array_new_1
00000000000004e0 T _mono_array_new_1

The end result is that we can't link dynamically with libmonosgen-2.0.dylib when using the AOT compiler, we have to link with the static libmonosgen-2.0.a library.

Example native link failures:

Undefined symbols for architecture x86_64:
  "_mini_llvm_init_method", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvm_init_method_intptr_intptr_intptr_intptr in System.Private.CoreLib.dll.o
  "_mini_llvmonly_get_interp_entry", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvmonly_get_interp_entry_intptr in System.Private.CoreLib.dll.o
  "_mini_llvmonly_init_delegate", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvmonly_init_delegate_object in System.Private.CoreLib.dll.o
  "_mini_llvmonly_init_delegate_virtual", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvmonly_init_delegate_virtual_object_object_intptr in System.Private.CoreLib.dll.o
  "_mini_llvmonly_init_vtable_slot", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvmonly_init_vtable_slot_intptr_int in System.Private.CoreLib.dll.o
  "_mini_llvmonly_pop_lmf", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvmonly_pop_lmf_intptr in System.Private.CoreLib.dll.o
  "_mini_llvmonly_resolve_generic_virtual_call", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvmonly_resolve_generic_virtual_call_intptr_int_intptr in System.Private.CoreLib.dll.o
  "_mini_llvmonly_resolve_generic_virtual_iface_call", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvmonly_resolve_generic_virtual_iface_call_intptr_int_intptr in System.Private.CoreLib.dll.o
  "_mini_llvmonly_resolve_iface_call_gsharedvt", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvmonly_resolve_iface_call_gsharedvt_object_int_intptr_intptr in System.Private.CoreLib.dll.o
  "_mini_llvmonly_resolve_vcall_gsharedvt", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvmonly_resolve_vcall_gsharedvt_object_int_intptr_intptr in System.Private.CoreLib.dll.o
  "_mini_llvmonly_throw_aot_failed_exception", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvmonly_throw_aot_failed_exception_intptr in System.Private.CoreLib.dll.o
  "_mini_llvmonly_throw_nullref_exception", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvmonly_throw_nullref_exception in System.Private.CoreLib.dll.o
  "_mono_array_new_1", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mono_array_new_1_intptr_int in System.Private.CoreLib.dll.o
[...]

Example binlog: msbuild.binlog.zip

@dotnet-issue-labeler dotnet-issue-labeler bot added the untriaged New issue has not been triaged by the area owner label Jul 1, 2021
@dotnet-issue-labeler
Copy link

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

@rolfbjarne
Copy link
Member Author

CC @steveisok @akoeplinger

@steveisok
Copy link
Member

@lambdageek @vargaz Can either one of you take a look and help us triage?

@lambdageek
Copy link
Member

mono-aot-cross execution started with arguments: --aot=mtriple=x86_64-ios,data-outfile=obj/Debug/net6.0-ios/iossimulator-x64/nativelibraries/aot-output/x86_64/System.Private.CoreLib.aotdata,static,asmonly,direct-icalls,interp,soft-debug,dwarfdebug,no-direct-calls,outfile=obj/Debug/net6.0-ios/iossimulator-x64/nativelibraries/aot-output/x86_64/System.Private.CoreLib.dll.s --debug /Users/rolf/work/maccore/arm64-dyn-msgsend/xamarin-macios/tests/dotnet/MyInterpretedApp/iOS/obj/Debug/net6.0-ios/iossimulator-x64/linked/System.Private.CoreLib.dll

@vargaz do these arguments make sense? I'm not sure if static,direct-icalls,interp is a combination that makes sense with dynamic linking.

@rolfbjarne Do you also get problems if you're doing a fullaot build, or just interp-only?

@vargaz vargaz self-assigned this Jul 6, 2021
@vargaz
Copy link
Contributor

vargaz commented Jul 6, 2021

Those are declared using ICALL_EXTERN_C, the runtime probably needs to be built with
ENABLE_ICALL_EXPORT.

@vargaz
Copy link
Contributor

vargaz commented Jul 6, 2021

Or direct-icalls needs to be dropped from the aot command line.

@steveisok steveisok removed the untriaged New issue has not been triaged by the area owner label Jul 6, 2021
@steveisok steveisok added this to the 6.0.0 milestone Jul 6, 2021
vargaz added a commit to vargaz/runtime that referenced this issue Jul 6, 2021
@ghost ghost added the in-pr There is an active PR which will close this issue when it is merged label Jul 6, 2021
@vargaz
Copy link
Contributor

vargaz commented Jul 6, 2021

Are there any tests in dotnet/runtime which run into this problem ?

@lambdageek
Copy link
Member

Are there any tests in dotnet/runtime which run into this problem ?

I think no, because we've been doing static linking in dotnet/runtime. I have a PR to switch to dynamic linking (#54241) but I didn't get far enough to finish it. I suspect it probably hits this issue.

@ghost ghost removed the in-pr There is an active PR which will close this issue when it is merged label Jul 6, 2021
@akoeplinger
Copy link
Member

We're actually missing <_MonoCMakeArgs Include="-DENABLE_ICALL_EXPORT=1"/> in mono.proj around L345 for iOS device builds to match the --enable-icall-export that we had in mono/mono.

@akoeplinger akoeplinger reopened this Jul 8, 2021
@ghost ghost added the in-pr There is an active PR which will close this issue when it is merged label Jul 9, 2021
@ghost ghost removed the in-pr There is an active PR which will close this issue when it is merged label Jul 9, 2021
@rolfbjarne
Copy link
Member Author

This is still an issue with 6.0.0-rc.1.21369.14:

$ nm  packages/microsoft.netcore.app.runtime.mono.iossimulator-x64/6.0.0-rc.1.21369.14/runtimes/iossimulator-x64/native/libmonosgen-2.0.dylib | grep mono_array_new_1
00000000001a3d70 t _mono_array_new_1

@rolfbjarne rolfbjarne reopened this Jul 22, 2021
@vargaz
Copy link
Contributor

vargaz commented Jul 22, 2021

The PR above only fixes it for device builds, simulator builds need the same define.

@vargaz
Copy link
Contributor

vargaz commented Jul 22, 2021

The problem is that those are jit icalls which are only marked with ICALL_EXTERN_C which doesn't actually export them.

@ghost ghost added the in-pr There is an active PR which will close this issue when it is merged label Jul 22, 2021
vargaz added a commit to vargaz/runtime that referenced this issue Jul 22, 2021
vargaz added a commit that referenced this issue Jul 22, 2021
@ghost ghost removed the in-pr There is an active PR which will close this issue when it is merged label Jul 22, 2021
@rolfbjarne
Copy link
Member Author

This is still a problem in Mac Catalyst and macOS (but not on iOS):

$ for i in packages/microsoft.netcore.app.runtime.*/6.0.0-*/runtimes/*/native/libmonosgen-2.0.dylib packages/microsoft.netcore.app.runtime.*/6.0.0-*/runtimes/*/native/libcoreclr.dylib; do echo $i; nm $i | grep mono_array_new_1; done
packages/microsoft.netcore.app.runtime.mono.ios-arm64/6.0.0-rc.1.21376.24/runtimes/ios-arm64/native/libmonosgen-2.0.dylib
000000000013e264 T _mono_array_new_1
packages/microsoft.netcore.app.runtime.mono.iossimulator-x64/6.0.0-rc.1.21376.24/runtimes/iossimulator-x64/native/libmonosgen-2.0.dylib
00000000001a3cb0 T _mono_array_new_1
packages/microsoft.netcore.app.runtime.mono.maccatalyst-arm64/6.0.0-rc.1.21376.24/runtimes/maccatalyst-arm64/native/libmonosgen-2.0.dylib
0000000000185f24 t _mono_array_new_1
packages/microsoft.netcore.app.runtime.mono.maccatalyst-x64/6.0.0-rc.1.21376.24/runtimes/maccatalyst-x64/native/libmonosgen-2.0.dylib
00000000001ace70 t _mono_array_new_1
packages/microsoft.netcore.app.runtime.mono.osx-x64/6.0.0-rc.1.21376.24/runtimes/osx-x64/native/libcoreclr.dylib
00000000001afb60 t _mono_array_new_1

@rolfbjarne rolfbjarne reopened this Jul 29, 2021
vargaz added a commit to vargaz/runtime that referenced this issue Jul 29, 2021
@ghost ghost added the in-pr There is an active PR which will close this issue when it is merged label Jul 29, 2021
@ghost ghost removed the in-pr There is an active PR which will close this issue when it is merged label Jul 29, 2021
@rolfbjarne
Copy link
Member Author

This isn't fully fixed yet - it's fixed for some symbols, but not others. I tested for mono_array_new_1 previously, and that seems to be public on all platforms now. However there are numerous other symbols which are still private (on all platforms):

Undefined symbols for architecture arm64:
  "_mini_llvm_init_method", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvm_init_method_intptr_intptr_intptr_intptr in System.Private.CoreLib.dll.o
  "_mini_llvmonly_get_interp_entry", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvmonly_get_interp_entry_intptr in System.Private.CoreLib.dll.o
  "_mini_llvmonly_init_delegate", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvmonly_init_delegate_object in System.Private.CoreLib.dll.o
  "_mini_llvmonly_init_delegate_virtual", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvmonly_init_delegate_virtual_object_object_intptr in System.Private.CoreLib.dll.o
  "_mini_llvmonly_init_vtable_slot", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvmonly_init_vtable_slot_intptr_int in System.Private.CoreLib.dll.o
  "_mini_llvmonly_pop_lmf", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvmonly_pop_lmf_intptr in System.Private.CoreLib.dll.o
  "_mini_llvmonly_resolve_generic_virtual_call", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvmonly_resolve_generic_virtual_call_intptr_int_intptr in System.Private.CoreLib.dll.o
  "_mini_llvmonly_resolve_generic_virtual_iface_call", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvmonly_resolve_generic_virtual_iface_call_intptr_int_intptr in System.Private.CoreLib.dll.o
  "_mini_llvmonly_resolve_iface_call_gsharedvt", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvmonly_resolve_iface_call_gsharedvt_object_int_intptr_intptr in System.Private.CoreLib.dll.o
  "_mini_llvmonly_resolve_vcall_gsharedvt", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvmonly_resolve_vcall_gsharedvt_object_int_intptr_intptr in System.Private.CoreLib.dll.o
  "_mini_llvmonly_throw_aot_failed_exception", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvmonly_throw_aot_failed_exception_intptr in System.Private.CoreLib.dll.o
  "_mini_llvmonly_throw_nullref_exception", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvmonly_throw_nullref_exception in System.Private.CoreLib.dll.o
  "_mono_debug_personality", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mono_debug_personality_int_int_intptr_intptr_intptr in System.Private.CoreLib.dll.o
  "_mono_gc_wbarrier_range_copy", referenced from:
      MonoTouchFixtures_Security_KeyTest_BenchmarkManaged4096 in monotouchtest.dll.o
      System_Collections_Immutable_ImmutableHashSet_1_T_REF_GetEnumerator in System.Collections.Immutable.dll.o
      System_Collections_Immutable_ImmutableHashSet_1_T_REF_System_Collections_Generic_IEnumerable_T_GetEnumerator in System.Collections.Immutable.dll.o
      System_Collections_Immutable_ImmutableHashSet_1_T_REF_System_Collections_IEnumerable_GetEnumerator in System.Collections.Immutable.dll.o
      System_Collections_Immutable_ImmutableHashSet_1_Builder_T_REF_GetEnumerator in System.Collections.Immutable.dll.o
      System_Collections_Immutable_ImmutableHashSet_1_Builder_T_REF_System_Collections_Generic_IEnumerable_T_GetEnumerator in System.Collections.Immutable.dll.o
      System_Collections_Immutable_ImmutableHashSet_1_Builder_T_REF_System_Collections_IEnumerable_GetEnumerator in System.Collections.Immutable.dll.o
      ...
  "_mono_interp_entry_from_trampoline", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mono_interp_entry_from_trampoline_intptr_intptr in System.Private.CoreLib.dll.o
  "_mono_interp_to_native_trampoline", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mono_interp_to_native_trampoline_intptr_intptr in System.Private.CoreLib.dll.o
  "_mono_llvm_load_exception", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mono_llvm_load_exception in System.Private.CoreLib.dll.o
  "_mono_llvm_match_exception", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mono_llvm_match_exception_intptr_int_int_intptr_object in System.Private.CoreLib.dll.o
  "_mono_llvm_resume_exception", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mono_llvm_resume_exception in System.Private.CoreLib.dll.o
  "_mono_llvm_rethrow_exception", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mono_llvm_rethrow_exception_object in System.Private.CoreLib.dll.o
  "_mono_llvm_throw_corlib_exception", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mono_llvm_throw_corlib_exception_int in System.Private.CoreLib.dll.o
  "_mono_llvm_throw_exception", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mono_llvm_throw_exception_object in System.Private.CoreLib.dll.o
  "_mono_resume_unwind", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mono_resume_unwind_intptr in System.Private.CoreLib.dll.o
  "_mono_tls_get_domain_extern", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mono_tls_get_domain_extern in System.Private.CoreLib.dll.o
  "_mono_tls_get_jit_tls_extern", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mono_tls_get_jit_tls_extern in System.Private.CoreLib.dll.o
  "_mono_tls_get_lmf_addr_extern", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mono_tls_get_lmf_addr_extern in System.Private.CoreLib.dll.o
  "_mono_tls_get_sgen_thread_info_extern", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mono_tls_get_sgen_thread_info_extern in System.Private.CoreLib.dll.o
  "_mono_tls_get_thread_extern", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mono_tls_get_thread_extern in System.Private.CoreLib.dll.o
  "_monoeg_g_free", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_monoeg_g_free_intptr in System.Private.CoreLib.dll.o

Testing for the first (mini_llvm_init_method) and last (monoeg_g_free) symbol shows they're private for all platforms:

$ for i in $(find microsoft.* -name libmonosgen-2.0.dylib -or -name libcoreclr.dylib | grep -v DWARF | grep mono); do echo $i; nm $i | grep monoeg_g_free; done
microsoft.netcore.app.runtime.mono.ios-arm/6.0.0-rc.1.21380.13/runtimes/ios-arm/native/libmonosgen-2.0.dylib
0015330a t _monoeg_g_free
0015b186 t _monoeg_g_free
microsoft.netcore.app.runtime.mono.ios-arm64/6.0.0-rc.1.21380.13/runtimes/ios-arm64/native/libmonosgen-2.0.dylib
00000000001b0e18 t _monoeg_g_free
microsoft.netcore.app.runtime.mono.iossimulator-x64/6.0.0-rc.1.21380.13/runtimes/iossimulator-x64/native/libmonosgen-2.0.dylib
00000000002a78a0 t _monoeg_g_free
microsoft.netcore.app.runtime.mono.iossimulator-x86/6.0.0-rc.1.21380.13/runtimes/iossimulator-x86/native/libmonosgen-2.0.dylib
0028dd50 t _monoeg_g_free
microsoft.netcore.app.runtime.mono.maccatalyst-arm64/6.0.0-rc.1.21380.13/runtimes/maccatalyst-arm64/native/libmonosgen-2.0.dylib
000000000025335c t _monoeg_g_free
microsoft.netcore.app.runtime.mono.maccatalyst-x64/6.0.0-rc.1.21380.13/runtimes/maccatalyst-x64/native/libmonosgen-2.0.dylib
00000000002ad0a0 t _monoeg_g_free
microsoft.netcore.app.runtime.mono.osx-arm64/6.0.0-rc.1.21380.13/runtimes/osx-arm64/native/libcoreclr.dylib
0000000000256df8 t _monoeg_g_free
microsoft.netcore.app.runtime.mono.osx-x64/6.0.0-rc.1.21380.13/runtimes/osx-x64/native/libcoreclr.dylib
00000000002b4200 t _monoeg_g_free
microsoft.netcore.app.runtime.mono.tvos-arm64/6.0.0-rc.1.21380.13/runtimes/tvos-arm64/native/libmonosgen-2.0.dylib
00000000001b1dec t _monoeg_g_free
microsoft.netcore.app.runtime.mono.tvossimulator-x64/6.0.0-rc.1.21380.13/runtimes/tvossimulator-x64/native/libmonosgen-2.0.dylib
00000000002a4910 t _monoeg_g_free

$ for i in $(find microsoft.* -name libmonosgen-2.0.dylib -or -name libcoreclr.dylib | grep -v DWARF | grep mono); do echo $i; nm $i | grep mini_llvm_init_method; done
microsoft.netcore.app.runtime.mono.ios-arm/6.0.0-rc.1.21380.13/runtimes/ios-arm/native/libmonosgen-2.0.dylib
0011cf8c t _mini_llvm_init_method
00121b9c t _mini_llvm_init_method
microsoft.netcore.app.runtime.mono.ios-arm64/6.0.0-rc.1.21380.13/runtimes/ios-arm64/native/libmonosgen-2.0.dylib
000000000016b618 t _mini_llvm_init_method
microsoft.netcore.app.runtime.mono.iossimulator-x64/6.0.0-rc.1.21380.13/runtimes/iossimulator-x64/native/libmonosgen-2.0.dylib
0000000000235d30 t _mini_llvm_init_method
microsoft.netcore.app.runtime.mono.iossimulator-x86/6.0.0-rc.1.21380.13/runtimes/iossimulator-x86/native/libmonosgen-2.0.dylib
00233580 t _mini_llvm_init_method
microsoft.netcore.app.runtime.mono.maccatalyst-arm64/6.0.0-rc.1.21380.13/runtimes/maccatalyst-arm64/native/libmonosgen-2.0.dylib
0000000000200c10 t _mini_llvm_init_method
microsoft.netcore.app.runtime.mono.maccatalyst-x64/6.0.0-rc.1.21380.13/runtimes/maccatalyst-x64/native/libmonosgen-2.0.dylib
000000000023b130 t _mini_llvm_init_method
microsoft.netcore.app.runtime.mono.osx-arm64/6.0.0-rc.1.21380.13/runtimes/osx-arm64/native/libcoreclr.dylib
00000000002046ac t _mini_llvm_init_method
microsoft.netcore.app.runtime.mono.osx-x64/6.0.0-rc.1.21380.13/runtimes/osx-x64/native/libcoreclr.dylib
0000000000242350 t _mini_llvm_init_method
microsoft.netcore.app.runtime.mono.tvos-arm64/6.0.0-rc.1.21380.13/runtimes/tvos-arm64/native/libmonosgen-2.0.dylib
000000000016ca00 t _mini_llvm_init_method
microsoft.netcore.app.runtime.mono.tvossimulator-x64/6.0.0-rc.1.21380.13/runtimes/tvossimulator-x64/native/libmonosgen-2.0.dylib
00000000002331c0 t _mini_llvm_init_method

@vargaz
Copy link
Contributor

vargaz commented Aug 4, 2021

Maybe we should avoid using direct-icalls in non-static mode as a simpler fix.

@rolfbjarne
Copy link
Member Author

Maybe we should avoid using direct-icalls in non-static mode as a simpler fix.

I can have a look and see if that works.

rolfbjarne added a commit to rolfbjarne/xamarin-macios that referenced this issue Aug 4, 2021
@vargaz
Copy link
Contributor

vargaz commented Aug 8, 2021

Did it work out ?

rolfbjarne added a commit to xamarin/xamarin-macios that referenced this issue Aug 9, 2021
* [dotnet] Remove workaround for private symbols for AOT.

* [tools] Make Application.AotArguments a list of string.

This is just a simple refactoring to make Application.AotArguments a list of
strings instead of a comma-separated list of values.

* [tools] Only use direct-icalls when linking mono statically.

Ref: dotnet/runtime#55000

* [mtouch] Fix aot arguments comparison.

* [tests] Adjust mtouch test according to mtouch changes.

* [tests] Add minimum OS version to the Mac Catalyst variation of the MySimpleApp test case.
@rolfbjarne
Copy link
Member Author

Did it work out ?

Yes, it did, so this can be closed.

@ghost ghost locked as resolved and limited conversation to collaborators Sep 8, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.