[LLJIT][AArch64] HowToUseLLJI: Symbols not found

[banach-space]

Hiya,

I'm trying to use HowToUseJIT on AArch64 running on Linux, but sadly it fails:

bin/HowToUseLLJIT: Symbols not found: [ llvm_orc_registerEHFrameSectionWrapper ]

Below is a full debug dump:

$ bin/HowToUseLLJIT --debug
Args: bin/HowToUseLLJIT --debug
Preparing to create LLJIT instance...
  No explicitly set JITTargetMachineBuilder. Detecting host...
  JITTargetMachineBuilder is   {
    Triple = "aarch64-unknown-linux-gnu"
    CPU = "neoverse-v1"
    Features = "+sve,+crc,+crypto,+lse,+neon,+fp-armv8"
    Options = <not-printable>
    Relocation Model = unspecified (will use target default)
    Code Model = unspecified (will use target default)
    Optimization Level = Default
  }
  Pre-constructed ExecutionSession: No
  DataLayout: None (will be created by JITTargetMachineBuilder)
  Custom object-linking-layer creator: No
  Custom compile-function creator: No
  Custom platform-setup function: No
  Number of compile threads: 0 (code will be compiled on the execution thread)
ExecutorProcessControl not specified, Creating SelfExecutorProcessControl instance
Ending ExecutionSession 0xaaaaf0ec5bc0
In main removing tracker 0xaaaaf0eeff80
In main destroying tracker 0xaaaaf0eeff80
Destroying JITDylib main
bin/HowToUseLLJIT: Symbols not found: [ llvm_orc_registerEHFrameSectionWrapper ]

Somewhat confusingly, this example works just fine on Mac M1:

bin/HowToUseLLJIT --debug-only=orc
Preparing to create LLJIT instance...
  No explicitly set JITTargetMachineBuilder. Detecting host...
  JITTargetMachineBuilder is   {
    Triple = "arm64-apple-darwin22.3.0"
    CPU = "apple-m1"
    Features = ""
    Options = <not-printable>
    Relocation Model = unspecified (will use target default)
    Code Model = unspecified (will use target default)
    Optimization Level = Default
  }
  Pre-constructed ExecutionSession: No
  DataLayout: None (will be created by JITTargetMachineBuilder)
  Custom object-linking-layer creator: No
  Custom compile-function creator: No
  Custom platform-setup function: No
  Number of compile threads: 0 (code will be compiled on the execution thread)
ExecutorProcessControl not specified, Creating SelfExecutorProcessControl instance
Setting up GenericLLVMIRPlatform support for LLJIT
Defining MU <Absolute Symbols> for main (tracker: 0x0, default will be used)
  { ("___lljit.cxa_atexit_helper", [Data][Hidden]), ("___lljit.platform_support_instance", [Data]) }
Defining MU <Absolute Symbols> for main (tracker: 0x0, default will be used)
  { ("___lljit.atexit_helper", [Data][Hidden]), ("___lljit.run_atexits_helper", [Data][Hidden]) }
Defining MU __standard_lib for main (tracker: default)  { ("___lljit_run_atexits", [Callable][Hidden]), ("_atexit", [Callable][Hidden]), ("___dso_handle", [Data]) }
Defining MU __standard_lib for main (tracker: default)  { ("___cxa_atexit", [Callable]) }
Defining MU test for main (tracker: default)  { ("_add1", [Callable]) }
Looking up { (_add1, RequiredSymbol) } in [ ("main", MatchAllSymbols) ] (required state: Ready)
Dispatching MaterializationUnits...
Done dispatching MaterializationUnits.
Entering OL_applyQueryPhase1:
  Lookup kind: Static
  Search order: [ ("main", MatchAllSymbols) ], Current index = 0 (entering new JITDylib)
  Lookup set: { (_add1, RequiredSymbol) }
  Definition generator candidates: { (_add1, RequiredSymbol) }
  Definition generator non-candidates: { }
Visiting "main" (MatchAllSymbols) with lookup set { (_add1, RequiredSymbol) }:
  First time visiting main, resetting candidate sets and building generator stack
  Updating candidate set...
    Remaining candidates = { }
  No generators to run for this JITDylib.
All symbols matched.
Phase 1 succeeded.
Entering OL_completeLookup:
  Lookup kind: Static
  Search order: [ ("main", MatchAllSymbols) ], Current index = 1
  Lookup set: { (_add1, RequiredSymbol) }
  Definition generator candidates: { }
  Definition generator non-candidates: { }
Visiting "main" (MatchAllSymbols) with lookup set { (_add1, RequiredSymbol) }:
  Attempting to match "_add1" (RequiredSymbol)... matched, preparing to dispatch MU@0x6000013700e0 (test)
Stripping unmatched weakly-referenced symbols
Query successfully lodged
Adding MUs to dispatch:
  For main: Adding 1 MUs.
No dependencies to register
Dispatching MaterializationUnits...
  Dispatching "test"
Dispatching: Materialization task: test in main
Emitting, for main, MU@0x6000013700e0 ("test", { })
In main defining materializing symbols { }
In main resolving { ("_add1": 0x000000010f8c4000 [Callable]) }
In main emitting { ("_add1", [Callable]) }
Dispatching: Execute query complete callback for { ("_add1": 0x000000010f8c4000 [Callable]) }
Finished emitting, for main, MU@0x6000013700e0 ("<null module>", { })
Done dispatching MaterializationUnits.
add1(42) = 43
Ending ExecutionSession 0x13f604970
In main removing tracker 0x600003a70130
In main destroying tracker 0x600003a70130
Destroying JITDylib main

These are uncharted waters for me, but from what I can see, LLJIT fails in CreateObjectLinkingLayer.

@lhames - you seem to be most active in this area. I'm happy to dive deeper, but would appreciate some pointers. This affects the behavior of mlir-cpu-runner (specfically, the host-supports-jit flag), that I'd like to use for end-to-end tests for AArch64.

Tested using ToT: 3245bcd

Thanks for taking a look,
-Andrzej

[llvmbot]

@llvm/issue-subscribers-orcjit

[lhames]

@banach-space Looks like the HowToUseLLJIT example was missing export_executable_symbols (Symbols in ELF executables aren't exported by default, and this causes lookup of precompiled symbols in the executable to fail as you saw here).

This should be fixed by 2ad8e6e.

[banach-space]

Brilliant, thank you so much!

Follow-up specifically for mlir-cpu-runner: https://reviews.llvm.org/D146935