-
Notifications
You must be signed in to change notification settings - Fork 3.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Python][Gandiva] Could not create LLJIT instance: Symbols not found: [ llvm_orc_registerEHFrameSectionWrapper ] #39695
Comments
@niyue Have you seen this error? |
I haven't seen such issue previously. I will give it a try later to see if I can reproduce it. What is the difference between your env and the envs we have in arrow CI? Are you using LLVM 17 (which may be related with this issue llvm/llvm-project#74671) |
Yes. I'm using LLVM 17. I'll try LLVM 16. |
I tried LLVM 16 and it worked. Thanks! Can we solve this? Or should we reject LLVM 17? |
I pulled the last main branch (55afcf0) tonight, and tried LLVM 14 + Ubuntu 20.04 + Docker for Mac, and it works (there is one test case skipped)
I am not sure what the exact cause for this issue yet, and I will give it another try for LLVM 17 to see if it works. |
I could reproduce the same issue using LLVM 17 + Ubuntu 23.10
So far I played with CMake for a few hours trying to use the |
Thanks! |
@kou Sorry I am still not able to figure it out yet. I tried using the |
Thanks! No problem. I'll also take a look at this later. |
Here are solutions for this case but all of them are subtlety...:
import sys
import os
sys.setdlopenflags(sys.getdlopenflags() + os.RTLD_GLOBAL)
import pyarrow.gandiva Python use
Use $ LD_PRELOAD=/lib/llvm-17/lib/libLLVM-17.so python ... Use import pyarrow.gandiva as gandiva
import ctypes
import os
import sys
ctypes.CDLL("libLLVM-17.so.1", sys.getdlopenflags() + os.RTLD_GLOBAL) Note: We can't use the |
#39622 (comment) is similar but a different case. |
That is much more difficult than I thought. It makes using gandiva in python much more difficult since it requires installation of LLVM 17 itself. I will try summarize this issue and consult LLVM community to see if there is alternative approach. |
Describe the bug, including details regarding any error messages, version, and platform.
I'm not sure that this is a problem of Gandiva/PyArrow or my environment.
I'm verifying Apache Arrow 15.0.0 RC1. I got the following failures:
Note that Gandiva tests written in C++ were passed.
Component(s)
C++ - Gandiva, Python
The text was updated successfully, but these errors were encountered: