Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MacOS + Graal AOT throws exception due to incorrect osArch value (amd64 instead of x86_64) #336

Closed
Songzhizong opened this issue Dec 20, 2024 · 3 comments

Comments

@Songzhizong
Copy link

Problem Description:

When running on MacOS with Graal AOT, the following exception occurs:

Unsupported OS/arch, cannot find /darwin/amd64/libzstd-jni-1.5.6-8.dylib or load zstd-jni-1.5.6-8 from system libraries.

This exception occurs because System.getProperty("os.arch") returns amd64 instead of the expected x86_64, causing the system to be unable to find the corresponding libzstd-jni file.

Root Cause:

On MacOS, when running Graal AOT, os.arch is set to amd64, which prevents Zstd JNI from loading the correct .dylib file since the path depends on the x86_64 architecture rather than amd64.

Possible Solution:

To fix this issue, it might be necessary to modify the implementation of the com.github.luben.zstd.util.Native.osArch() method. Specifically, when osName is darwin and osArch is amd64, it should return x86_64, allowing the system to find the correct library file.

Expected Behavior:

After the fix, Graal AOT on MacOS should be able to correctly load the zstd-jni library and no longer throw the related exception.

Environment Information:

  • Operating System: MacOS
  • Architecture: amd64 (returned value)
  • Tool: Graal AOT
  • Zstd Version: 1.5.6-8
@luben
Copy link
Owner

luben commented Jan 1, 2025

Yes, looks like the proposed work-around will work. Interesting though that the normal JVM returns x86_64 but Graal returns amd64

luben added a commit that referenced this issue Jan 1, 2025
The normal JVM returns `x86_64` but Graal AOT returns `amd64`.

Addresses #336
@luben
Copy link
Owner

luben commented Jan 1, 2025

I pushed v1.5.6-9 with suggested workaround. Give it a try.

@Songzhizong
Copy link
Author

v1.5.6-9 solves this problem

dongjoon-hyun added a commit to apache/spark that referenced this issue Jan 7, 2025
### What changes were proposed in this pull request?

This PR aims to upgrade `zstd-jni` to 1.5.6-9.

### Why are the changes needed?

To bring the latest bug fix:
- luben/zstd-jni#336

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Pass the CIs.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #49375 from dongjoon-hyun/SPARK-50741.

Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants