Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build fat binary for arm64 and x86_64 on MacOS in build.sbt #12

Merged
merged 3 commits into from
Nov 24, 2020
Merged

Build fat binary for arm64 and x86_64 on MacOS in build.sbt #12

merged 3 commits into from
Nov 24, 2020

Conversation

eatkins
Copy link
Contributor

@eatkins eatkins commented Nov 24, 2020

This PR reworks #11 so that we build a fat binary for osx that includes both arm64 and x86_64 code. It also adds the logic to the build.sbt to build the multi-arch version so that we can easily create new binaries in the future.

Rather than providing platform specific binaries, we can just make a fat
binary with x86_64 and arm64. This is done by separately compiling the
library for both architectures on osx into the target directory and then
combining them using the lipo tool into a fat binary in the resource
directory.

Using a fat binary simplifies the logic for loading the native library.
In the future we could easily ship separate versions of the x86 and
arm64 binaries. To do this, we would get rid of the buildDarwin task and
have buildDarwinX86_64 and buildDarwinArm64 write directly into an
appropriate resource directory.
By refactoring the the build slightly, we can use automatic Def.taskIf
and avoid lint warnings that occurred due to the definition of keys that
were only used in the dynamic task implementation.
We need a new osx image on appveyor that xcode 12 support to build the
apple fat binary in ci. Github actions has such an image, but I'm not
sure that we have an sbt version that would work on that image yet.
@joan38
Copy link

joan38 commented Jan 30, 2021

Hi @eatkins and @eed3si9n, I'm trying to make mill work with Apple's arm64 and we have a dependency on this lib.
I upgraded to 1.3.0 but I'm still getting this error:

Exception in thread "main" java.lang.UnsatisfiedLinkError: /Users/jgoyeau/Library/Caches/JNA/temp/jna10824615888576569736.tmp: dlopen(/Users/jgoyeau/Library/Caches/JNA/temp/jna10824615888576569736.tmp, 1): no suitable image found.  Did find:
        /Users/jgoyeau/Library/Caches/JNA/temp/jna10824615888576569736.tmp: no matching architecture in universal wrapper
        /Users/jgoyeau/Library/Caches/JNA/temp/jna10824615888576569736.tmp: no matching architecture in universal wrapper
        at java.base/java.lang.ClassLoader$NativeLibrary.load0(Native Method)
        at java.base/java.lang.ClassLoader$NativeLibrary.load(ClassLoader.java:2442)
        at java.base/java.lang.ClassLoader$NativeLibrary.loadLibrary(ClassLoader.java:2498)
        at java.base/java.lang.ClassLoader.loadLibrary0(ClassLoader.java:2694)
        at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2627)
        at java.base/java.lang.Runtime.load0(Runtime.java:768)
        at java.base/java.lang.System.load(System.java:1837)
        at com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:1018)
        at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:988)
        at com.sun.jna.Native.<clinit>(Native.java:195)
        at org.scalasbt.ipcsocket.UnixDomainSocketLibrary.<clinit>(UnixDomainSocketLibrary.java:129)
        at org.scalasbt.ipcsocket.JNAUnixDomainSocketLibraryProvider.socket(UnixDomainSocketLibrary.java:165)
        at org.scalasbt.ipcsocket.UnixDomainServerSocket.<init>(UnixDomainServerSocket.java:109)
        at org.scalasbt.ipcsocket.UnixDomainServerSocket.<init>(UnixDomainServerSocket.java:87)
        at mill.main.Server.$anonfun$run$2(MillServerMain.scala:89)
        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
        at mill.main.Server$.lockBlock(MillServerMain.scala:201)
        at mill.main.Server.$anonfun$run$1(MillServerMain.scala:82)
        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
        at mill.main.Server$.tryLockBlock(MillServerMain.scala:208)
        at mill.main.Server.run(MillServerMain.scala:79)
        at mill.main.MillServerMain$.main(MillServerMain.scala:43)
        at mill.main.MillServerMain.main(MillServerMain.scala)

I was wondering if by any chance you had any hints? Are we supposed to change our usage of the lib to take advantage of this change?
Thanks

@eed3si9n
Copy link
Member

I don't think there's JNA for ARM Macs.

@eatkins eatkins deleted the fat-binary branch January 30, 2021 17:03
@eatkins
Copy link
Contributor Author

eatkins commented Jan 30, 2021

There is a JNI implementation that targets arm64 on osx but you have to change your code to optionally use it. JNI isn't supported on all platforms so you will probably have to handle the logic for platform detection yourself.

public UnixDomainServerSocket(String path, boolean useJNI) throws IOException {

@joan38
Copy link

joan38 commented Jan 30, 2021

Are you talking about the boolean useJNI parameter? Because we are currently passing false all the time.
We are using this constructor:

public UnixDomainServerSocket(String path) throws IOException {
this(DEFAULT_BACKLOG, path, false);

@joan38
Copy link

joan38 commented Jan 31, 2021

Ah I see, you mean I need to pass JNI true there.
I tried and now I'm one step further hitting:

Exception in thread "MillServerActionRunner" java.lang.NoClassDefFoundError: Could not initialize class org.scalasbt.ipcsocket.UnixDomainSocketLibrary
        at org.scalasbt.ipcsocket.JNAUnixDomainSocketLibraryProvider.write(UnixDomainSocketLibrary.java:247)
        at org.scalasbt.ipcsocket.UnixDomainSocket$UnixDomainSocketOutputStream.doWrite(UnixDomainSocket.java:185)
        at org.scalasbt.ipcsocket.UnixDomainSocket$UnixDomainSocketOutputStream.write(UnixDomainSocket.java:169)

It's like if the UnixDomainSocketLibraryProvider.get(useJNI) gives me JNAUnixDomainSocketLibraryProvider.instance() instead of JNIUnixDomainSocketLibraryProvider.instance().

@eatkins
Copy link
Contributor Author

eatkins commented Feb 6, 2021

You also need to set the JNI parameter when constructing the UnixDomainSocket not just the UnixDomainServerSocket.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants