Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ruby: build extensions for arm64-darwin #8232

Merged
merged 3 commits into from
Oct 14, 2021

Conversation

qnighy
Copy link
Contributor

@qnighy qnighy commented Jan 25, 2021

This PR adds builds for arm64-darwin. It also renames universal-darwin back to x86_64-darwin to avoid confusion. These arch names are already used in nokogiri, so I think it's fine to rename it back to them.

See #8231 for a different approach.

@haberman
Copy link
Member

I'm seeing the following error in the "MacOS Ruby Release" test. Perhaps this means we need our macOS build machine to be updated to something that supports aarch64?

Found Ruby version 3.0.0 for platform aarch64-darwin (/Users/kbuilder/.rake-compiler/ruby/arm64-darwin/ruby-3.0.0/lib/ruby/3.0.0/aarch64-darwin/rbconfig.rb)
Found Ruby version 2.7.0 for platform x86_64-darwin (/Users/kbuilder/.rake-compiler/ruby/x86_64-darwin/ruby-2.7.0/lib/ruby/2.7.0/x86_64-darwin/rbconfig.rb)
Found Ruby version 3.0.0 for platform x86_64-darwin (/Users/kbuilder/.rake-compiler/ruby/x86_64-darwin/ruby-3.0.0/lib/ruby/3.0.0/x86_64-darwin/rbconfig.rb)
+ rake -f tmpfile.KytCYhXz cross-ruby VERSION=2.7.0 HOST=arm64-darwin 'MAKE=make -j8'
mkdir -p /Users/kbuilder/.rake-compiler/builds/arm64-darwin/ruby-2.7.0
cd /Users/kbuilder/.rake-compiler/builds/arm64-darwin/ruby-2.7.0
/Users/kbuilder/.rake-compiler/sources/ruby-2.7.0/configure --host=arm64-darwin --target=arm64-darwin --build=x86_64-apple-darwin18.7.0 --enable-static --disable-shared --disable-install-doc --without-gmp --with-ext= LDFLAGS=-pipe --prefix=/Users/kbuilder/.rake-compiler/ruby/arm64-darwin/ruby-2.7.0
checking for ruby... /Users/kbuilder/.rvm/rubies/ruby-2.7.0/bin/ruby
tool/config.guess already exists
tool/config.sub already exists
checking build system type... x86_64-apple-darwin18.7.0
checking host system type... Invalid configuration `arm64-darwin': machine `arm64-unknown' not recognized
configure: error: /bin/sh /Users/kbuilder/.rake-compiler/sources/ruby-2.7.0/tool/config.sub arm64-darwin failed
rake aborted!
Command failed with status (1): [/Users/kbuilder/.rake-compiler/sources/rub...]

@dlj-NaN
Copy link
Contributor

dlj-NaN commented Feb 3, 2021

What is the reason for splitting apart the darwins, instead of just using darwin-universal?

@qnighy
Copy link
Contributor Author

qnighy commented Feb 3, 2021

What is the reason for splitting apart the darwins, instead of just using darwin-universal?

Because there's no such architecture to compile Ruby for. The following command fails:

$rake -f tasks/bin/cross-ruby.rake cross-ruby VERSION="3.0.0" HOST=universal-darwin
mkdir -p /Users/qnighy/.rake-compiler/builds/universal-darwin/ruby-3.0.0
cd /Users/qnighy/.rake-compiler/builds/universal-darwin/ruby-3.0.0
/Users/qnighy/.rake-compiler/sources/ruby-3.0.0/configure --host=universal-darwin --target=universal-darwin --build=x86_64-apple-darwin19.4.0 --enable-static --disable-shared --disable-install-doc --without-gmp --with-ext= LDFLAGS=-pipe --prefix=/Users/qnighy/.rake-compiler/ruby/universal-darwin/ruby-3.0.0
checking for ruby... /Users/qnighy/.rbenv/versions/2.7.1/bin/ruby
tool/config.guess already exists
tool/config.sub already exists
checking build system type... x86_64-apple-darwin19.4.0
checking host system type... Invalid configuration `universal-darwin': machine `universal-unknown' not recognized
configure: error: /bin/sh /Users/qnighy/.rake-compiler/sources/ruby-3.0.0/tool/config.sub universal-darwin failed
rake aborted!
Command failed with status (1): [/Users/qnighy/.rake-compiler/sources/ruby-...]
/Users/qnighy/workdir/rake-compiler/tasks/bin/cross-ruby.rake:143:in `block (2 levels) in <top (required)>'
/Users/qnighy/workdir/rake-compiler/tasks/bin/cross-ruby.rake:140:in `block in <top (required)>'
Tasks: TOP => cross-ruby => install => /Users/qnighy/.rake-compiler/ruby/universal-darwin/ruby-3.0.0/bin/ruby.exe => /Users/qnighy/.rake-compiler/builds/universal-darwin/ruby-3.0.0/ruby.exe => /Users/qnighy/.rake-compiler/builds/universal-darwin/ruby-3.0.0/Makefile
(See full trace by running task with --trace)

Previously we just used x86_64-darwin as a fake universal-darwin, which is not universal at all. That's what the following lines did:

sed 's/x86_64-darwin-11/universal-darwin/' ~/.rake-compiler/config.yml > "$CROSS_RUBY"
mv "$CROSS_RUBY" ~/.rake-compiler/config.yml

@qnighy
Copy link
Contributor Author

qnighy commented Feb 3, 2021

I'm seeing the following error in the "MacOS Ruby Release" test. Perhaps this means we need our macOS build machine to be updated to something that supports aarch64?

That's probably old autoconf's problem. Ruby 3.0 builds on my local machine but Ruby 2.7 not. I should have checked that.

@qnighy
Copy link
Contributor Author

qnighy commented Feb 3, 2021

Moreover, in my local machine, it seems to produce x86_64 binaries regardless of HOST. I needed to specify the following environments to actually produce aarch64 binaries:

CC=clang --target=arm64-darwin
CXX=clang++ --target=arm64-darwin
CPP=clang -E --target=arm64-darwin
CPPFLAGS=-I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include
CFLAGS=-Wno-nullability-completeness
LDFLAGS=-L/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/lib -L/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/lib/system -F/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/

I'll incorporate it into the PR later.

@bouk
Copy link

bouk commented Mar 23, 2021

Any progress on this? I guess we need a googler to do something with this.

@bouk
Copy link

bouk commented May 21, 2021

Maybe you could update this PR to just build x86_64 for Darwin, and rely on the installing machine to build for arm64. I.e. just remove the universal-darwin name

@qnighy
Copy link
Contributor Author

qnighy commented Sep 29, 2021

That's probably old autoconf's problem.

This part was actually easy, I just needed to use aarch64-darwin instead of arm64-darwin.

@deannagarcia
Copy link
Member

Sorry for the delay and thanks for the contribution. Everything's passing right now, so going to merge this in!

@deannagarcia deannagarcia merged commit 3d223dc into protocolbuffers:master Oct 14, 2021
@ngan
Copy link

ngan commented Oct 25, 2021

@qnighy was this PR suppose produce a arm64-darwin variant? If so, unfortunately, I'm not seeing it on Rubygems:
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants