Build ruby with shared library

[mloughran]

I'm not sure whether there is a good reason for this, but ruby-build does not build a ruby shared library.

This caused an issue for me when trying to build the zookeeper gem (ruby 1.9.2-p290), which fails to compile.

I solved the issue by recompiling ruby with

./configure --prefix=$HOME/.rbenv/versions/1.9.2-p290 --enable-shared

RVM appears to do this by default - can ruby-build do the same?

Thanks!

[sstephenson]

That seems like a good default.

[sstephenson]

I just added support for passing options to configure with the CONFIGURE_OPTS variable. You can use this to set --enable-shared until I've had time to test the option with all the various definitions.

$ RUBY_CONFIGURE_OPTS=--enable-shared ruby-build 1.9.2-p290 …

[mloughran]

Thanks!

[lilith]

diff_match_patch_native also requires the shared library. Upvote for --enable-shared by default?

[mislav]

From my limited knowledge, it seems that --enable-shared can't hurt to always have enabled, so I'm reopening this until we decide on it.

@nathanaeljones You didn't argue for your case well enough. What is diff_match_patch_native and why do you need it?

[lilith]

It's the only reasonably fast way to get accurate diffs between large strings in ruby. The native ruby version takes upwards of 30 seconds on a 33kb file.

Any gem that depends on Rice will require --enable-shared... It's pretty important for C++ interop.

[lilith]

Over the last two months we've wasted more than 80 hours due to this bug. CONFIGURE_OPTS does not work reliably; often it takes 4-5 re-installs before it works, with no clear difference as to why. Identical ~/.bash_profile and a clean shell state doesn't help. All 8 machines are brand-new MBPs with Mavericks. Another fails to build the shared libraries no matter how many attempts are made. I've compared the version numbers of all the build tools used during the process, and they're all the same.

This is probably the most costly bug we've encountered in the last 5 years. C++ and Ruby interop is important, no? Shared libs are built reliably, by default, with RVM, and despite its multitude of flaws, that one detail turned out more important than anything else in the rvm/rbenv balance.

[mislav]

I'm sorry that you lost a lot of time on this. Any idea why setting CONFIGURE_OPTS would not have effect 4 of out 5 times? It smells like it might be a bug in Ruby's build system.

Do you have a Ruby script I could run on a freshly compiled Ruby to test whether it was successfully built with --enable-shared?

[lilith]

I haven't found an easier way to test for --enable-shared than gem install rice. If there's an easier way to test for the presence of the .so files, let me know and I'll update our docs.

I don't have any solid leads on the inconsistent behavior. I lined up 6 of the laptops last week and did the full uninstall/reinstall on all of them. Multiple times. There were a few errors, but I excluded all those that I found.

For example, CONFIGURE_OPTS=--enable-shared rbenv install 2.0.0-p353 doesn't work, but CONFIGURE_OPTS="--enable-shared" rbenv install 2.0.0-p353 does (eventually). But not if you put them on two separate lines, or if you copy/paste from GitHub (I'm guessing whitespace weirdness, perhaps non-breaking spaces or something?).

The ruby build never fails - it always succeeds. The flag just isn't getting to the actual compilation step.

My uneducated guess is that somehow the bash variable is going out of scope too early, and not making it to ruby-build. IMHO, this should be a default, not an unreliable flag.

[DanielHeath]

Ouch. Just switched to chruby + ruby-build from RVM last week.

+1, no downside to building libruby & it's used by (e.g.) mapnik_ruby (via rice).

[kiranos]

why not add this as default? I fail to see the reason. Everyone seems to be in favor of rbenv adding this as default?

[mislav]

What exactly requires the shared library and would there be any downsides to enabling it for all builds?

[jeremy]

Ruby doesn't build as a shared lib by default. Most things link fine against the static lib. Generally, those who need a shared lib know. We could expose a toplevel option to make it easier/more discoverable, though.

Rice does explicitly state "Also Rice requires a Ruby built with –enable-shared and will not install properly against a Ruby with only static libraries." but it doesn't give any guidance on how to do that with rbenv, rvm, etc.

A little more background: https://www.ruby-forum.com/topic/4422189

Also, can be more specific than blanket CONFIGURE_OPTS:

RUBY_CONFIGURE_OPTS=--enable-shared rbenv install 2.2.3

[mislav]

Is there a short code snippet one could use in ruby -e "..." to test if the current Ruby has shared lib support?

[ghost]

Shared libraries are libraries that are loaded by programs when they start. When a shared library is installed properly, all programs that start afterwards automatically use the new shared library. So basically if you have a program that needs a shared library (in this case ruby libruby.so.2.2.0 or similar) the program will fail (to start hopefully) if it doesn't exist. if you want to know if you have the shared library you can do a simple search for the file libruby.so*, but as jeremy indicated, you don't need it, and those that do will know.

[evie404]

seems like you can test it with RbConfig::CONFIG["ENABLE_SHARED"] == "no"

https://github.com/jasonroelofs/rice/blob/master/extconf.rb#L21

[cmol]

Or ruby -e "require 'rbconfig'; puts RbConfig::CONFIG['ENABLE_SHARED']" for the lazy.

I just came across the same issue with the cwiid gem, that didn't compile until ruby was installed with --enable-shared. My "system" ruby comes with --enable-shared as well so I cannot see why this shouldn't be default for ruby-build.

[Ruin0x11]

I wasted several hours trying to install qtbindings before realizing --enable-shared wasn't set by default. I also don't see any reason not to set it by default.

[lilith]

Can this please be re-opened? Lack of shared by default breaks so many applications and gems.

[mislav]

@nathanaeljones This issue was open all along.

If someone wants to contribute a workaround that does --enable-shared by default for MRI, that would be welcome. I didn't realize this affected so many people, and so far we are not aware of a potential downside of doing that by default.

[rubypanther]

Is the most desirable way to add this by adding an enable_shared package target?
For example right now ruby-2.3.0 has this line in the definition file:
install_package "ruby-2.3.0" "https://cache.ruby-lang.org/pub/ruby/2.3/ruby-2.3.0.tar.bz2#ec7579eaba2e4c402a089dbc86c98e5f1f62507880fd800b9b34ca30166bfa5e" ldflags_dirs standard verify_openssl
and I've got a fork that adds a build_package_enable_shared function in the ruby-build script, and then the package options just change to ldflags_dirs enable_shared standard verify_openssl
That way it can be controlled from the definitions file. Good, bad, comments?

edit to add: Here is what I came up with, as you can see it is a minimalist approach without any safety checks: rubypanther@4f7e4c7

[yashi]

Why not just pass --enable-shared when building Ruby? Even if we change the current MRI, we don't get backported to older ones. I'm not sure what's holding the change to ruby-build. This little bug seems have wasted enough time for so many people already. Can we fix this now?

[eregon]

Something to consider is that --enable-shared might affect performance.

[megatux]

Is this option still valid? I remember it was working, but trying it now throws me a configure: error: unrecognized option: --enable_shared'` error :

$ RUBY_CONFIGURE_OPTS=--enable_shared asdf install ruby 2.5.7
Downloading ruby-build...
Clonando en 'ruby-build-source'...
remote: Enumerating objects: 81, done.
remote: Counting objects: 100% (81/81), done.
remote: Compressing objects: 100% (51/51), done.
remote: Total 9872 (delta 40), reused 48 (delta 18), pack-reused 9791
Recibiendo objetos: 100% (9872/9872), 2.10 MiB | 248.00 KiB/s, listo.
Resolviendo deltas: 100% (6405/6405), listo.
Nota: actualizando el árbol de trabajo 'v20191004'.

Te encuentras en estado 'detached HEAD'. Puedes revisar por aquí, hacer
cambios experimentales y confirmarlos, y puedes descartar cualquier
commit que hayas hecho en este estado sin impactar a tu rama realizando
otro checkout.

Si quieres crear una nueva rama para mantener los commits que has creado,
puedes hacerlo (ahora o después) usando -b con el comando checkout. Ejemplo:

  git checkout -b <nombre-de-nueva-rama>

HEAD está ahora en da8e0b4 Bump version to v20191004
Downloading ruby-2.5.7.tar.bz2...
-> https://cache.ruby-lang.org/pub/ruby/2.5/ruby-2.5.7.tar.bz2
Installing ruby-2.5.7...

BUILD FAILED (elementary 5.0 using ruby-build 20191004)

Inspect or clean up the working tree at /tmp/ruby-build.20191004095220.15967
Results logged to /tmp/ruby-build.20191004095220.15967.log

Last 10 log lines:
/tmp/ruby-build.20191004095220.15967 ~/code/other/r10k
/tmp/ruby-build.20191004095220.15967/ruby-2.5.7 /tmp/ruby-build.20191004095220.15967 ~/code/other/r10k
configure: error: unrecognized option: `--enable_shared'
Try `./configure --help' for more information

[eregon]

@megatux It's --enable-shared, not --enable_shared

[megatux]

🤦‍♂️ thx @eregon (I copied it from a comment above).

[ankane]

Hey, would love to try and move this forward. I think getting Rice and the rbenv defaults to play nicely would be a big/foundational step forward for the Ruby ML ecosystem (more context here: ruby-rice/rice#112 (comment)).

@eregon it sounds like performance may be a concern. I'm happy to help test this. What's the best way to do that?

To all: are there any other concerns with making --enable-shared the default?

Edit: from what I can tell, rvm, Ubuntu, and Heroku all enable this by default.

[mislav]

@ankane Thank you for offering to help! This would be a good starting point: #1368

[ankane]

Awesome, thanks @mislav (for this and all you do for JS and Rails)! That PR does the trick on my machine.

[shehackedyou]

Happy to see this, wanted to experiment with connecting directly with the Ruby c libraries with Go purego and finally build sensible access to ruby that doesn't involve embedding a binary.