Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Homebrew compatibility #1135

Merged
merged 4 commits into from
Sep 8, 2020
Merged

Improve Homebrew compatibility #1135

merged 4 commits into from
Sep 8, 2020

Conversation

stefansundin
Copy link
Contributor

Hi!

When I develop on my Mac, I usually install the Homebrew package mysql-client instead of mysql, since I don't really need the server (I usually run that in Docker). But unfortunately mysql2 doesn't compile out of the box.

I have worked around it by first running:

bundle config build.mysql2 --with-mysql-dir="/usr/local/opt/mysql-client"

But now I decided to look into contributing a fix.

Hopefully this will be accepted and included in the next version. Thanks!

Copy link
Collaborator

@sodabrew sodabrew left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

@sodabrew
Copy link
Collaborator

sodabrew commented Sep 7, 2020

Where does the mysql-server package install in Homebrew? The list of paths may be a bit out of date at this point, given MySQL 8 being the primary version to use now.

@stefansundin
Copy link
Contributor Author

stefansundin commented Sep 7, 2020

It looks like with mysql, it finds /usr/local/bin/mysql_config and uses that. I am going to run some more testing to try to make this gem more compatible with different macOS setups. There isn't a separate mysql-server package for the server, so the mysql package installs both the server and the client.

I don't have the full Xcode installed, so I was having some issues with linking openssl with the mysql package.

ld: library not found for -lssl

I was able to fix it by adding $LDFLAGS << " -L/usr/local/opt/openssl/lib" to extconf.rb. Not sure if there's a better way. Not sure why this didn't happen with mysql-client.

I also realized that there's a [email protected] package. There's also [email protected] and [email protected]. I can add more search paths until it works with all of the combinations.

$ brew search mysql | cat
==> Formulae
automysqlbackup
mysql
mysql++
mysql-client
[email protected]
mysql-connector-c++
[email protected]
mysql-sandbox
mysql-search-replace
[email protected]
[email protected]
mysqltuner
==> Casks
mysql-connector-python
mysql-shell
mysql-utilities
mysqlworkbench
navicat-for-mysql
sqlpro-for-mysql

(used cat to just get one entry per line, like ls -1)

I will do some more testing and report back.

@stefansundin
Copy link
Contributor Author

stefansundin commented Sep 7, 2020

This explains the -lssl issue. For some reason there are libssl files that comes with mysql-client but not mysql.

$ /usr/local/bin/mysql_config --libs
-L/usr/local/Cellar/mysql/8.0.21_1/lib -lmysqlclient -lssl -lcrypto
$ /usr/local/opt/mysql-client/bin/mysql_config --libs
-L/usr/local/opt/mysql-client/lib -lmysqlclient -lssl -lcrypto
$ ls -1 /usr/local/Cellar/mysql/8.0.21_1/lib
libmysqlclient.21.dylib
libmysqlclient.a
libmysqlclient.dylib
libmysqlharness.1.dylib
libmysqlrouter.1.dylib
libmysqlrouter_http.1.dylib
libmysqlrouter_http_auth_backend.1.dylib
libmysqlrouter_http_auth_realm.1.dylib
libmysqlservices.a
mysqlrouter
pkgconfig
plugin
$ ls -1 /usr/local/opt/mysql-client/lib
libcrypto.1.1.dylib
libcrypto.dylib
libmysqlclient.21.dylib
libmysqlclient.a
libmysqlclient.dylib
libssl.1.1.dylib
libssl.dylib
pkgconfig
plugin

It may be because mysql sets -DWITH_SSL=#{Formula["[email protected]"].opt_prefix} while mysql-client sets -DWITH_SSL=yes.

Edit: It deletes a bunch of libssl files too, so maybe that's the actual reason. https://github.com/Homebrew/homebrew-core/blob/9e4e4f9c53e83d5ff555f6faaa2afc7f72266f13/Formula/mysql.rb#L72-L78

If you are ok with it, I think we should add something like this:

if RUBY_PLATFORM =~ /darwin/
  $LDFLAGS << ' -L/usr/local/opt/openssl/lib'
end

I imagine that plenty of Rubyists may be installing full Xcode to get this gem to work.

@stefansundin
Copy link
Contributor Author

I added two more commits. Let me know if you just want me to change /usr/local/opt/mysql@* to /usr/local/opt/mysql* instead, as that should catch a lot of paths. And let me know if you want me to squash the commits (or you can do it when you merge).

Thanks!

@stefansundin stefansundin changed the title Add search path for Homebrew's mysql-client Improve Homebrew compatibility Sep 7, 2020
@sodabrew
Copy link
Collaborator

sodabrew commented Sep 8, 2020

All good, will squash and merge!

@sodabrew sodabrew merged commit e2503dc into brianmario:master Sep 8, 2020
junaruga added a commit to junaruga/mysql2 that referenced this pull request Feb 12, 2021
@olivierlacan
Copy link
Contributor

Hey @stefansundin. Unless I'm mistaken there's no openssl recipe for Homebrew, only https://formulae.brew.sh/formula/[email protected]#default which doesn't install in /openssl/ anymore:

$ brew --prefix openssl
/opt/homebrew/opt/[email protected]

Homebrew itself recommends this flag:

For compilers to find [email protected] you may need to set:
  export LDFLAGS="-L/opt/homebrew/opt/[email protected]/lib"
  export CPPFLAGS="-I/opt/homebrew/opt/[email protected]/include"

So at the risk of slowing things down a bit it might be good to ask Homebrew if it's installed and then ask it where openssl is installed if at all. Something like:

brew_installed = system("command -v brew") 
if brew_installed # because brew --prefix would explode otherwise
  openssl_location = `brew --prefix openssl`.strip # because output includes \n character
  $LDFLAGS << " -L#{openssl_location}/lib" if RUBY_PLATFORM =~ /darwin/ && openssl_location
end

I'll gladly submit a follow-up PR if this seems reasonable.

@stefansundin
Copy link
Contributor Author

@olivierlacan Sounds reasonable to me!

olivierlacan added a commit to olivierlacan/mysql2 that referenced this pull request Aug 16, 2021
This is a follow-up to brianmario#1135 which added the OpenSSL flag assuming that
if the `RUBY_PLATFORM` is `darwin` (macOS):
- Homebrew is installed
- OpenSSL is installed via Homebrew

This PR:
- no longer assumes Homebrew is installed if we're on macOS
- no longer assumes OpenSSL is installed via Homebrew
- asks Homebrew for the openssl location (which will also work with the
newer [email protected] recipe)

Should prevent issues like these when running bundle install on the
rails codebase:

<details>
<summary>Bundle Install error due to: ld: warning: directory not found for option '-L/usr/local/opt/openssl/lib'</summary>
<pre>
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.

    current directory: /Users/olivierlacan/.rbenv/versions/3.0.1/lib/ruby/gems/3.0.0/bundler/gems/mysql2-7f4e844fccf6/ext/mysql2
/Users/olivierlacan/.rbenv/versions/3.0.1/bin/ruby -I /Users/olivierlacan/.rbenv/versions/3.0.1/lib/ruby/site_ruby/3.0.0 -r ./siteconf20210816-60984-ogpmsf.rb extconf.rb
checking for rb_absint_size()... yes
checking for rb_absint_singlebit_p()... yes
checking for rb_wait_for_single_fd()... yes
checking for rb_enc_interned_str() in ruby.h... yes
-----
Using mysql_config at /opt/homebrew/bin/mysql_config
-----
checking for mysql.h... yes
checking for errmsg.h... yes
checking for SSL_MODE_DISABLED in mysql.h... yes
checking for SSL_MODE_PREFERRED in mysql.h... yes
checking for SSL_MODE_REQUIRED in mysql.h... yes
checking for SSL_MODE_VERIFY_CA in mysql.h... yes
checking for SSL_MODE_VERIFY_IDENTITY in mysql.h... yes
checking for MYSQL.net.vio in mysql.h... yes
checking for MYSQL.net.pvio in mysql.h... no
checking for MYSQL_ENABLE_CLEARTEXT_PLUGIN in mysql.h... yes
checking for SERVER_QUERY_NO_GOOD_INDEX_USED in mysql.h... yes
checking for SERVER_QUERY_NO_INDEX_USED in mysql.h... yes
checking for SERVER_QUERY_WAS_SLOW in mysql.h... yes
checking for MYSQL_OPTION_MULTI_STATEMENTS_ON in mysql.h... yes
checking for MYSQL_OPTION_MULTI_STATEMENTS_OFF in mysql.h... yes
checking for my_bool in mysql.h... no
-----
Don't know how to set rpath on your system, if MySQL libraries are not in path mysql2 may not load
-----
-----
Setting libpath to /opt/homebrew/Cellar/mysql/8.0.26/lib
-----
creating Makefile

current directory: /Users/olivierlacan/.rbenv/versions/3.0.1/lib/ruby/gems/3.0.0/bundler/gems/mysql2-7f4e844fccf6/ext/mysql2
make DESTDIR\= clean

current directory: /Users/olivierlacan/.rbenv/versions/3.0.1/lib/ruby/gems/3.0.0/bundler/gems/mysql2-7f4e844fccf6/ext/mysql2
make DESTDIR\=
compiling client.c
compiling infile.c
compiling mysql2_ext.c
compiling result.c
result.c:259:55: warning: format specifies type 'long' but the argument has type 'int' [-Wformat]
        rb_field_type = rb_sprintf("decimal(%ld,%d)", precision, field->decimals);
                                            ~~~       ^~~~~~~~~
                                            %d
result.c:258:35: warning: implicit conversion loses integer precision: 'unsigned long' to 'int' [-Wshorten-64-to-32]
        precision = field->length - (field->decimals > 0 ? 2 : 1);
                  ~ ~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2 warnings generated.
compiling statement.c
linking shared-object mysql2/mysql2.bundle
ld: warning: directory not found for option '-L/usr/local/opt/openssl/lib'
ld: library not found for -lzstd
clang-12: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [mysql2.bundle] Error 1

make failed, exit code 2

Gem files will remain installed in /Users/olivierlacan/.rbenv/versions/3.0.1/lib/ruby/gems/3.0.0/bundler/gems/mysql2-7f4e844fccf6 for inspection.
Results logged to /Users/olivierlacan/.rbenv/versions/3.0.1/lib/ruby/gems/3.0.0/bundler/gems/extensions/arm64-darwin-20/3.0.0/mysql2-7f4e844fccf6/gem_make.out

An error occurred while installing mysql2 (0.5.3), and Bundler cannot continue.
</pre>
</details>
olivierlacan added a commit to olivierlacan/mysql2 that referenced this pull request Sep 6, 2021
This is a follow-up to brianmario#1135 which added the OpenSSL flag assuming that
if the `RUBY_PLATFORM` is `darwin` (macOS):
- Homebrew is installed
- OpenSSL is installed via Homebrew

This PR:
- no longer assumes Homebrew is installed if we're on macOS
- no longer assumes OpenSSL is installed via Homebrew
- asks Homebrew for the openssl location (which will also work with the
newer [email protected] recipe)

Should prevent issues like these when running bundle install on the
rails codebase:

<details>
<summary>Bundle Install error due to: ld: warning: directory not found for option '-L/usr/local/opt/openssl/lib'</summary>
<pre>
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.

    current directory: /Users/olivierlacan/.rbenv/versions/3.0.1/lib/ruby/gems/3.0.0/bundler/gems/mysql2-7f4e844fccf6/ext/mysql2
/Users/olivierlacan/.rbenv/versions/3.0.1/bin/ruby -I /Users/olivierlacan/.rbenv/versions/3.0.1/lib/ruby/site_ruby/3.0.0 -r ./siteconf20210816-60984-ogpmsf.rb extconf.rb
checking for rb_absint_size()... yes
checking for rb_absint_singlebit_p()... yes
checking for rb_wait_for_single_fd()... yes
checking for rb_enc_interned_str() in ruby.h... yes
-----
Using mysql_config at /opt/homebrew/bin/mysql_config
-----
checking for mysql.h... yes
checking for errmsg.h... yes
checking for SSL_MODE_DISABLED in mysql.h... yes
checking for SSL_MODE_PREFERRED in mysql.h... yes
checking for SSL_MODE_REQUIRED in mysql.h... yes
checking for SSL_MODE_VERIFY_CA in mysql.h... yes
checking for SSL_MODE_VERIFY_IDENTITY in mysql.h... yes
checking for MYSQL.net.vio in mysql.h... yes
checking for MYSQL.net.pvio in mysql.h... no
checking for MYSQL_ENABLE_CLEARTEXT_PLUGIN in mysql.h... yes
checking for SERVER_QUERY_NO_GOOD_INDEX_USED in mysql.h... yes
checking for SERVER_QUERY_NO_INDEX_USED in mysql.h... yes
checking for SERVER_QUERY_WAS_SLOW in mysql.h... yes
checking for MYSQL_OPTION_MULTI_STATEMENTS_ON in mysql.h... yes
checking for MYSQL_OPTION_MULTI_STATEMENTS_OFF in mysql.h... yes
checking for my_bool in mysql.h... no
-----
Don't know how to set rpath on your system, if MySQL libraries are not in path mysql2 may not load
-----
-----
Setting libpath to /opt/homebrew/Cellar/mysql/8.0.26/lib
-----
creating Makefile

current directory: /Users/olivierlacan/.rbenv/versions/3.0.1/lib/ruby/gems/3.0.0/bundler/gems/mysql2-7f4e844fccf6/ext/mysql2
make DESTDIR\= clean

current directory: /Users/olivierlacan/.rbenv/versions/3.0.1/lib/ruby/gems/3.0.0/bundler/gems/mysql2-7f4e844fccf6/ext/mysql2
make DESTDIR\=
compiling client.c
compiling infile.c
compiling mysql2_ext.c
compiling result.c
result.c:259:55: warning: format specifies type 'long' but the argument has type 'int' [-Wformat]
        rb_field_type = rb_sprintf("decimal(%ld,%d)", precision, field->decimals);
                                            ~~~       ^~~~~~~~~
                                            %d
result.c:258:35: warning: implicit conversion loses integer precision: 'unsigned long' to 'int' [-Wshorten-64-to-32]
        precision = field->length - (field->decimals > 0 ? 2 : 1);
                  ~ ~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2 warnings generated.
compiling statement.c
linking shared-object mysql2/mysql2.bundle
ld: warning: directory not found for option '-L/usr/local/opt/openssl/lib'
ld: library not found for -lzstd
clang-12: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [mysql2.bundle] Error 1

make failed, exit code 2

Gem files will remain installed in /Users/olivierlacan/.rbenv/versions/3.0.1/lib/ruby/gems/3.0.0/bundler/gems/mysql2-7f4e844fccf6 for inspection.
Results logged to /Users/olivierlacan/.rbenv/versions/3.0.1/lib/ruby/gems/3.0.0/bundler/gems/extensions/arm64-darwin-20/3.0.0/mysql2-7f4e844fccf6/gem_make.out

An error occurred while installing mysql2 (0.5.3), and Bundler cannot continue.
</pre>
</details>
sodabrew pushed a commit that referenced this pull request Sep 6, 2021
This is a follow-up to #1135 which added the OpenSSL flag assuming that
if the `RUBY_PLATFORM` is `darwin` (macOS):
- Homebrew is installed
- OpenSSL is installed via Homebrew

This PR:
- no longer assumes Homebrew is installed if we're on macOS
- no longer assumes OpenSSL is installed via Homebrew
- asks Homebrew for the openssl location (which will also work with the
newer [email protected] recipe)

Should prevent issues like these when running bundle install on the
rails codebase:

    Bundle Install error due to: ld: warning: directory not found for option '-L/usr/local/opt/openssl/lib'
Hamms added a commit to code-dot-org/code-dot-org that referenced this pull request Jul 5, 2022
To pick up several bugfixes in this version, including support for Ruby
2.7 and a fix for an OSX-specific issue.

See brianmario/mysql2#1135 and https://github.com/brianmario/mysql2/releases
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants