nokogiri install fails on mac because homebrew libxml is too new

[dwt]

Hi there,

following the instructions on https://nokogiri.org/tutorials/installing_nokogiri.html#homebrew_09 doesn't seem to work anymore.

Following it I get this output:

gem install nokogiri -- --use-system-libraries --with-xml2-include=`brew --prefix`/opt/libxml2/include/libxml2  --with-xml2-lib=`brew --prefix`/opt/libxml2/lib  --with-xslt-dir=`brew --prefix`/opt/libxslt --with-iconv-include=`brew --prefix`/opt/libiconv/include/
Building native extensions with: '--use-system-libraries --with-xml2-include=/Users/dwt/Library/Homebrew/opt/libxml2/include/libxml2 --with-xml2-lib=/Users/dwt/Library/Homebrew/opt/libxml2/lib --with-xslt-dir=/Users/dwt/Library/Homebrew/opt/libxslt --with-iconv-include=/Users/dwt/Library/Homebrew/opt/libiconv/include/'
This could take a while...
Building nokogiri using system libraries.
ERROR:  Error installing nokogiri:
    ERROR: Failed to build gem native extension.

    /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/bin/ruby extconf.rb --use-system-libraries --with-xml2-include=/Users/dwt/Library/Homebrew/opt/libxml2/include/libxml2 --with-xml2-lib=/Users/dwt/Library/Homebrew/opt/libxml2/lib --with-xslt-dir=/Users/dwt/Library/Homebrew/opt/libxslt --with-iconv-include=/Users/dwt/Library/Homebrew/opt/libiconv/include/
Building nokogiri using system libraries.
libxml2 version 2.9.0 and later is not yet supported, but proceeding anyway.
checking for xmlParseDoc() in libxml/parser.h... no
checking for xmlParseDoc() in -lxml2... no
checking for xmlParseDoc() in -llibxml2... no
-----
libxml2 is missing.  please visit http://nokogiri.org/tutorials/installing_nokogiri.html for help with installing dependencies.
-----
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers.  Check the mkmf.log file for more details.  You may
need configuration options.

Provided configuration options:
    --with-opt-dir
    --without-opt-dir
    --with-opt-include
    --without-opt-include=${opt-dir}/include
    --with-opt-lib
    --without-opt-lib=${opt-dir}/lib
    --with-make-prog
    --without-make-prog
    --srcdir=.
    --curdir
    --ruby=/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/bin/ruby
    --help
    --clean
    --use-system-libraries
    --with-zlib-dir
    --without-zlib-dir
    --with-zlib-include
    --without-zlib-include=${zlib-dir}/include
    --with-zlib-lib
    --without-zlib-lib=${zlib-dir}/lib
    --with-xml2-dir
    --without-xml2-dir
    --with-xml2-include=${xml2-dir}/include
    --with-xml2-lib=${xml2-dir}/lib
    --with-xslt-dir
    --with-xslt-include
    --without-xslt-include=${xslt-dir}/include
    --with-xslt-lib
    --without-xslt-lib=${xslt-dir}/lib
    --with-exslt-dir
    --without-exslt-dir
    --with-exslt-include
    --without-exslt-include=${exslt-dir}/include
    --with-exslt-lib
    --without-exslt-lib=${exslt-dir}/lib
    --with-libexslt-config
    --without-libexslt-config
    --with-pkg-config
    --without-pkg-config
    --with-xml2lib
    --without-xml2lib
    --with-libxml2lib
    --without-libxml2lib

extconf failed, exit code 1

As far as I can tell the homebrew version of libxml is deemed too new, while the system version is deemed too old.

Perhaps a nokogiri update is advised?

Installed versions of libraries:

% brew info libxml2 --json=v1 |jq '.[0].installed[0].version'
"2.9.1"
% brew info libxslt --json=v1 |jq '.[0].installed[0].version'
"1.1.28"
% brew info libiconv --json=v1 |jq '.[0].installed[0].version'
"1.14"
uname -v
Darwin Kernel Version 13.4.0: Sun Aug 17 19:50:11 PDT 2014; root:xnu-2422.115.4~1/RELEASE_X86_64

What is the recommended way to install nokogiri in this environment?

[dwt]

This workaround allowed me to get the current nokogiri at least installed.

gem install nokogiri -- --use-system-libraries --with-xml2-include=/usr/include/libxml2 --with-xml2-lib=/usr/lib/

Not sure why this directory wasn't consulted beforehand? Seems to work for now.

However the instructions at https://nokogiri.org/tutorials/installing_nokogiri.html#homebrew_09 still are faulty and don't give you / me a working nokogiri.

[lamont-granquist]

try this:

brew unlink libxml2
brew unlink libxslt
brew unlink libiconv
sudo xcode-select --install
gem install nokogiri

you shouldn't have libiconv and friends linked into /usr/local/include and brew explicitly warns you about that:

% brew link libiconv
Warning: libiconv is keg-only and must be linked with --force
Note that doing so can interfere with building software.

you also shouldn't be doing --use-system-libraries, nokogiri wants very specific libraries that its is packaged with.

and if you have Mac 10.9 and have a correctly working installation of xcode you should be able to simply 'gem install nokogiri' now.

[dwt]

I have to say I'm not quite sure what you're suggesting, here's the output from the brew unlinks:

dwt@cirdan ~ % brew unlink libxml2
Unlinking /Users/dwt/Library/Homebrew/Cellar/libxml2/2.9.1... 0 symlinks removed
dwt@cirdan ~ % brew unlink libxslt
Unlinking /Users/dwt/Library/Homebrew/Cellar/libxslt/1.1.28... 0 symlinks removed
dwt@cirdan ~ % brew unlink libiconv
Unlinking /Users/dwt/Library/Homebrew/Cellar/libiconv/1.14... 0 symlinks removed

None of those libraries where linked. I did try to use them with the explicit build settings as described at https://nokogiri.org/tutorials/installing_nokogiri.html#homebrew_09 and that didn't work with the error I reported initially.

So my bug report still stands, the instructions provided at https://nokogiri.org/tutorials/installing_nokogiri.html#homebrew_09 for homebrew do not work and should be fixed as the installed version of libxml2 seems to be too new.

[dwt]

Looking deeper into this issue I just realized that I already reported the root cause of this problem earlier #1119 and forgot about it...

[lamont-granquist]

So your title suggests "because homebrew libxml is too new"

Not linking against homebrew's libxml2/libxslt/libiconv will solve the problem. Nokogiri comes packaged with libxml2 and libxslt, and the libiconv in /usr/{lib,include} on 10.9 works fine. So don't use --use-system-libraries, don't use any of the --with-XXX options and don't set NOKOGIRI_USE_SYSTEM_LIBRARIES=1

That does work. If it does not work then either your Xcode install on your Mac is broken (mine was broken and /usr/include/iconv.h was entirely missing for some unknown reason related possibly to O/S upgrades, reinstall and restores from time machine over the years), or else nokogiri is finding your homebrew libraries linked up (which is sounds like it isn't) or is finding other libxml2/libxslt/libconv libraries from MacPorts or other installs that you have on your system that aren't the base O/S.

On a blank Mac 10.9 box with xcode freshly installed, even with homebrew installed properly without the libs being linked up then 'gem install nokogiri' with no options should work.

[dwt]

Thanks for taking the time to reply.

I'd like to tackle your suggestions one by one.

First for the suggestion not to use --system-libraries: The problem here is that the version of libxml2 that is packaged with nokogiri does not find some of the lzma library, as documented in #1119 - so sadly that is not an option. Looking at that specific error it seems that it is searching for those symbols in the packaged libxml2 instead of in some system library, which is why I suggested the problem might be with nokogiris packaged library instead of the system - but of course that might be because my C is a bit rusty. I would love to get a reply from you there though (or here, but about that topic).

Second: /usr/include/iconv.h is perfectly there and in fact nokogiri builds fine if invoked with these flags:

$ cat ~/.bundle/config
---
BUNDLE_BUILD__NOKOGIRI: --use-system-libraries --with-xml2-include=/usr/include/libxml2 --with-xml2-lib=/usr/lib/

Which to my understanding uses /usr/include/iconv.h to build and therefore I conclude that this is not the source of these build problems.

Third: I would like you to take notice that the instructions presented at https://nokogiri.org/tutorials/installing_nokogiri.html#homebrew_09 do not work anymore and need adjusting.

[lamont-granquist]

I've got the same Mac O/S version that you do, I've got homebrew and an identical (on paper) setup, and 'gem install nokogiri' works fine. Whatever is going on in #1119 there is another fix that does not involve using any flags to nokogiri.

[EvanTedesco]

I am running into the same issue and I have spend literally an entire day trying to resolve it. I hate configuration hell!

[dwt]

@EvanTedesco: I'd like to add that it turned out in the end that some homebrew libraries that I had installed and that got linked for some reason I can't fathom but shouldn't turned out to cause this for me.

So you should probably try deactivating / removing homebrew and see if that solves the problem for you too - then find out which linked homebrew libraries are the problem for you (aren't that many candidates).

[flavorjones]

I believe we've captured a working installation procedure in the updated tutorial here:

http://www.nokogiri.org/tutorials/installing_nokogiri.html

If I'm mistaken, please let me know what I'm missing and I'll reopen.

[sr-murthy]

@dwt you're a genius, I had the exact same issue as your original issue, and will be closing the issue I recently reported on jekyll-help

[vakila]

Yes thanks @dwt - I was having the same issue on Mac OS 10.8.5 and tried a bunch of different things, but the only thing that finally worked was your suggestion:

gem install nokogiri -- --use-system-libraries --with-xml2-include=/usr/include/libxml2 --with-xml2-lib=/usr/lib/

So thank you!

[Illbay]

I'm not a coder, so I have to rely on recipes such as this.

The command-line formula that @dwt specifies worked for me using OS X 10.11.1 (El Capitan).

[megazoic]

@dwt suggestion also helped me running Mac OS 10.9.5, RVM 1.26.11, Homebrew 0.9.5. Thank you!!

[dwt]

@maintainers: Hey guys, since this problem is really old and recurring and the workaround I posted is really not the right solution. What about building in some better error checking into the formula so it can better deal with the underlying cause of this issue?

I mean, for every user who comes here, there are probably a thousand who have this problem, so it would be really worth it.

[flavorjones]

@dwt As I suggested before, let's start by documenting the problem and the solution in the installation tutorial. If you feel those instructions are incomplete or misleading, I'd love a PR that fixes it.

In the meantime, #1118 should be addressed by #1218 which is on master and will be in 1.6.7.

[rafaeleyng]

gem install nokogiri -- --use-system-libraries --with-xml2-include=/usr/include/libxml2 --with-xml2-lib=/usr/lib/ didn't work for me.

Running:
El Capitan
ruby 2.1.2p95 (2014-05-08 revision 45877) [x86_64-darwin15.0]
rbenv 0.4.0

[rafaeleyng]

The error is

Building native extensions with: '--use-system-libraries --with-xml2-include=/usr/include/libxml2 --with-xml2-lib=/usr/lib/'
This could take a while...
ERROR:  Error installing nokogiri:
    ERROR: Failed to build gem native extension.

    current directory: /Users/rafael/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/nokogiri-1.6.7.1/ext/nokogiri
/Users/rafael/.rbenv/versions/2.1.2/bin/ruby -r ./siteconf20151222-12102-4rmicl.rb extconf.rb --use-system-libraries --with-xml2-include=/usr/include/libxml2 --with-xml2-lib=/usr/lib/
checking if the C compiler accepts ... yes
checking if the C compiler accepts -Wno-error=unused-command-line-argument-hard-error-in-future... no
Building nokogiri using system libraries.
libxml2 version 2.6.21 or later is required!
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers.  Check the mkmf.log file for more details.  You may
need configuration options.

Provided configuration options:
    --with-opt-dir
    --without-opt-dir
    --with-opt-include
    --without-opt-include=${opt-dir}/include
    --with-opt-lib
    --without-opt-lib=${opt-dir}/lib
    --with-make-prog
    --without-make-prog
    --srcdir=.
    --curdir
    --ruby=/Users/rafael/.rbenv/versions/2.1.2/bin/ruby
    --help
    --clean
    --use-system-libraries
    --with-zlib-dir
    --without-zlib-dir
    --with-zlib-include
    --without-zlib-include=${zlib-dir}/include
    --with-zlib-lib
    --without-zlib-lib=${zlib-dir}/lib
    --with-xml2-dir
    --without-xml2-dir
    --with-xml2-include=${xml2-dir}/include
    --with-xml2-lib=${xml2-dir}/lib
    --with-xslt-dir
    --without-xslt-dir
    --with-xslt-include
    --without-xslt-include=${xslt-dir}/include
    --with-xslt-lib
    --without-xslt-lib=${xslt-dir}/lib
    --with-libxslt-config
    --without-libxslt-config
    --with-pkg-config
    --without-pkg-config
    --with-exslt-dir
    --without-exslt-dir
    --with-exslt-include
    --without-exslt-include=${exslt-dir}/include
    --with-exslt-lib
    --without-exslt-lib=${exslt-dir}/lib
    --with-libexslt-config
    --without-libexslt-config

To see why this extension failed to compile, please check the mkmf.log which can be found here:

  /Users/rafael/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/extensions/x86_64-darwin-15/2.1.0-static/nokogiri-1.6.7.1/mkmf.log

extconf failed, exit code 1

Gem files will remain installed in /Users/rafael/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/nokogiri-1.6.7.1 for inspection.
Results logged to /Users/rafael/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/extensions/x86_64-darwin-15/2.1.0-static/nokogiri-1.6.7.1/gem_make.out

[brauliobo]

got the same problem here with archlinux. doing gem install nokogiri -- --use-system-libraries --with-xml2-include=/usr/include/libxml2 --with-xml2-lib=/usr/lib/ made it work

[itsNikolay]

(El Captain) I resolved the issue by:
bundle update nokogiri

[flavorjones]

Hi all,

I'll again encourage everyone to please please please take a look at the installation tutorial and give us feedback if it doesn't contain correct information, preferably by opening an issue either here or at https://github.com/sparklemotion/nokogiri.org-tutorials

I'm going to lock this issue down to disincentivize people from continuing to post here.

It's in everyone's interests to invest in improving the documentation; so please help us help future developers.