Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Can't be built on macOS #16

Closed
HiromuHota opened this issue Apr 1, 2020 · 7 comments
Closed

Can't be built on macOS #16

HiromuHota opened this issue Apr 1, 2020 · 7 comments

Comments

@HiromuHota
Copy link
Contributor

Fugashi cannot be built hence cannot be installed on macOS.

$ pip install fugashi
Collecting fugashi
  Using cached https://files.pythonhosted.org/packages/2e/88/156c51c78ee4ccfd54000e720f0c9814d073993b4e1f5d400d01416ddb6d/fugashi-0.1.10.tar.gz
Requirement already satisfied: Cython in /Users/hiromu/miniconda3/envs/fonduer-dev/lib/python3.7/site-packages (from fugashi) (0.29.16)
Building wheels for collected packages: fugashi
  Building wheel for fugashi (setup.py) ... error
  ERROR: Complete output from command /Users/hiromu/miniconda3/envs/fonduer-dev/bin/python -u -c 'import setuptools, tokenize;__file__='"'"'/private/var/folders/6j/mctnhv6n2zx4zf8c657hbr2w0000gn/T/pip-install-adei_m45/fugashi/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/6j/mctnhv6n2zx4zf8c657hbr2w0000gn/T/pip-wheel-7supwlz0 --python-tag cp37:
  ERROR: running bdist_wheel
  running build
  running build_ext
  cythoning fugashi/fugashi.pyx to fugashi/fugashi.c
  /Users/hiromu/miniconda3/envs/fonduer-dev/lib/python3.7/site-packages/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /private/var/folders/6j/mctnhv6n2zx4zf8c657hbr2w0000gn/T/pip-install-adei_m45/fugashi/fugashi/fugashi.pyx
    tree = Parsing.p_module(s, pxd, full_module_name)
  building 'fugashi' extension
  creating build/temp.macosx-10.7-x86_64-3.7
  creating build/temp.macosx-10.7-x86_64-3.7/fugashi
  gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/hiromu/miniconda3/envs/fonduer-dev/include -arch x86_64 -I/Users/hiromu/miniconda3/envs/fonduer-dev/include -arch x86_64 -I/usr/local/Cellar/mecab/0.996/include -I/Users/hiromu/miniconda3/envs/fonduer-dev/include/python3.7m -c fugashi/fugashi.c -o build/temp.macosx-10.7-x86_64-3.7/fugashi/fugashi.o
  In file included from fugashi/fugashi.c:598:
  /usr/local/Cellar/mecab/0.996/include/mecab.h:380:47: warning: this function declaration is not a prototype [-Wstrict-prototypes]
    MECAB_DLL_EXTERN const char*   mecab_version();
                                                ^
                                                 void
  /usr/local/Cellar/mecab/0.996/include/mecab.h:520:54: warning: this function declaration is not a prototype [-Wstrict-prototypes]
    MECAB_DLL_EXTERN mecab_lattice_t *mecab_lattice_new();
                                                       ^
                                                        void
  fugashi/fugashi.c:4828:13: warning: assigning to 'char *' from 'const char *' discards qualifiers [-Wincompatible-pointer-types-discards-qualifiers]
    __pyx_t_5 = mecab_nbest_sparse_tostr(__pyx_v_self->c_tagger, __pyx_t_3, __pyx_t_4);
              ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  fugashi/fugashi.c:9121:26: warning: code will never be executed [-Wunreachable-code]
                  module = PyImport_ImportModuleLevelObject(
                           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  4 warnings generated.
  creating build/lib.macosx-10.7-x86_64-3.7
  gcc -bundle -undefined dynamic_lookup -L/Users/hiromu/miniconda3/envs/fonduer-dev/lib -arch x86_64 -L/Users/hiromu/miniconda3/envs/fonduer-dev/lib -arch x86_64 -arch x86_64 build/temp.macosx-10.7-x86_64-3.7/fugashi/fugashi.o -L/usr/local/Cellar/mecab/0.996/lib -lmecab -lstdc++ -o build/lib.macosx-10.7-x86_64-3.7/fugashi.cpython-37m-darwin.so
  clang: warning: libstdc++ is deprecated; move to libc++ with a minimum deployment target of OS X 10.9 [-Wdeprecated]
  ld: library not found for -lstdc++
  clang: error: linker command failed with exit code 1 (use -v to see invocation)
  error: command 'gcc' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for fugashi
  Running setup.py clean for fugashi
Failed to build fugashi
Installing collected packages: fugashi
  Running setup.py install for fugashi ... error
    ERROR: Complete output from command /Users/hiromu/miniconda3/envs/fonduer-dev/bin/python -u -c 'import setuptools, tokenize;__file__='"'"'/private/var/folders/6j/mctnhv6n2zx4zf8c657hbr2w0000gn/T/pip-install-adei_m45/fugashi/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/6j/mctnhv6n2zx4zf8c657hbr2w0000gn/T/pip-record-y_d6t5au/install-record.txt --single-version-externally-managed --compile:
    ERROR: running install
    running build
    running build_ext
    skipping 'fugashi/fugashi.c' Cython extension (up-to-date)
    building 'fugashi' extension
    creating build/temp.macosx-10.7-x86_64-3.7
    creating build/temp.macosx-10.7-x86_64-3.7/fugashi
    gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/hiromu/miniconda3/envs/fonduer-dev/include -arch x86_64 -I/Users/hiromu/miniconda3/envs/fonduer-dev/include -arch x86_64 -I/usr/local/Cellar/mecab/0.996/include -I/Users/hiromu/miniconda3/envs/fonduer-dev/include/python3.7m -c fugashi/fugashi.c -o build/temp.macosx-10.7-x86_64-3.7/fugashi/fugashi.o
    In file included from fugashi/fugashi.c:598:
    /usr/local/Cellar/mecab/0.996/include/mecab.h:380:47: warning: this function declaration is not a prototype [-Wstrict-prototypes]
      MECAB_DLL_EXTERN const char*   mecab_version();
                                                  ^
                                                   void
    /usr/local/Cellar/mecab/0.996/include/mecab.h:520:54: warning: this function declaration is not a prototype [-Wstrict-prototypes]
      MECAB_DLL_EXTERN mecab_lattice_t *mecab_lattice_new();
                                                         ^
                                                          void
    fugashi/fugashi.c:4828:13: warning: assigning to 'char *' from 'const char *' discards qualifiers [-Wincompatible-pointer-types-discards-qualifiers]
      __pyx_t_5 = mecab_nbest_sparse_tostr(__pyx_v_self->c_tagger, __pyx_t_3, __pyx_t_4);
                ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    fugashi/fugashi.c:9121:26: warning: code will never be executed [-Wunreachable-code]
                    module = PyImport_ImportModuleLevelObject(
                             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    4 warnings generated.
    creating build/lib.macosx-10.7-x86_64-3.7
    gcc -bundle -undefined dynamic_lookup -L/Users/hiromu/miniconda3/envs/fonduer-dev/lib -arch x86_64 -L/Users/hiromu/miniconda3/envs/fonduer-dev/lib -arch x86_64 -arch x86_64 build/temp.macosx-10.7-x86_64-3.7/fugashi/fugashi.o -L/usr/local/Cellar/mecab/0.996/lib -lmecab -lstdc++ -o build/lib.macosx-10.7-x86_64-3.7/fugashi.cpython-37m-darwin.so
    clang: warning: libstdc++ is deprecated; move to libc++ with a minimum deployment target of OS X 10.9 [-Wdeprecated]
    ld: library not found for -lstdc++
    clang: error: linker command failed with exit code 1 (use -v to see invocation)
    error: command 'gcc' failed with exit status 1
    ----------------------------------------
ERROR: Command "/Users/hiromu/miniconda3/envs/fonduer-dev/bin/python -u -c 'import setuptools, tokenize;__file__='"'"'/private/var/folders/6j/mctnhv6n2zx4zf8c657hbr2w0000gn/T/pip-install-adei_m45/fugashi/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/6j/mctnhv6n2zx4zf8c657hbr2w0000gn/T/pip-record-y_d6t5au/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/6j/mctnhv6n2zx4zf8c657hbr2w0000gn/T/pip-install-adei_m45/fugashi/

I installed Mecab using Homebrew

$ brew info mecab
mecab: stable 0.996 (bottled)
Yet another part-of-speech and morphological analyzer
https://taku910.github.io/mecab/
Conflicts with:
  mecab-ko (because both install mecab binaries)
/usr/local/Cellar/mecab/0.996 (20 files, 4.2MB) *
  Poured from bottle on 2019-03-25 at 14:19:36
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/mecab.rb
==> Analytics
install: 7,671 (30 days), 24,244 (90 days), 50,937 (365 days)
install-on-request: 653 (30 days), 2,051 (90 days), 10,150 (365 days)
build-error: 0 (30 days)
polm added a commit that referenced this issue Apr 2, 2020
@polm
Attempted fix of OSX build issue #16
d6f4696 
It looks like this is related to libstdc++ being handled in an unusual
way on OSX. The fix comes from this post:

https://stackoverflow.com/questions/52460913/compiling-cython-with-xcode-10

However, I'm not sure this will work since it looks like the OSX version
in the issue is 10.7. Worth a shot though.
@polm
Copy link
Owner

polm commented Apr 2, 2020

Thanks for the bug report. I had a bit of trouble understanding what's going on here because the actual error doesn't seem to be in the output above; the only detail is this:

ld: library not found for -lstdc++

It looks like libstdc++ is a little weird on OSX, and at some point was deprecated in favor of libc++? I have attempted a fix that sets compiler flags to use the other library based on this StackOverflow post. I'm not sure it'll work since it seems to refer to a newer version of OSX than you're using - it looks like you have 10.7, but this is for 10.9+. However, it seems like libstdc++ should be normally usable before 10.9 so I'm still not entirely sure what's going on...

I have pushed an attempted fix to the osx-fix branch. Could you check that out, try a python setup.py install, and tell me what the output is? If it doesn't work you can try removing the -mmacosx-version-min=10.9 bits from setup.py.

@HiromuHota
Copy link
Contributor Author

Thank you for your quick reply.

it looks like you have 10.7, but this is for 10.9+

For the record, my macOS version is 10.15.4.

I tried the osx-fix branch and failed.

$ python setup.py install
running install
running bdist_egg
running egg_info
writing fugashi.egg-info/PKG-INFO
writing dependency_links to fugashi.egg-info/dependency_links.txt
writing requirements to fugashi.egg-info/requires.txt
writing top-level names to fugashi.egg-info/top_level.txt
reading manifest file 'fugashi.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'fugahsi/mecab.pxd'
writing manifest file 'fugashi.egg-info/SOURCES.txt'
installing library code to build/bdist.macosx-10.7-x86_64/egg
running install_lib
running build_ext
skipping 'fugashi/fugashi.c' Cython extension (up-to-date)
building 'fugashi' extension
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/hiromu/miniconda3/envs/fonduer-dev/include -arch x86_64 -I/Users/hiromu/miniconda3/envs/fonduer-dev/include -arch x86_64 -I/usr/local/Cellar/mecab/0.996/include -I/Users/hiromu/miniconda3/envs/fonduer-dev/include/python3.7m -c fugashi/fugashi.c -o build/temp.macosx-10.7-x86_64-3.7/fugashi/fugashi.o -std=c++11 -mmacosx-version-min=10.9
error: invalid argument '-std=c++11' not allowed with 'C'
error: command 'gcc' failed with exit status 1

The error message says '-std=c++11' not allowed with 'C', so I modified the setup.py as in #17 and build passed and installed without an error.

$ python setup.py install
running install
running bdist_egg
running egg_info
writing fugashi.egg-info/PKG-INFO
writing dependency_links to fugashi.egg-info/dependency_links.txt
writing requirements to fugashi.egg-info/requires.txt
writing top-level names to fugashi.egg-info/top_level.txt
reading manifest file 'fugashi.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'fugahsi/mecab.pxd'
writing manifest file 'fugashi.egg-info/SOURCES.txt'
installing library code to build/bdist.macosx-10.7-x86_64/egg
running install_lib
running build_ext
skipping 'fugashi/fugashi.c' Cython extension (up-to-date)
creating build/bdist.macosx-10.7-x86_64/egg
copying build/lib.macosx-10.7-x86_64-3.7/fugashi.cpython-37m-darwin.so -> build/bdist.macosx-10.7-x86_64/egg
creating stub loader for fugashi.cpython-37m-darwin.so
byte-compiling build/bdist.macosx-10.7-x86_64/egg/fugashi.py to fugashi.cpython-37.pyc
creating build/bdist.macosx-10.7-x86_64/egg/EGG-INFO
copying fugashi.egg-info/PKG-INFO -> build/bdist.macosx-10.7-x86_64/egg/EGG-INFO
copying fugashi.egg-info/SOURCES.txt -> build/bdist.macosx-10.7-x86_64/egg/EGG-INFO
copying fugashi.egg-info/dependency_links.txt -> build/bdist.macosx-10.7-x86_64/egg/EGG-INFO
copying fugashi.egg-info/requires.txt -> build/bdist.macosx-10.7-x86_64/egg/EGG-INFO
copying fugashi.egg-info/top_level.txt -> build/bdist.macosx-10.7-x86_64/egg/EGG-INFO
writing build/bdist.macosx-10.7-x86_64/egg/EGG-INFO/native_libs.txt
zip_safe flag not set; analyzing archive contents...
__pycache__.fugashi.cpython-37: module references __file__
creating 'dist/fugashi-0.1.10-py3.7-macosx-10.7-x86_64.egg' and adding 'build/bdist.macosx-10.7-x86_64/egg' to it
removing 'build/bdist.macosx-10.7-x86_64/egg' (and everything under it)
Processing fugashi-0.1.10-py3.7-macosx-10.7-x86_64.egg
removing '/Users/hiromu/miniconda3/envs/fonduer-dev/lib/python3.7/site-packages/fugashi-0.1.10-py3.7-macosx-10.7-x86_64.egg' (and everything under it)
creating /Users/hiromu/miniconda3/envs/fonduer-dev/lib/python3.7/site-packages/fugashi-0.1.10-py3.7-macosx-10.7-x86_64.egg
Extracting fugashi-0.1.10-py3.7-macosx-10.7-x86_64.egg to /Users/hiromu/miniconda3/envs/fonduer-dev/lib/python3.7/site-packages
fugashi 0.1.10 is already the active version in easy-install.pth

Installed /Users/hiromu/miniconda3/envs/fonduer-dev/lib/python3.7/site-packages/fugashi-0.1.10-py3.7-macosx-10.7-x86_64.egg
Processing dependencies for fugashi==0.1.10
Searching for Cython==0.29.16
Best match: Cython 0.29.16
Adding Cython 0.29.16 to easy-install.pth file
Installing cygdb script to /Users/hiromu/miniconda3/envs/fonduer-dev/bin
Installing cython script to /Users/hiromu/miniconda3/envs/fonduer-dev/bin
Installing cythonize script to /Users/hiromu/miniconda3/envs/fonduer-dev/bin

Using /Users/hiromu/miniconda3/envs/fonduer-dev/lib/python3.7/site-packages
Finished processing dependencies for fugashi==0.1.10

@HiromuHota
Copy link
Contributor Author

The test also passed.

$ pytest fugashi/tests/test_basic.py 
============================================================== test session starts ==============================================================
platform darwin -- Python 3.7.3, pytest-5.2.1, py-1.8.0, pluggy-0.13.0
rootdir: /Users/hiromu/workspace/fugashi
plugins: monkeytype-1.0.1
collected 10 items                                                                                                                              

fugashi/tests/test_basic.py .........s                                                                                                    [100%]

========================================================= 9 passed, 1 skipped in 0.04s ==========================================================

@polm
Copy link
Owner

polm commented Apr 2, 2020

The error message says '-std=c++11' not allowed with 'C', so I modified the setup.py as in #17 and build passed and installed without an error.

Great, thank you! I'll merge that, clean it up a bit (the code should probably be in fugashi_util.py), and do a release.

For the record, my macOS version is 10.15.4.

Huh, OK. I assumed it was 10.7 based on the following line:

creating build/temp.macosx-10.7-x86_64-3.7

But I guess that's a version number for something else.

@polm
Copy link
Owner

polm commented Apr 2, 2020

Released a new version where this should be fixed, let me know if you continue to have issues.

@polm polm closed this as completed Apr 2, 2020
@HiromuHota
Copy link
Contributor Author

Thank you. Now I'm able to build it without an error.

creating build/temp.macosx-10.7-x86_64-3.7

This was actually puzzling me too, but now I have a better view of what was happening.
According to pandas-dev/pandas#24274,

current 64/32b universal builds of python, which target macOS 10.7

I was in a conda environment when I was building fugashi==0.1.10, which may have targeted macOS 10.7.
As the above PR did, if macOS 10.9 is targeted where possible, this error does not happen.

@polm
Copy link
Owner

polm commented Apr 3, 2020

Ah, that makes sense about Anaconda - thanks for the explanation!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@polm @HiromuHota