install itself is failing

[ravi160822]

pip install lightgbm
Exception: Please install CMake and all required dependencies first

python 3.9
macOS

[jameslamb]

Thanks for using LightGBM. This is not enough information for us to help you.

Based only on what you've provided, I'm guessing that you tried running

pip install lightgbm

And got this error.

Please install CMake and all required dependencies first

Try the following.

brew install cmake
pip install lightgbm

If that doesn't work, we need more information.

• exactly what commands you ran to install lightgbm
• output of cmake --version
• output of conda info if using conda

[ravi160822]

poetry add cmake
->

The following packages are already present in the pyproject.toml and will be skipped:

  • cmake

poetry add lightgbm
->

Exception: Please install CMake and all required dependencies first
        The full version of error log was saved into /Users/user_name/LightGBM_compilation.log
        [end of output]
    
    note: This error originates from a subprocess, and is likely not a problem with pip.
    ERROR: Failed building wheel for lightgbm
  Failed to build lightgbm
  ERROR: Could not build wheels for lightgbm, which is required to install pyproject.toml-based projects

cmake version 3.24.2

Poetry
Version: 1.2.2
Python:  3.9.6

Platform:   darwin
OS:         posix

when I do,

poetry add cmake@latest

->

Package operations: 0 installs, 0 updates, 4 removals

  • Removing scikit-learn (1.1.3)
  • Removing scipy (1.9.3)
  • Removing threadpoolctl (3.1.0)
  • Removing wheel (0.38.1)

then I do,

poetry add lightgbm

->

Package operations: 5 installs, 0 updates, 0 removals

  • Installing scipy (1.9.3)
  • Installing threadpoolctl (3.1.0)
  • Installing scikit-learn (1.1.3)
  • Installing wheel (0.38.1)
  • Installing lightgbm (3.3.3): Failed

[jameslamb]

The full version of error log was saved into /Users/user_name/LightGBM_compilation.log

Did you check that file? It should contain more information about why the wheel can't be built.

[ravi160822]

Could NOT find OpenMP_C (missing: OpenMP_C_FLAGS OpenMP_C_LIB_NAMES)

then I did this,

ln -s /usr/local/opt/llvm/bin/clang /usr/local/bin/clang-omp
ln -s /usr/local/opt/llvm/bin/clang++ /usr/local/bin/clang-omp++

as described here, https://stackoverflow.com/questions/48825416/missing-openmp-c-flags-openmp-c-lib-names

now it is giving
CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage

Could not find compiler set in environment variable CC:
  
  /usr/local/opt/llvm/bin/clang.

CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage
CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage

then I did,

export CC=clang-omp CXX=clang-omp++

now it is giving,

Could not find compiler set in environment variable CC:

  clang-omp.

then I did,

arch -arm64 brew install libomp

export CXX=g++-12 CC=gcc-12

now it gives,

ld: symbol(s) not found for architecture arm64
collect2: error: ld returned 1 exit status

ld: warning: directory not found for option '-L/usr/local/opt/llvm/lib'
Undefined symbols for architecture arm64:
  "__ZTHN8LightGBM7Network13num_machines_E", referenced from:
      __ZN8LightGBM27ObtainAutomaticInitialScoreEPKNS_17ObjectiveFunctionEi in gbdt.cpp.o
      __ZN8LightGBM4GBDT16BoostFromAverageEib in gbdt.cpp.o
      __ZN8LightGBM7Network11GlobalArrayIiEESt6vectorIT_SaIS3_EES3_ in dataset_loader.cpp.o
ld: symbol(s) not found for architecture arm64

then I did,

brew reinstall libomp

export LDFLAGS="-L/opt/homebrew/opt/libomp/lib"
export CPPFLAGS="-I/opt/homebrew/opt/libomp/include"

brew reinstall llvm

 LDFLAGS="-L/opt/homebrew/opt/llvm/lib/c++ -Wl,-rpath,/opt/homebrew/opt/llvm/lib/c++"
echo 'export PATH="/opt/homebrew/opt/llvm/bin:$PATH"' >> ~/.profile
export LDFLAGS="-L/opt/homebrew/opt/llvm/lib"
export CPPFLAGS="-I/opt/homebrew/opt/llvm/include"

but am getting the same symbol not found error

[ravi160822]

could the cmake error be displayed while I am installing lightgbm, the bottleneck of having to go to another directory is inconvenient.
plus the log file is showing me all the old errors, I would only want to see the installation errors of the current pip install.

plus could you do a gcc check, I had to find the gcc version manually, could this directly be suggested while installation, such as,

your gcc version is 12
please run this

export CXX=g++-12 CC=gcc-12

or just give a yes/no option,

your gcc version is 12
would you like to run
export CXX=g++-12 CC=gcc-12
enter YES or NO

could do it for more steps also

cmake is not installed
would you like to run
pip install cmake
enter YES or NO

libomp is not installed
would you like to run
pip install libomp
enter YES or NO

[jameslamb]

Based on the information you've provided, it looks like after following the other error messages you've resolved missing OpenMP and are now facing exactly the same issue as #5269 (comment).

Because no free continuous integration (CI) services support M1 Mac (ARM) machines and because none of this project's active maintainers have free access to an M1 Mac machine, it's very difficult for LightGBM to test support for compiling on that architecture. You can try the other possibilities mentioned in #5269, such as installing lightgbm from conda-forge as suggested by #5269 (comment).

We understand that this is frustrating for M1 Mac users but it's unlikely to change unless:

• a free CI service allows frequent testing on M1 Mac machines
• one or more contributors with access to M1s submit patches that add support for compiling LightGBM on the arm64 Macs

---

could the cmake error be displayed while I am installing lightgbm. the bottleneck of having to go to another directory is inconvenient

It is an intentional choice that all stderr / stdout from compilation isn't pulled back into the output stream of pip install lightgbm.

If you feel strongly that lightgbm should support that, please open a separate feature request issue explaining why you believe that behavior would be preferable to the current state where logs are dumped to a file.

---

plus could you do a gcc check, I had to find the gcc version manually, could this directly be suggested while installation
...
could do it for more steps also

lightgbm's Python package is very unlikely to support an interactive installation mode like that.

1. many installations of the Python package are non-interactive (e.g. in container builds or automated processes running on remote servers)
   • an interactive installation guide like that wouldn't be desirable in those settings
2. this project supports a wide range of compilers and operating systems, making support for such an interface likely very expensive in terms of development time and maintainers' attention
3. lightgbm would prefer not to be opinionated about how users install build toolchain tools, except in situations where absolutely necessary
   • for example, you may want to use pip to install cmake but others will prefer conda, brew (macOS), asdf, directly downloading cmake official releases, and other options

If you feel very strongly that such an interactive installation mode would be worth the effort, please open a separate feature request issue explaining why you think it would be valuable and, in as much detail as possible, how you'd like it to work.

[ravi160822]

currently this works,

pip install lightgbm --install-option=--nomp

I will create the required issues

[jameslamb]

Ok glad you got it working. I'm going to close this discussion.

For you and anyone else finding it from search engines.... please follow and contribute to

• unable to install lightgbm with pip in macos 12.3.1 (chip M1) #5269
• LightGBM is incompatible with libomp 12 and 13 on macOS #4229

[github-actions]

This issue has been automatically locked since there has not been any recent activity since it was closed.
To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues
including a reference to this.