Python wheels not PEP 599 compliant; wheels should include libgomp.so.1

[AWSjswinney]

Description

PEP 599 and PEP 513 describe a spec for broadly compatible wheels. They include a list of dynamic libraries that wheels are allowed to dynamically link against without bundling the so in the wheel. The LightGBM wheel depends on libgomp.so.1 but does not include in the wheel package. To fix this, you can use auditwheel. I have not tested this, but it probably involves adding line like this, here.

auditwheel repair dist/lightgbm-$LGB_VER-py3-none-$PLATFORM.whl

Specs:
PEP 599 -- The manylinux2014 Platform Tag
PEP 513 -- A Platform Tag for Portable Linux Built Distributions

Reproducible example

This works on Linux on x86_64 and aarch64.

docker run -it --rm amazonlinux sh -c "yum install -y python3-pip && pip3 install lightgbm && python3 -c 'import lightgbm'"

...
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.7/site-packages/lightgbm/__init__.py", line 8, in <module>
    from .basic import Booster, Dataset, register_logger
  File "/usr/local/lib/python3.7/site-packages/lightgbm/basic.py", line 95, in <module>
    _LIB = _load_lib()
  File "/usr/local/lib/python3.7/site-packages/lightgbm/basic.py", line 86, in _load_lib
    lib = ctypes.cdll.LoadLibrary(lib_path[0])
  File "/usr/lib64/python3.7/ctypes/__init__.py", line 437, in LoadLibrary
    return self._dlltype(name)
  File "/usr/lib64/python3.7/ctypes/__init__.py", line 359, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: libgomp.so.1: cannot open shared object file: No such file or directory

Environment info

LightGBM version or commit hash: lightgbm-3.2.1

Command(s) you used to install LightGBM

pip3 install lightgbm

[jameslamb]

Thank you very much for this excellent report @AWSjswinney ! Are you interested in contributing this feature? If not, would it be alright for maintainers to @ you for a review in the future if/when this work is picked up?

[AWSjswinney]

Yes, I will definitely be happy to answer any questions from maintainers. Also, if I find the the time, I may test and submit a pull request myself.

[jameslamb]

Thanks very much!

[StrikerRUS]

If I remember correctly, we are not including OpenMP libraries in wheels intendedly to avoid the following error:

OMP: Error #15: Initializing libiomp5.dylib, but found libomp.dylib already initialized.
https://lightgbm.readthedocs.io/en/latest/FAQ.html#lightgbm-crashes-randomly-with-the-error-like-initializing-libiomp5-dylib-but-found-libomp-dylib-already-initialized

Related: dmlc/xgboost#1715 and apache/mxnet#20095 (comment).

Instead, we are asking to install OpenMP system-widely. But we do it only for macOS. I think we can do the same for Linux.
https://github.com/microsoft/LightGBM/tree/master/python-package#install-from-pypi

Speaking about manylinux*** tags, we are planning to migrate to PEP600 once this tag becomes more or less mature (#3421 (comment)).

[AWSjswinney]

I opened a pull request to update the documentation to mention this. Does this seem like an acceptable compromise?