Run cibuildwheel failed after integrating conan our project.

[gurgenyegoryan]

Description

Hi everyone,

We have a C++ project and we are creating a python package. For quick builds, we integrated Conan, and after that the cibuildwheel do not pass, but when I try with old method, it pass. Can you help me figure out what the problem is.

Old method in CMakeLists.txt without conan
target_link_libraries(${wrap_lib} PRIVATE pthread fmt::fmt arrow::dataset arrow_python arrow::flight arrow::bundled arrow::arrow ${client_lib} ${client_dependencies})

New method in CMakeLists.txt with conan (targets are taken from conanbuildinfo.cmake file)
target_link_libraries(${wrap_lib} PRIVATE pthread fmt arrow_dataset arrow_python arrow_flight arrow_bundled arrow ${client_lib} ${client_dependencies})

***Error
ImportError: /project/build/lib.linux-x86_64-cpython-39/package/package.cpython-39-x86_64-linux-gnu.so: undefined symbol: __libc_single_threaded

Build log

No response

CI config

No response

[henryiii]

Conan distributes binaries, right? All binaries must be built as manylinux binaries to work in manylinux. Downloading pre-built binaries usually doesn’t work on linux or macOS.

[gurgenyegoryan]

I am creating a conan package with my dependencies, I created the package on ubuntu 22.04 with gcc11 and ldd 2.32 and also created on ubuntu 18.04 with gcc 10 and ldd 2.27. But in both cases it gives an same error. The build passes without problems, but during import it gives this error. And in the regular build with cmake and make everything goes well, but it seems that cibuildwheel changes something. I use manylinux_2_28

[henryiii]

You have to build in manylinux. Manylinux requires only small subset of system libraries and an old GLIBC. The manylinux image uses RHEL's developer kit to provide a newer version of GCC that only links with the older GLIBC. The manylinux docker image is at quay.io/pypa/manylinux2014_x86_64:latest. manylinux2014 is GLIBC 2.17 based, which is much older than ubuntu 18.04 (2.27). If you used manylinux_2_28 (which is GLIBC 2.28) in cibuildwheel, then your 18.04 one might work. But don't do that, just build everything inside manylinux.

[gurgenyegoryan]

I tried updating the glibc version using the pyproject.toml file before building, but that didn't help either. I'll try to create on this docker image.

[gurgenyegoryan]

I tried right during the build of cibuildwheel in the pyproject.toml file to create a conan package and use it, but now it just failed, and not any specific error, just a few warnings.

lto-wrapper: warning: using serial compilation of 15 LTRANS jobs
  lto-wrapper: note: see the ‘-flto’ option documentation for more information

And one question, why when we use manylinux_2_28, but when do pip install pyarrow==10.0.0 it installed from other manylinux?

Maybe our problem from that?

[Czaki]

Because pyarrow do not prowide wheels for newer manylinux. And manylinux 2.17 should run without any problem on manylinux 2.28.

[gurgenyegoryan]

@henryiii We created the conan package right in the manylinux container and use that package, but cibuildwheel does not give anything specific what the problem is, only exit -11.