No automatic musllinux compliance

[tmtenbrink]

Please provide the following information:

• Your python version (python -V): Python 3.9.5
• Your pip version (pip -V): Pip 21.3
• The version of the bindings you're using, if any: PyO3 0.14.5

I'm running the latest version of maturin (0.11.5) and saw that recently maturin should be able to auto-detect musl (#642). However, when I try to build my project using maturin:

poetry run maturin build --release --compatibility musllinux_1_2

I still seem to get the same issue as in #642 even though I think the fix is supposed to be in the most recent version.

💥 maturin failed
  Caused by: Error ensuring musllinux_1_2 compliance
  Caused by: Your library is not musllinux_1_2 compliant because it links the following forbidden libraries: ["libgcc_s.so.1"]

I am running the rust:alpine docker image, with default apk python3, etc.

I am able to fix the problem using auditwheel manually (below the output of auditwheel show, auditwheel repair then fixes it):

rustfrc-1.0.2-cp39-cp39-linux_x86_64.whl is consistent with the
following platform tag: "linux_x86_64".

The wheel references external versioned symbols in these
system-provided shared libraries: libgcc_s.so.1 with versions
{'GCC_4.2.0', 'GCC_3.0', 'GCC_3.3'}

The following external shared libraries are required by the wheel:
{
    "libc.musl-x86_64.so.1": "/lib/ld-musl-x86_64.so.1",
    "libgcc_s.so.1": "/usr/lib/libgcc_s.so.1"
}

In order to achieve the tag platform tag "musllinux_1_2_x86_64" the
following shared library dependencies will need to be eliminated:

libgcc_s.so.1

[tmtenbrink]

Somewhat related question, is there a reason the musllinux wheels for maturin itself are not published to PyPI?

[messense]

See #642 (comment), rust-lang/rust#82521

maturin haven't implemented the repairment part of auditwheel yet, so it can't bundle libgcc_s.so.1 into built wheel to make it musllinux compliant.

[messense]

Somewhat related question, is there a reason the musllinux wheels for maturin itself are not published to PyPI?

I've opened #650 to track this.

[messense]

musllinux_1_1 wheels for maturin is now available in v0.11.6-beta.1: https://pypi.org/project/maturin/0.11.6_beta.1/#files

/ # apk add py3-pip
fetch https://dl-cdn.alpinelinux.org/alpine/edge/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/edge/community/x86_64/APKINDEX.tar.gz
(1/39) Installing libbz2 (1.0.8-r1)
(2/39) Installing expat (2.4.1-r0)
(3/39) Installing libffi (3.3-r2)
(4/39) Installing gdbm (1.20-r2)
(5/39) Installing xz-libs (5.2.5-r0)
(6/39) Installing libgcc (10.3.1_git20210921-r1)
(7/39) Installing libstdc++ (10.3.1_git20210921-r1)
(8/39) Installing mpdecimal (2.5.1-r1)
(9/39) Installing ncurses-terminfo-base (6.2_p20211009-r0)
(10/39) Installing ncurses-libs (6.2_p20211009-r0)
(11/39) Installing readline (8.1.1-r0)
(12/39) Installing sqlite-libs (3.36.0-r0)
(13/39) Installing python3 (3.9.7-r2)
(14/39) Installing py3-appdirs (1.4.4-r2)
(15/39) Installing py3-chardet (4.0.0-r2)
(16/39) Installing py3-idna (3.2-r0)
(17/39) Installing py3-urllib3 (1.26.7-r0)
(18/39) Installing py3-certifi (2020.12.5-r1)
(19/39) Installing py3-requests (2.25.1-r4)
(20/39) Installing py3-msgpack (1.0.2-r1)
(21/39) Installing py3-lockfile (0.12.2-r4)
(22/39) Installing py3-cachecontrol (0.12.6-r1)
(23/39) Installing py3-colorama (0.4.4-r1)
(24/39) Installing py3-contextlib2 (21.6.0-r1)
(25/39) Installing py3-distlib (0.3.3-r0)
(26/39) Installing py3-distro (1.6.0-r0)
(27/39) Installing py3-six (1.15.0-r1)
(28/39) Installing py3-webencodings (0.5.1-r4)
(29/39) Installing py3-html5lib (1.1-r1)
(30/39) Installing py3-parsing (2.4.7-r2)
(31/39) Installing py3-packaging (20.9-r1)
(32/39) Installing py3-tomli (1.2.1-r0)
(33/39) Installing py3-pep517 (0.11.0-r1)
(34/39) Installing py3-progress (1.6-r0)
(35/39) Installing py3-retrying (1.3.3-r2)
(36/39) Installing py3-ordered-set (4.0.2-r2)
(37/39) Installing py3-setuptools (52.0.0-r4)
(38/39) Installing py3-toml (0.10.2-r2)
(39/39) Installing py3-pip (20.3.4-r1)
Executing busybox-1.33.0-r2.trigger
OK: 77 MiB in 53 packages
/ # pip install -U pip
Requirement already satisfied: pip in /usr/lib/python3.9/site-packages (20.3.4)
Collecting pip
  Downloading pip-21.3-py3-none-any.whl (1.7 MB)
     |████████████████████████████████| 1.7 MB 461 kB/s
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 20.3.4
    Uninstalling pip-20.3.4:
      Successfully uninstalled pip-20.3.4
Successfully installed pip-21.3
/ # pip --version
pip 21.3 from /usr/lib/python3.9/site-packages/pip (python 3.9)
/ # pip install --pre maturin
Collecting maturin
  Downloading maturin-0.11.6_beta.1-py3-none-musllinux_1_1_x86_64.whl (5.6 MB)
     |████████████████████████████████| 5.6 MB 10.7 MB/s
Requirement already satisfied: toml~=0.10.2 in /usr/lib/python3.9/site-packages (from maturin) (0.10.2)
Installing collected packages: maturin
Successfully installed maturin-0.11.6b1
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
/ # maturin
maturin 0.11.6-beta.1
Build and publish crates with pyo3, rust-cpython and cffi bindings as well as rust binaries as python packages

USAGE:
    maturin <SUBCOMMAND>

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information

SUBCOMMANDS:
    build          Build the crate into python packages
    develop        Installs the crate as module in the current virtualenv
    help           Prints this message or the help of the given subcommand(s)
    list-python    Searches and lists the available python installations
    publish        Build and publish the crate as python packages to pypi
    sdist          Build only a source distribution (sdist) without compiling
    upload         Uploads python packages to pypi

[tmtenbrink]

maturin haven't implemented the repairment part of auditwhAeel yet, so it can't bundle libgcc_s.so.1 into built wheel to make it musllinux compliant.

Ah my apologies I misinterpreted the problem in that issue, anyways thanks for the quick reply and your work on this library, it makes building Rust/Python libraries (which imo are going to be used a lot for scientific purposes in the future) a blast! You can close this if you want because if I understand you correctly it is not actually intended that maturin fixes the wheels itself?

[messense]

You can close this if you want because if I understand you correctly it is not actually intended that maturin fixes the wheels itself?

It will great if we implement wheel repair eventually in maturin, so let's keep it open to track the process.

[jnicholls]

Thanks for tracking this issue, I too am running into the issue of libgcc being linked. I'm just skipping auditwheel for now, but looking forward to seeing the auto-repair functionality!