auditwheel repair marked internal so files as shared library dependencies #212

wdirons · 2019-12-12T22:27:12Z

Thanks to everyone that worked on adding manylinux 2014 support to auditwheel.

I'm trying to get build a manylinux 2014 whl file of TensorFlow for ppc64le, using the docker image quay.io/pypa/manylinux2014_ppc64le

This image contains auditwheel 3.0.0.0rc1

auditwheel show says I'm close to being compliant, I just have one external dependency to resolve:

# auditwheel show tf_nightly-2.1.0.dev20191210-cp36-cp36m-linux_ppc64le.whl

tf_nightly-2.1.0.dev20191210-cp36-cp36m-linux_ppc64le.whl is
consistent with the following platform tag: "linux_ppc64le".

The wheel references external versioned symbols in these system-
provided shared libraries: libc.so.6 with versions {'GLIBC_2.17'},
libpthread.so.0 with versions {'GLIBC_2.17'}, libgcc_s.so.1 with
versions {'GCC_4.2.0', 'GCC_3.0', 'GCC_3.3', 'GLIBC_2.0'}, libm.so.6
with versions {'GLIBC_2.17'}, ld64.so.2 with versions {'GLIBC_2.17'},
libdl.so.2 with versions {'GLIBC_2.17'}, librt.so.1 with versions
{'GLIBC_2.17'}, _pywrap_tensorflow_internal.so with versions
{'tensorflow'}

The following external shared libraries are required by the wheel:
{
    "ld64.so.2": "/lib64/ld-2.17.so",
    "libc.so.6": "/lib64/libc-2.17.so",
    "libdl.so.2": "/lib64/libdl-2.17.so",
    "libgcc_s.so.1": "/lib64/libgcc_s-4.8.5-20150702.so.1",
    "libm.so.6": "/lib64/libm-2.17.so",
    "libpthread.so.0": "/lib64/libpthread-2.17.so",
    "librt.so.1": "/lib64/librt-2.17.so"
}

In order to achieve the tag platform tag "manylinux2014_ppc64le" the
following shared library dependencies will need to be eliminated:

ld64.so.2

auditwheel repair appears to resolve the issue:

# auditwheel repair tf_nightly-2.1.0.dev20191210-cp36-cp36m-linux_ppc64le.whl
INFO:auditwheel.main_repair:Repairing tf_nightly-2.1.0.dev20191210-cp36-cp36m-linux_ppc64le.whl
INFO:auditwheel.wheeltools:Previous filename tags: linux_ppc64le
INFO:auditwheel.wheeltools:New filename tags: manylinux2014_ppc64le
INFO:auditwheel.wheeltools:Previous WHEEL info tags: cp36-cp36m-linux_ppc64le
INFO:auditwheel.wheeltools:New WHEEL info tags: cp36-cp36m-manylinux2014_ppc64le
INFO:auditwheel.main_repair:
Fixed-up wheel written to /root/tensorflow/tensorflow_pkg/wheelhouse/tf_nightly-2.1.0.dev20191210-cp36-cp36m-manylinux2014_ppc64le.whl

However when I run auditwheel show on the generated file, it list two internal so files as library dependencies:

# auditwheel show wheelhouse/tf_nightly-2.1.0.dev20191210-cp36-cp36m-manylinux2014_ppc64le.whl

tf_nightly-2.1.0.dev20191210-cp36-cp36m-manylinux2014_ppc64le.whl is
consistent with the following platform tag: "linux_ppc64le".

The wheel references external versioned symbols in these system-
provided shared libraries: libc.so.6 with versions {'GLIBC_2.17'},
libpthread.so.0 with versions {'GLIBC_2.17'}, libgcc_s.so.1 with
versions {'GCC_4.2.0', 'GLIBC_2.0', 'GCC_3.0', 'GCC_3.3'}, libm.so.6
with versions {'GLIBC_2.17'}, ld64.so.2 with versions {'GLIBC_2.17'},
libdl.so.2 with versions {'GLIBC_2.17'}, ld-2-cc40d7b0.17.so with
versions {'GLIBC_2.17'}, librt.so.1 with versions {'GLIBC_2.17'},
_pywrap_tensorflow_internal.so with versions {'tensorflow'}

The following external shared libraries are required by the wheel:
{
    "_pywrap_tensorflow_internal.so": null,
    "ld64.so.2": "/lib64/ld-2.17.so",
    "libc.so.6": "/lib64/libc-2.17.so",
    "libdl.so.2": "/lib64/libdl-2.17.so",
    "libgcc_s.so.1": "/lib64/libgcc_s-4.8.5-20150702.so.1",
    "libm.so.6": "/lib64/libm-2.17.so",
    "libpthread.so.0": "/lib64/libpthread-2.17.so",
    "librt.so.1": "/lib64/librt-2.17.so",
    "libtensorflow_framework.so.2": null
}

In order to achieve the tag platform tag "manylinux2014_ppc64le" the
following shared library dependencies will need to be eliminated:

_pywrap_tensorflow_internal.so, libtensorflow_framework.so.2

Any idea what might be wrong in this case? Thank you

gist with debug (-vv) enabled:

auditwheel show of original wheel: https://gist.github.com/wdirons/2ee22c009890e3436caf1c982ab5e81d
auditwheel repair: https://gist.github.com/wdirons/550039b96f9fe8ddbccea5cbec228c3f
auditwheel show of repaired wheel: https://gist.github.com/wdirons/7bb37290e722ede677cd13a8bac0b6a1

The text was updated successfully, but these errors were encountered:

mayeut · 2019-12-24T10:41:09Z

Thanks for the report.
Regarding the first run of audit wheel show, ld64.so.2 shall not be listed here and I opened a PR for this.

As for the issue itself, I did not have time to investigate this yet. Is it reproducible on x86_64 ? TensorFlow being quite complex, do they use something other than auditwheel to repair/rename wheels in official builds ?

wdirons · 2019-12-24T15:35:31Z

Thank you @mayeut for identifying that problem, if ld64.so.2 is not listed, I shouldn't need to run auditwheel repair in the first place.

I will try to create the problem on x86 and see if i can (after the holidays)

wdirons · 2020-01-07T18:45:14Z

I was unable to recreate the problem with auditwheel repair on x86.

di · 2020-02-04T18:10:08Z

Hi @wdirons, are you able to try the change proposed in #213 and see if this resolves your issue?

pip install git+https://github.com/mayeut/auditwheel.git@ld-linux

wdirons · 2020-02-04T18:18:14Z

Yes, I did verify the fix works. Sorry for not posting it here.

git clone -b ld-linux https://github.com/mayeut/auditwheel .
pip install .
# /opt/python/cp36-cp36m/bin/auditwheel show /root/tensorflow/tensorflow_pkg/tf_nightly-2.2.0.dev20200124-cp36-cp36m-linux_ppc64le.whl

tf_nightly-2.2.0.dev20200124-cp36-cp36m-linux_ppc64le.whl is
consistent with the following platform tag: "manylinux2014_ppc64le".

The wheel references external versioned symbols in these system-
provided shared libraries: libgcc_s.so.1 with versions {'GCC_3.0',
'GCC_3.3', 'GLIBC_2.0'}, libdl.so.2 with versions {'GLIBC_2.17'},
libm.so.6 with versions {'GLIBC_2.17'}, libpthread.so.0 with versions
{'GLIBC_2.17'}, libstdc++.so.6 with versions {'GLIBCXX_3.4.17',
'GLIBCXX_3.4.9', 'CXXABI_1.3', 'GLIBCXX_3.4.18', 'GLIBCXX_3.4',
'CXXABI_1.3.2', 'CXXABI_1.3.7', 'CXXABI_1.3.5', 'CXXABI_1.3.3',
'GLIBCXX_3.4.11', 'GLIBCXX_3.4.14', 'GLIBCXX_3.4.19',
'GLIBCXX_3.4.15'}, libc.so.6 with versions {'GLIBC_2.17'}, ld64.so.2
with versions {'GLIBC_2.17'}, librt.so.1 with versions {'GLIBC_2.17'},
_pywrap_tensorflow_internal.so with versions {'tensorflow'}

The following external shared libraries are required by the wheel:
{
    "libc.so.6": "/lib64/libc-2.17.so",
    "libdl.so.2": "/lib64/libdl-2.17.so",
    "libgcc_s.so.1": "/lib64/libgcc_s-4.8.5-20150702.so.1",
    "libm.so.6": "/lib64/libm-2.17.so",
    "libpthread.so.0": "/lib64/libpthread-2.17.so",
    "librt.so.1": "/lib64/librt-2.17.so",
    "libstdc++.so.6": "/lib64/libstdc++.so.6.0.19"
}

di · 2020-02-04T19:01:19Z

Thanks @wdirons. Would you be able to provide us with a .whl file that minimally reproduces this? It seems like adding a test to our test suite is what's holding up #213.

wdirons · 2020-02-04T23:44:00Z

I'm trying to determine a minimal reproduction outside of tensorflow, so far no luck but will keep trying.

di · 2020-02-05T00:06:05Z

@wdirons How big is the Tensorflow wheel that reproduces this?

wdirons · 2020-02-05T16:14:19Z

@di , the nightly build of TensorFlow for CPU is 147MB and is available here: https://powerci.osuosl.org/job/TensorFlow_PPC64LE_CPU_Nightly_Artifact/lastSuccessfulBuild/artifact/tensorflow_pkg/

(being a nightly build the file name changes nightly, I haven't yet done a release build from the quay manylinux 2014 container)

wdirons · 2020-02-05T20:52:10Z

@di , this can be recreated building numpy (v1.18.1) on ppc64le. Here is the generated whl file:

https://github.com/wdirons/auditwheel/blob/temp_212/numpy-1.18.1-cp36-cp36m-linux_ppc64le.whl

# auditwheel show numpy-1.18.1-cp36-cp36m-linux_ppc64le.whl

numpy-1.18.1-cp36-cp36m-linux_ppc64le.whl is consistent with the
following platform tag: "linux_ppc64le".

The wheel references external versioned symbols in these system-
provided shared libraries: libc.so.6 with versions {'GLIBC_2.17'},
libpthread.so.0 with versions {'GLIBC_2.17'}, ld64.so.2 with versions
{'GLIBC_2.17'}, libm.so.6 with versions {'GLIBC_2.17'}

The following external shared libraries are required by the wheel:
{
    "ld64.so.2": "/lib64/ld-2.17.so",
    "libc.so.6": "/lib64/libc-2.17.so",
    "libm.so.6": "/lib64/libm-2.17.so",
    "libpthread.so.0": "/lib64/libpthread-2.17.so"
}

In order to achieve the tag platform tag "manylinux2014_ppc64le" the
following shared library dependencies will need to be eliminated:

ld64.so.2

di · 2020-02-07T00:09:02Z

Thanks for the tip @wdirons, I was able to build a s390x numpy wheel which has this behavior as well, and added them to #213.

wdirons · 2020-04-20T22:18:58Z

The above fix is now merged and I verified it works using the master branch of auditwheel. Thank you all!

# /opt/python/cp37-cp37m/bin/pip install git+https://github.com/pypa/auditwheel
Collecting git+https://github.com/pypa/auditwheel
  Cloning https://github.com/pypa/auditwheel to /tmp/pip-req-build-9xuwhle6
  Running command git clone -q https://github.com/pypa/auditwheel /tmp/pip-req-build-9xuwhle6
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Requirement already satisfied: pyelftools>=0.24 in /opt/_internal/cpython-3.7.7/lib/python3.7/site-packages (from auditwheel==3.1.1.dev6) (0.26)
Collecting wheel==0.34.1
  Downloading wheel-0.34.1-py2.py3-none-any.whl (26 kB)
Building wheels for collected packages: auditwheel
  Building wheel for auditwheel (PEP 517) ... done
  Created wheel for auditwheel: filename=auditwheel-3.1.1.dev6-py3-none-any.whl size=34399 sha256=bf4cd4e8aa0f71378eaaf2828bd731cec60c78267e05dfcf6931cf9259e150a0
  Stored in directory: /tmp/pip-ephem-wheel-cache-x61ez06l/wheels/26/7e/b4/78471189ae4465a311261627eeab006af0f7108c88e3327887
Successfully built auditwheel
Installing collected packages: wheel, auditwheel
  Attempting uninstall: wheel
    Found existing installation: wheel 0.34.2
    Uninstalling wheel-0.34.2:
      Successfully uninstalled wheel-0.34.2
  Attempting uninstall: auditwheel
    Found existing installation: auditwheel 3.1.0
    Uninstalling auditwheel-3.1.0:
      Successfully uninstalled auditwheel-3.1.0
Successfully installed auditwheel-3.1.1.dev6 wheel-0.34.1
# auditwheel show tf_nightly_cpu-2.2.0.dev20200420-cp36-cp36m-linux_ppc64le.whl

tf_nightly_cpu-2.2.0.dev20200420-cp36-cp36m-linux_ppc64le.whl is
consistent with the following platform tag: "manylinux2014_ppc64le".

The wheel references external versioned symbols in these system-
provided shared libraries: libstdc++.so.6 with versions
{'GLIBCXX_3.4.17', 'CXXABI_1.3.5', 'GLIBCXX_3.4.9', 'GLIBCXX_3.4.18',
'CXXABI_1.3.3', 'CXXABI_1.3.7', 'CXXABI_1.3.6', 'GLIBCXX_3.4',
'GLIBCXX_3.4.11', 'GLIBCXX_3.4.14', 'GLIBCXX_3.4.15',
'GLIBCXX_3.4.19', 'CXXABI_1.3.2', 'CXXABI_1.3'}, libc.so.6 with
versions {'GLIBC_2.17'}, libgcc_s.so.1 with versions {'GCC_3.3',
'GLIBC_2.0', 'GCC_3.0'}, libpthread.so.0 with versions {'GLIBC_2.17'},
libm.so.6 with versions {'GLIBC_2.17'}, libdl.so.2 with versions
{'GLIBC_2.17'}, librt.so.1 with versions {'GLIBC_2.17'},
_pywrap_tensorflow_internal.so with versions {'tensorflow'}

The following external shared libraries are required by the wheel:
{
    "libc.so.6": "/lib64/libc-2.17.so",
    "libdl.so.2": "/lib64/libdl-2.17.so",
    "libgcc_s.so.1": "/lib64/libgcc_s-4.8.5-20150702.so.1",
    "libm.so.6": "/lib64/libm-2.17.so",
    "libpthread.so.0": "/lib64/libpthread-2.17.so",
    "librt.so.1": "/lib64/librt-2.17.so",
    "libstdc++.so.6": "/lib64/libstdc++.so.6.0.19"
}

mayeut mentioned this issue Dec 24, 2019

Always exclude ELF dynamic linker/loader #213

Merged

wdirons closed this as completed Apr 20, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

auditwheel repair marked internal so files as shared library dependencies #212

auditwheel repair marked internal so files as shared library dependencies #212

wdirons commented Dec 12, 2019

mayeut commented Dec 24, 2019

wdirons commented Dec 24, 2019 •

edited

Loading

wdirons commented Jan 7, 2020

di commented Feb 4, 2020

wdirons commented Feb 4, 2020

di commented Feb 4, 2020

wdirons commented Feb 4, 2020

di commented Feb 5, 2020

wdirons commented Feb 5, 2020

wdirons commented Feb 5, 2020 •

edited

Loading

di commented Feb 7, 2020

wdirons commented Apr 20, 2020

auditwheel repair marked internal so files as shared library dependencies #212

auditwheel repair marked internal so files as shared library dependencies #212

Comments

wdirons commented Dec 12, 2019

mayeut commented Dec 24, 2019

wdirons commented Dec 24, 2019 • edited Loading

wdirons commented Jan 7, 2020

di commented Feb 4, 2020

wdirons commented Feb 4, 2020

di commented Feb 4, 2020

wdirons commented Feb 4, 2020

di commented Feb 5, 2020

wdirons commented Feb 5, 2020

wdirons commented Feb 5, 2020 • edited Loading

di commented Feb 7, 2020

wdirons commented Apr 20, 2020

wdirons commented Dec 24, 2019 •

edited

Loading

wdirons commented Feb 5, 2020 •

edited

Loading