ENH: strip debug symbols from macOS wheels

[jameslamb]

Feature Type

• Adding new functionality to pandas

• Changing existing functionality in pandas

• Removing existing functionality in pandas

Problem Description

I found tonight that .so files in pandas macOS wheels contain debug symbols.

By my estimate, these add around 1MB compressed and around 4 MB uncompressed to the cp311-cp311-macosx_10_9_universal2 wheel.

how I estimated that (click me)

On my macbook tonight (macOS 12.2, Intel CPU), I downloaded the latest CPython 3.11, macOS universal wheel and checked its size with du.

WHEEL_FILENAME='pandas-1.5.3-cp311-cp311-macosx_10_9_universal2.whl'
du -h ${WHEEL_FILENAME}
# 18M

Next, I unzipped it and used dsymutil to check for debug symbols.

mkdir unpack-dir
cp \
    ./${WHEEL_FILENAME} \
    ./unpack-dir

cd unpack-dir
unzip ./${WHEEL_FILENAME}

# check uncompressed size
du -sh pandas
# 65M

du -h pandas/_libs/missing.cpython-311-darwin.so
# 524K

dsymutil -s pandas/_libs/missing.cpython-311-darwin.so | grep N_OSO

That showed some (and interesting that they seem to include filepaths from a CI system's runner image 👀 )

# [   765] 00007670 66 (N_OSO        ) 03     0001   0000000063c8cc76 '/Users/runner/work/1/s/pandas/build/temp.macosx-10.9-x86_64-cpython-311/pandas/_libs/missing.o'
# [   261] 00002a95 66 (N_OSO        ) 00     0001   0000000063c8cef7 '/Users/runner/work/1/s/pandas/build/temp.macosx-11.0-arm64-cpython-311/pandas/_libs/missing.o'

Next, I tried stripping all of the .so files using OSX strip (docs link).

find \
    $(pwd)/pandas \
    -type f \
    -name '*.so' \
    -exec strip -S '{}' \;

Warning

This did produce warnings like the following: /Library/Developer/CommandLineTools/usr/bin/strip: warning: changes being made to the file will invalidate the code signature in: /private/tmp/check-pandas/unpack-dir/pandas/_libs/interval.cpython-311-darwin.so (for architecture arm64)

So maybe that's not the best approach for pandas actual build pipeline.

Packed the wheel back up

# check uncompressed size again
du -sh pandas
# 61M

rm ./${WHEEL_FILENAME}
zip -r ${WHEEL_FILENAME} .

# check compressed size again
du -h ${WHEEL_FILENAME}
# 17M

Then installed it (making sure the environment didn't have pandas installed before), and ran the tests.

pip uninstall --yes pandas
pip install ${WHEEL_FILENAME}
pip install pytest hypothesis
python -c "import pandas as pd; pd.test()"

Installation worked, and the tests all ran to completion, with the following results:

= 1 failed, 152571 passed, 24052 skipped, 1362 xfailed, 12 xpassed, 1762 warnings, 23 errors in 1144.98s (0:19:04) =

I checked all of the wheels (for all platforms) from the 1.5.3 release (PyPI link) and only found debug symbols in the macOS ones.

I did not repeat the analysis above to estimate the size impact of those symbols for any wheels other than the cp311-cp311-macosx_10_9_universal2 one.

Feature Description

If the inclusion of these symbols is not intentional and if I'm right that they're not necessary, please consider removing them.

This might be accomplished by avoiding them in the first place, e.g.:

• preventing compiler flag -g or similar from being used at build time
  • e.g. by setting -DCMAKE_BUILD_TYPE=Release if CMake is being used)*
• passing build-time flags to the linker or compiler which explicitly strips debug symbols
  • e.g. -Wl,strip-all for gcc as suggested in Shared libraries are not stripped in the manylinux1 wheels #19531 (comment)

Or by stripping those built objects after the fact.

• e.g. with strip -S _file.so on MacOS

I'm not familiar enough with pandas build system and preferred toolchain to offer more specific recommendations, sorry.

Alternative Solutions

Instead of a pandas-specific fix, it might be worth adding support to delocate similar to how auditwheel supports stripping after the fact with auditwheel repair --strip for Linux.

I'm not aware of another such tool that works with macOS wheels containing mach-o format objects.

Additional Context

Relevant Discussions

I can see there was some discussion about stripping this project's wheels back in 2018, although that looks to be mostly about Linux wheels:

• Update multibuild to a version that strips binaries in wheels MacPython/pandas-wheels#24
• Strip binaries on non-OSX builds multi-build/multibuild#140
• Pandas install includes hefty 29MB tests/ directory #19681 (comment)
• Shared libraries are not stripped in the manylinux1 wheels #19531
• Manylinux wheel size could be reduced by 54% #20719

This conversation from 2020 contains some details about building pandas on macOS with `clang:

• BUG: clang-9: error: -Wl,-export_dynamic: 'linker' input unused (OSX) #33314

Other misc. discussions from similar projects about stripping debug symbols while building Python wheels.

• Option to strip binaries generated by auditwheel pypa/auditwheel#63
• ENH:Distutils Remove debugging symbols by default numpy/numpy#16110
• Strip debug symbols of wheels pypa/cibuildwheel#331
• strip when linking MacPython/numpy-wheels#82

Notes for Reviewers

Thanks very much for your time and consideration!

[jameslamb]

One other note for reviewers... I checked all wheels from all files in the the 2.0.0 release candidate (PyPI link) and found that in addition to the macOS wheels, the manylinux wheels there also contain debug symbols.

evidence of that (click me)

mkdir /tmp/linux-wheels
cd /tmp/linux-wheels

curl -O \
https://files.pythonhosted.org/packages/3e/55/9212e3cca8c3c2ac8cf6ea85491080e011523bf5a0886782110ef69dfbe5/pandas-2.0.0rc0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

unzip pandas-2.0.0rc0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

# list all symbols
docker run \
    --rm \
    -v $(pwd):/opt/check \
    --workdir /opt/check \
    python:3.11  \
   nm --debug-syms pandas/_libs/algos.cpython-311-x86_64-linux-gnu.so

# show that strip reduces the size
docker run \
    --rm \
    -v $(pwd):/opt/check \
    --workdir /opt/check \
    python:3.11  \
    du -h pandas/_libs/algos.cpython-311-x86_64-linux-gnu.so
# 2.2M

docker run \
    --rm \
    -v $(pwd):/opt/check \
    --workdir /opt/check \
    python:3.11  \
    /bin/bash -c 'strip --strip-unneeded pandas/_libs/algos.cpython-311-x86_64-linux-gnu.so && du -h pandas/_libs/algos.cpython-311-x86_64-linux-gnu.so'
# 2.0M

Is that expected? If not, would you like me to open a separate issue documenting it?

[lithomas1]

Hi @jameslamb,
Thanks for reporting this.

The macOS issue is expected since we can't pass flags for stripping to the linker.
I'll try to look into this (I think if this is done before delocate is run, delocate might be able to fix the signatures?).

I'll try to look into the manylinux issue more. We migrated to a new build system with cibuildwheel, and currently strip the wheels with -Wl, strip-debug.

When I added this, I kind of eyeballed the wheel sizes, but I'll try strip-all to see if that makes a difference.
multi-build/multibuild#162 may also be related.

[jameslamb]

The macOS issue is expected since we can't pass flags for stripping to the linker

Thanks for looking into it! I'm not that familiar with pandas build toolchain, so sorry that I can't offer more specific advice, but as a general comment about building C/C++ wheels stripping at linking time doesn't have to be the only place to try to fix this.

You could also investigate whether -g or any of its variants are making it into CXXFLAGS passed to the compiler (and try to avoid that), or try unzipping the wheel after building, stripping objects after-the-fact with the strip utility, and then re-zipping the wheel (like in the example code I provided above).