BLD: Try g0 on macos?

[lithomas1]

• closes ENH: strip debug symbols from macOS wheels #51900 (Replace xxxx with the GitHub issue number)
• Tests added and passed if fixing a bug or adding a new feature
• All code checks passed.
• Added type annotations to new arguments/methods/functions.
• Added an entry in the latest doc/source/whatsnew/vX.X.X.rst file if fixing a bug or adding a new feature.

[WillAyd]

Is the issue still seeing debug symbols on macOS? I think there is a Darwin option for gused that is the default which may be generating that:

https://gcc.gnu.org/onlinedocs/gcc/Darwin-Options.html

[lithomas1]

Is the issue still seeing debug symbols on macOS? I think there is a Darwin option for gused that is the default which may be generating that:

https://gcc.gnu.org/onlinedocs/gcc/Darwin-Options.html

Doesn't Apple use clang nowadays?

In the logs, I'm seeing a -g option being passed to clang, so I'm trying to cancel it out with the -g0 option (I read somewhere on Stack Overflow that that works, will need to see results, though).

[lithomas1]

I'm not sure who's passing it though.

[WillAyd]

Yea gcc is an alias for clang on macOS by default. I think if you are using conda-compilers though it may actually install gcc.

AFAIK the APIs between the two are supposed to be similar, but I didn't see any documentation for what clang has on gused

https://clang.llvm.org/docs/ClangCommandLineReference.html

[WillAyd]

In the logs, I'm seeing a -g option being passed to clang, so I'm trying to cancel it out with the -g0 option (I read somewhere on Stack Overflow that that works, will need to see results, though).

Yea I think that is true. The GCC documentation for options is really good and thorough. This also confirms that -g0 should negate -g. Let's see

https://gcc.gnu.org/onlinedocs/gcc/Debugging-Options.html#Debugging-Options

[WillAyd]

Where in the logs do you see -g showing up? I must be looking in the wrong places.

If I had to guess it's something conda might be adding for us. I think Ralf pointed that out on the meson PR, which was pretty enlightening.

[lithomas1]

Where in the logs do you see -g showing up? I must be looking in the wrong places.

If I had to guess it's something conda might be adding for us. I think Ralf pointed that out on the meson PR, which was pretty enlightening.

Wheels are built with cibuildwheel(uses the GHA python's and sometimes Docker on Linux), so there's absolutely no conda involved right now.

Right now what's really infuriating me is that, even though I'm passing -g0 via CFLAGS, it is somehow showing up as -g, and I'm not sure what I'm doing wrong.

gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -g -arch x86_64 -DNPY_NO_DEPRECATED_API=0 -I/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-build-env-e0sc4bd0/overlay/lib/python3.8/site-packages/numpy/core/include -I/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-am2shr8g/cp38-macosx_x86_64/build/venv/include -I/Library/Frameworks/Python.framework/Versions/3.8/include/python3.8 -c pandas/_libs/groupby.c -o build/temp.macosx-10.9-x86_64-cpython-38/pandas/_libs/groupby.o -Wno-error=deprecated-declarations -Wno-error=unreachable-code

Here is an example command. The second -g(before -arch) is mine, but the first one is from something(sysconfig? default flags).

You can find this by ctrl+f for gcc in the logs.

[lithomas1]

Where in the logs do you see -g showing up? I must be looking in the wrong places.
If I had to guess it's something conda might be adding for us. I think Ralf pointed that out on the meson PR, which was pretty enlightening.

Wheels are built with cibuildwheel(uses the GHA python's and sometimes Docker on Linux), so there's absolutely no conda involved right now.

Right now what's really infuriating me is that, even though I'm passing -g0 via CFLAGS, it is somehow showing up as -g, and I'm not sure what I'm doing wrong.

gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -g -arch x86_64 -DNPY_NO_DEPRECATED_API=0 -I/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-build-env-e0sc4bd0/overlay/lib/python3.8/site-packages/numpy/core/include -I/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-am2shr8g/cp38-macosx_x86_64/build/venv/include -I/Library/Frameworks/Python.framework/Versions/3.8/include/python3.8 -c pandas/_libs/groupby.c -o build/temp.macosx-10.9-x86_64-cpython-38/pandas/_libs/groupby.o -Wno-error=deprecated-declarations -Wno-error=unreachable-code

Here is an example command. The second -g(before -arch) is mine, but the first one is from something(sysconfig? default flags).

You can find this by ctrl+f for gcc in the logs.

Well, I'm just being stupid. It turns out I was overriding the cibuildwheel environment vars for macos 🤦.

[WillAyd]

nice find. a bit orthogonal but wonder if we should strip all symbols here too

[WillAyd]

Guess it would help if I read the comment two lines up

[lithomas1]

@jameslamb

I think I got the macOS symbols as well now. Savings are on par with what you mentioned

Savings are currently 0.4 MB compressed for the x86_64 wheel, and 0.2 MB for the arm64 wheel.
(Since universal2 has both x86_64 and arm64, that's why the savings here are like half your savings. We also don't make those wheels anymore as of 2.0rc0, since numpy dropped them as well).

If you'd like to check one of the wheels yourself, here is a link to one of the wheels
https://github.com/pandas-dev/pandas/suites/11567086673/artifacts/599127251
(contains both x86_64 and arm64 in that zip file)

[mroeschke]

Thanks @lithomas1

[jameslamb]

@jameslamb

I think I got the macOS symbols as well now. Savings are on par with what you mentioned...

Thanks so much again for the quick resolution on this! I just pulled those wheels from the link you provided... don't see debug symbols in either of them 🎉

If it interests you... I originally found these debug symbols (#51900) using a side-project I've been working on since last year's SciPy conference: https://github.com/jameslamb/pydistcheck. It's intended to be a CLI you run against wheels and sdists in CI, to catch issues like this (click here for a full list of what it checks) and ensure that changes to your build process don't introduce regressions like letting the debug symbols back in.

Anyway, cheers and enjoy the rest of your week 👋🏻