Skip to content

Commit

Permalink
macOS: disable fixup chains when linking extension modules
Browse files Browse the repository at this point in the history
On macOS, extension library builds usually rely on the flag ``-undefined
dynamic_lookup`` to get the linker to temporarily accept missing CPython
API symbols. As the name implies, those symbols are then dynamically
resolved when Python imports such an extension library. Binaries
produced in this way are more portable since they don't contain a
hardcoded path to the CPython library. This is critical when
distributing binary wheels on PyPI, etc..

When targeting macOS>=12, XCode recently started to generate a somewhat
ominous warning:

``-undefined dynamic_lookup may not work with chained fixups``

For further detail on what chained fixups are, see the following
informative WWDC
https://developer.apple.com/videos/play/wwdc2022/110362/ (the relevant
part starts about 20mins into the video)

The message that the dynamic resolution functionality became broken.
I reported this to Apple via feedback request FB11767124.

Apple investigated the request and updated the behavior of ``ld``
starting with XCode 14.3b1+: it now disables the fixup chain linker
optimization whenever ``-undefined dynamic_lookup`` is specified. The
feedback from Apple also stated that the parameter
``-Wl,-no_fixup_chains`` should be specified on pre-14.3 XCode versions
to ensure correct behavior.

This commit realizes those changes by passing a flag to *always* disable
fixup chains. Related discussion is available in the CPython repository
(python/cpython#97524).
  • Loading branch information
wjakob committed Feb 27, 2023
1 parent 3cc7e42 commit 70a0a37
Show file tree
Hide file tree
Showing 2 changed files with 6 additions and 4 deletions.
4 changes: 2 additions & 2 deletions docs/compiling.rst
Original file line number Diff line number Diff line change
Expand Up @@ -583,12 +583,12 @@ using ``pip`` or ``conda``. If it hasn't, you can also manually specify
``python3-config --includes``.

On macOS: the build command is almost the same but it also requires passing
the ``-undefined dynamic_lookup`` flag so as to ignore missing symbols when
the ``-undefined suppress -flat_namespace`` flag so as to ignore missing symbols when
building the module:

.. code-block:: bash
$ c++ -O3 -Wall -shared -std=c++11 -undefined dynamic_lookup $(python3 -m pybind11 --includes) example.cpp -o example$(python3-config --extension-suffix)
$ c++ -O3 -Wall -shared -std=c++11 -undefined suppress -flat_namespace $(python3 -m pybind11 --includes) example.cpp -o example$(python3-config --extension-suffix)
In general, it is advisable to include several additional build parameters
that can considerably reduce the size of the created binary. Refer to section
Expand Down
6 changes: 4 additions & 2 deletions tools/pybind11Common.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -75,15 +75,17 @@ if(CMAKE_VERSION VERSION_LESS 3.13)
set_property(
TARGET pybind11::python_link_helper
APPEND
PROPERTY INTERFACE_LINK_LIBRARIES "$<$<PLATFORM_ID:Darwin>:-undefined dynamic_lookup>")
PROPERTY INTERFACE_LINK_LIBRARIES
"$<$<PLATFORM_ID:Darwin>:-undefined dynamic_lookup -no_fixup_chains>")
else()
# link_options was added in 3.13+
# This is safer, because you are ensured the deduplication pass in CMake will not consider
# these separate and remove one but not the other.
set_property(
TARGET pybind11::python_link_helper
APPEND
PROPERTY INTERFACE_LINK_OPTIONS "$<$<PLATFORM_ID:Darwin>:LINKER:-undefined,dynamic_lookup>")
PROPERTY INTERFACE_LINK_OPTIONS
"$<$<PLATFORM_ID:Darwin>:LINKER:-undefined,dynamic_lookup,-no_fixup_chains>")
endif()

# ------------------------ Windows extras -------------------------
Expand Down

0 comments on commit 70a0a37

Please sign in to comment.