Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prevent segfaults by adding macOS weaklinking for mkfifoat and mknodat system calls added in macOS 13 Ventura #97897

Closed
ned-deily opened this issue Oct 5, 2022 · 3 comments
Labels
3.9 only security fixes 3.10 only security fixes 3.11 only security fixes 3.12 bugs and security fixes OS-mac release-blocker type-bug An unexpected behavior, bug, or error type-crash A hard crash of the interpreter, possibly with a core dump

Comments

@ned-deily
Copy link
Member

macOS 13 Ventura adds support for the POSIX mkfifoat and mknodat system calls which are used in the Python os module via posixmodule.c when available on the OS. In order to continue to support building Python on a newer macOS version to also run on older systems, we need to add weaklinking support for them in posixmodule.c as was done for other similar system calls added in recent macOS releases.

Unfortunately, while macOS 13 has not yet been released as of this moment, Apple has pushed out a new version of the Command Line Tools for macOS 12 Monterey, Command Line Tools beta 3 for Xcode 14.1, which changes the default macOS SDK version to MacOSX13.0. When building on macOS 12 with the macOS 13 SDK, posixmodule compiles support for mkfifoat and mknodat which will cause Python segfaults when attempting to use the dir_fd option of os.mkfifo() or os.mknod() on macOS 12 (or older). An easily overlooked symptom of the problem is the following set of warning messages during compilation:

./Modules/posixmodule.c:10627:22: warning: 'mkfifoat' is only available on macOS 13.0 or newer [-Wunguarded-availability-new]
            result = mkfifoat(dir_fd, path->narrow, mode);
                     ^~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/stat.h:394:9: note: 'mkfifoat' has been marked as being introduced in macOS 13.0 here, but the deployment target is macOS 12.6.0
int     mkfifoat(int, const char *, mode_t) __API_AVAILABLE(macos(13.0), ios(16.0), tvos(16.0), watchos(9.0));
        ^
./Modules/posixmodule.c:10627:22: note: enclose 'mkfifoat' in a __builtin_available check to silence this warning
            result = mkfifoat(dir_fd, path->narrow, mode);
                     ^~~~~~~~
./Modules/posixmodule.c:10679:22: warning: 'mknodat' is only available on macOS 13.0 or newer [-Wunguarded-availability-new]
            result = mknodat(dir_fd, path->narrow, mode, device);
                     ^~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/stat.h:395:9: note: 'mknodat' has been marked as being introduced in macOS 13.0 here, but the deployment target is macOS 12.6.0
int     mknodat(int, const char *, mode_t, dev_t) __API_AVAILABLE(macos(13.0), ios(16.0), tvos(16.0), watchos(9.0));
        ^
./Modules/posixmodule.c:10679:22: note: enclose 'mknodat' in a __builtin_available check to silence this warning
            result = mknodat(dir_fd, path->narrow, mode, device);
                     ^~~~~~~

One workaround is to force use of the macOS 12 SDK:

SDKROOT=$(xcrun --show-sdk-path --sdk macosx12.3) ./configure ...

@ned-deily ned-deily added type-bug An unexpected behavior, bug, or error OS-mac needs backport to 3.10 only security fixes 3.11 only security fixes 3.10 only security fixes type-crash A hard crash of the interpreter, possibly with a core dump 3.12 bugs and security fixes needs backport to 3.11 only security fixes labels Oct 5, 2022
@ned-deily
Copy link
Member Author

Since this could potentially cause segfaults on 3.9, perhaps a 3.9 backport should be considered (@ambv ?). (We do not support this kind of macOS weaklinking on 3.8 and earlier systems.)

ned-deily added a commit to ned-deily/cpython that referenced this issue Oct 5, 2022
…3 SDK

The macOS 13 SDK includes support for the mkfifoat and mknodat system calls.
Using the dir_fd opton with either os.mkfifo() or os.mknod() can result in a
segfault if cpython is built with the macOS 13 SDK but run on an earlier
version of macOS. Prevent this by adding runtime support for detection of
these system calls ("weaklinking") as is done for other newer syscalls on
macOS.
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Oct 6, 2022
…3 SDK (pythonGH-97944)

The macOS 13 SDK includes support for the `mkfifoat` and `mknodat` system calls.
 Using the `dir_fd` option with either `os.mkfifo` or `os.mknod` could result in a
 segfault if cpython is built with the macOS 13 SDK but run on an earlier
 version of macOS. Prevent this by adding runtime support for detection of
 these system calls ("weaklinking") as is done for other newer syscalls on
 macOS.
(cherry picked from commit 6d0a019)

Co-authored-by: Ned Deily <[email protected]>
ned-deily added a commit that referenced this issue Oct 6, 2022
…H-97944)

The macOS 13 SDK includes support for the `mkfifoat` and `mknodat` system calls.
 Using the `dir_fd` option with either `os.mkfifo` or `os.mknod` could result in a
 segfault if cpython is built with the macOS 13 SDK but run on an earlier
 version of macOS. Prevent this by adding runtime support for detection of
 these system calls ("weaklinking") as is done for other newer syscalls on
 macOS.
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Oct 6, 2022
…3 SDK (pythonGH-97944)

The macOS 13 SDK includes support for the `mkfifoat` and `mknodat` system calls.
 Using the `dir_fd` option with either `os.mkfifo` or `os.mknod` could result in a
 segfault if cpython is built with the macOS 13 SDK but run on an earlier
 version of macOS. Prevent this by adding runtime support for detection of
 these system calls ("weaklinking") as is done for other newer syscalls on
 macOS.
(cherry picked from commit 6d0a019)

Co-authored-by: Ned Deily <[email protected]>
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Oct 6, 2022
…3 SDK (pythonGH-97944)

The macOS 13 SDK includes support for the `mkfifoat` and `mknodat` system calls.
 Using the `dir_fd` option with either `os.mkfifo` or `os.mknod` could result in a
 segfault if cpython is built with the macOS 13 SDK but run on an earlier
 version of macOS. Prevent this by adding runtime support for detection of
 these system calls ("weaklinking") as is done for other newer syscalls on
 macOS.
(cherry picked from commit 6d0a019)

Co-authored-by: Ned Deily <[email protected]>
@ned-deily ned-deily added release-blocker 3.9 only security fixes labels Oct 6, 2022
@ned-deily
Copy link
Member Author

Marking as release blocker for @pablogsal's review with regard to 3.11.0.

@ned-deily ned-deily removed needs backport to 3.10 only security fixes needs backport to 3.11 only security fixes labels Oct 6, 2022
carljm added a commit to carljm/cpython that referenced this issue Oct 6, 2022
* main:
  pythonGH-88050: fix race in closing subprocess pipe in asyncio  (python#97951)
  pythongh-93738: Disallow pre-v3 syntax in the C domain (python#97962)
  pythongh-95986: Fix the example using match keyword (python#95989)
  pythongh-97897: Prevent os.mkfifo and os.mknod segfaults with macOS 13 SDK (pythonGH-97944)
  pythongh-94808: Cover `PyUnicode_Count` in CAPI (python#96929)
  pythongh-94808: Cover `PyObject_PyBytes` case with custom `__bytes__` method (python#96610)
  pythongh-95691: Doc BufferedWriter and BufferedReader (python#95703)
  pythonGH-88968: Add notes about socket ownership transfers (python#97936)
  pythongh-96865: [Enum] fix Flag to use CONFORM boundary (pythonGH-97528)
ambv pushed a commit that referenced this issue Oct 6, 2022
…13 SDK (GH-97944) (#97969)

The macOS 13 SDK includes support for the `mkfifoat` and `mknodat` system calls.
 Using the `dir_fd` option with either `os.mkfifo` or `os.mknod` could result in a
 segfault if cpython is built with the macOS 13 SDK but run on an earlier
 version of macOS. Prevent this by adding runtime support for detection of
 these system calls ("weaklinking") as is done for other newer syscalls on
 macOS.
(cherry picked from commit 6d0a019)

Co-authored-by: Ned Deily <[email protected]>
ambv pushed a commit that referenced this issue Oct 6, 2022
…13 SDK (GH-97944) (#97967)

The macOS 13 SDK includes support for the `mkfifoat` and `mknodat` system calls.
 Using the `dir_fd` option with either `os.mkfifo` or `os.mknod` could result in a
 segfault if cpython is built with the macOS 13 SDK but run on an earlier
 version of macOS. Prevent this by adding runtime support for detection of
 these system calls ("weaklinking") as is done for other newer syscalls on
 macOS.
(cherry picked from commit 6d0a019)

Co-authored-by: Ned Deily <[email protected]>
ambv pushed a commit that referenced this issue Oct 6, 2022
…3 SDK (GH-97944) (#97968)

The macOS 13 SDK includes support for the `mkfifoat` and `mknodat` system calls.
 Using the `dir_fd` option with either `os.mkfifo` or `os.mknod` could result in a
 segfault if cpython is built with the macOS 13 SDK but run on an earlier
 version of macOS. Prevent this by adding runtime support for detection of
 these system calls ("weaklinking") as is done for other newer syscalls on
 macOS.
(cherry picked from commit 6d0a019)

Co-authored-by: Ned Deily <[email protected]>
@ned-deily
Copy link
Member Author

Backports to all current branches have been merged.

Repository owner moved this from Todo to Done in Release and Deferred blockers 🚫 Oct 7, 2022
carljm added a commit to carljm/cpython that referenced this issue Oct 8, 2022
* main: (53 commits)
  pythongh-94808: Coverage: Test that maximum indentation level is handled (python#95926)
  pythonGH-88050: fix race in closing subprocess pipe in asyncio  (python#97951)
  pythongh-93738: Disallow pre-v3 syntax in the C domain (python#97962)
  pythongh-95986: Fix the example using match keyword (python#95989)
  pythongh-97897: Prevent os.mkfifo and os.mknod segfaults with macOS 13 SDK (pythonGH-97944)
  pythongh-94808: Cover `PyUnicode_Count` in CAPI (python#96929)
  pythongh-94808: Cover `PyObject_PyBytes` case with custom `__bytes__` method (python#96610)
  pythongh-95691: Doc BufferedWriter and BufferedReader (python#95703)
  pythonGH-88968: Add notes about socket ownership transfers (python#97936)
  pythongh-96865: [Enum] fix Flag to use CONFORM boundary (pythonGH-97528)
  pythongh-65961: Raise `DeprecationWarning` when `__package__` differs from `__spec__.parent` (python#97879)
  docs(typing): add "see PEP 675" to LiteralString (python#97926)
  pythongh-97850: Remove all known instances of module_repr() (python#97876)
  I changed my surname early this year (python#96671)
  pythongh-93738: Documentation C syntax (:c:type:<C type> -> :c:expr:<C type>) (python#97768)
  pythongh-91539: improve performance of get_proxies_environment  (python#91566)
  build(deps): bump actions/stale from 5 to 6 (python#97701)
  pythonGH-95172 Make the same version `versionadded` oneline (python#95172)
  pythongh-88050: Fix asyncio subprocess to kill process cleanly when process is blocked (python#32073)
  pythongh-93738: Documentation C syntax (Function glob patterns -> literal markup) (python#97774)
  ...
mpage pushed a commit to mpage/cpython that referenced this issue Oct 11, 2022
…3 SDK (pythonGH-97944)

The macOS 13 SDK includes support for the `mkfifoat` and `mknodat` system calls.
 Using the `dir_fd` option with either `os.mkfifo` or `os.mknod` could result in a
 segfault if cpython is built with the macOS 13 SDK but run on an earlier
 version of macOS. Prevent this by adding runtime support for detection of
 these system calls ("weaklinking") as is done for other newer syscalls on
 macOS.
pablogsal pushed a commit that referenced this issue Oct 22, 2022
…13 SDK (GH-97944) (#97969)

The macOS 13 SDK includes support for the `mkfifoat` and `mknodat` system calls.
 Using the `dir_fd` option with either `os.mkfifo` or `os.mknod` could result in a
 segfault if cpython is built with the macOS 13 SDK but run on an earlier
 version of macOS. Prevent this by adding runtime support for detection of
 these system calls ("weaklinking") as is done for other newer syscalls on
 macOS.
(cherry picked from commit 6d0a019)

Co-authored-by: Ned Deily <[email protected]>
rt121212121 added a commit to electrumsv/electrumsv that referenced this issue May 11, 2023
The reason we are doing this is that the signed builds are done on MacOS 13.3
and this introduces linkage against things that are not available for earlier
versions of MacOS. See the following Python development link for details:

python/cpython#97897

The earliest release of Python with the fix is 3.9.15, with the latest 3.9
release at this time being 3.9.16. So we have to move to that, to solve have
this problem solved for us.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.9 only security fixes 3.10 only security fixes 3.11 only security fixes 3.12 bugs and security fixes OS-mac release-blocker type-bug An unexpected behavior, bug, or error type-crash A hard crash of the interpreter, possibly with a core dump
Projects
Development

No branches or pull requests

1 participant