Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Have debug builds support gdb extension by default #109270

Closed
sorcio opened this issue Sep 11, 2023 · 3 comments
Closed

Have debug builds support gdb extension by default #109270

sorcio opened this issue Sep 11, 2023 · 3 comments
Labels
build The build process and cross-build OS-mac type-feature A feature request or enhancement

Comments

@sorcio
Copy link
Contributor

sorcio commented Sep 11, 2023

Feature or enhancement

Proposal:

While investigating #91961 and specifically #108999, we noticed that test_gdb fails (or used to fail) very frequently on buildbots, or some tests are skipped.

Most of the times, the skips/failures are on clang builds, which often don't have the necessary information to inspect Python frames. Compiling with the right optimization options (or, rather, without optimization) allows gdb to inspect Python frames on clang builds just fine (or almost).

I would like to propose to set an expectation that a --with-pydebug build should be debuggable with the Python extension to gdb in most configurations by default (that is, without additional compiler flags or shenanigans), and attempt to guarantee this as much as possible. This should be possible on basically all platforms where gcc or clang are used, and it's already basically certain with gcc.


Some details of what the proposal entails as far as I understand:

  1. Easy one: don't imply -Og for debug builds on clang. -Og on clang is the same as -O1 (but it could change in the future) and performs certain (which?) optimization passes that don't work well with debug inspection. -fno-inline is not sufficient to prevent that, although some other option might be. Otherwise, defaulting to -O0 will work, but might take longer to compile and generate very inefficient code. If this is acceptable, it could be all there is to it :)

  2. During my testing, I saw one test failed/skipped (test_print_after_up) that had one more python frame than expected, i.e. it worked with one more py-up. I haven't investigated why, so it could be a legitimate case that tests should consider, or a bug, or just an artifact given by different compile options (maybe gcc -Og inlines something and clang -O0 doesn't). In any case, skipping might mask a potential issue that could be fixable. Which brings to point 3.

  3. How to test? test_gdb skips in many cases that are indistinguishable from failures. Knowing that you cannot read a Python frame could mean that the build doesn't include debug information, but it can also mean that the libpython.py extension is making incorrect assumptions. IMO it would be good to have some setups where tests are expected to pass (and ensure buildbots are covering those) and fewer or no tests are allowed to skip. This way test_gdb itself can be used to cover not only that the gdb extension works, but also that (that specific build of) CPython is debuggable. It could be an explicit flag like "this build should be debuggable", exposed via sysconfig1.


Even if the proposal to offer some kind of debuggability-by-default with --with-pydebug is rejected, maybe some work can be done to improve the clang situation.

Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

No response

Footnotes

  1. test_gdb checks sysconfig for compiler flags to infer whether the build is optimized. This is on a similar track as what I'm suggesting in point 3, but the current behavior is to use this information to skip tests early. Even if the build is not optimized, tests can be skipped anyway, and failures are masked. Besides, the check looks for -Og which works for gcc but not for clang.

@sorcio sorcio added the type-feature A feature request or enhancement label Sep 11, 2023
@terryjreedy terryjreedy added build The build process and cross-build OS-mac labels Sep 11, 2023
@terryjreedy
Copy link
Member

I added OS-mac since I believe that clang is mostly used for Python on macOS, though not only there.

@sorcio
Copy link
Contributor Author

sorcio commented Sep 12, 2023

Yeah, clang is the default on macOS and, since a few years, FreeBSD.

For the sake of completeness, GDB still doesn't support aarch64 Darwin. LLDB is the recommended debugger, and it also can be extended with Python scripts, but of course not in a compatible way. https://github.com/malor/cpython-lldb seems to cover that case for now.

@vstinner
Copy link
Member

vstinner commented Sep 2, 2024

gdb only works reliably if Python is built with -O0. I proposed once to use -O0 for --with-pydebug, but the idea was rejected since it makes Python way slower and many developers only hack Python code, not C code.

test_gdb skips tests when it's known that gdb doesn't work reliably on the Python executable. test_gdb does its best to adapt to the Python executable.

In short, there are reasons to keep -Og by default.

In Fedora/RHEL, the python-debug executable is built with -O0 to be able to use gdb: https://developers.redhat.com/articles/2021/09/08/debugging-python-c-extensions-gdb

I close the issue.

@vstinner vstinner closed this as completed Sep 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build The build process and cross-build OS-mac type-feature A feature request or enhancement
Projects
None yet
Development

No branches or pull requests

3 participants