You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
While investigating #91961 and specifically #108999, we noticed that test_gdb fails (or used to fail) very frequently on buildbots, or some tests are skipped.
Most of the times, the skips/failures are on clang builds, which often don't have the necessary information to inspect Python frames. Compiling with the right optimization options (or, rather, without optimization) allows gdb to inspect Python frames on clang builds just fine (or almost).
I would like to propose to set an expectation that a --with-pydebug build should be debuggable with the Python extension to gdb in most configurations by default (that is, without additional compiler flags or shenanigans), and attempt to guarantee this as much as possible. This should be possible on basically all platforms where gcc or clang are used, and it's already basically certain with gcc.
Some details of what the proposal entails as far as I understand:
Easy one: don't imply -Og for debug builds on clang. -Og on clang is the same as -O1 (but it could change in the future) and performs certain (which?) optimization passes that don't work well with debug inspection. -fno-inline is not sufficient to prevent that, although some other option might be. Otherwise, defaulting to -O0 will work, but might take longer to compile and generate very inefficient code. If this is acceptable, it could be all there is to it :)
During my testing, I saw one test failed/skipped (test_print_after_up) that had one more python frame than expected, i.e. it worked with one more py-up. I haven't investigated why, so it could be a legitimate case that tests should consider, or a bug, or just an artifact given by different compile options (maybe gcc -Og inlines something and clang -O0 doesn't). In any case, skipping might mask a potential issue that could be fixable. Which brings to point 3.
How to test? test_gdb skips in many cases that are indistinguishable from failures. Knowing that you cannot read a Python frame could mean that the build doesn't include debug information, but it can also mean that the libpython.py extension is making incorrect assumptions. IMO it would be good to have some setups where tests are expected to pass (and ensure buildbots are covering those) and fewer or no tests are allowed to skip. This way test_gdb itself can be used to cover not only that the gdb extension works, but also that (that specific build of) CPython is debuggable. It could be an explicit flag like "this build should be debuggable", exposed via sysconfig1.
Even if the proposal to offer some kind of debuggability-by-default with --with-pydebug is rejected, maybe some work can be done to improve the clang situation.
Has this already been discussed elsewhere?
This is a minor feature, which does not need previous discussion elsewhere
Links to previous discussion of this feature:
No response
Footnotes
test_gdb checks sysconfig for compiler flags to infer whether the build is optimized. This is on a similar track as what I'm suggesting in point 3, but the current behavior is to use this information to skip tests early. Even if the build is not optimized, tests can be skipped anyway, and failures are masked. Besides, the check looks for -Og which works for gcc but not for clang. ↩
The text was updated successfully, but these errors were encountered:
Yeah, clang is the default on macOS and, since a few years, FreeBSD.
For the sake of completeness, GDB still doesn't support aarch64 Darwin. LLDB is the recommended debugger, and it also can be extended with Python scripts, but of course not in a compatible way. https://github.com/malor/cpython-lldb seems to cover that case for now.
gdb only works reliably if Python is built with -O0. I proposed once to use -O0 for --with-pydebug, but the idea was rejected since it makes Python way slower and many developers only hack Python code, not C code.
test_gdb skips tests when it's known that gdb doesn't work reliably on the Python executable. test_gdb does its best to adapt to the Python executable.
In short, there are reasons to keep -Og by default.
Feature or enhancement
Proposal:
While investigating #91961 and specifically #108999, we noticed that test_gdb fails (or used to fail) very frequently on buildbots, or some tests are skipped.
Most of the times, the skips/failures are on clang builds, which often don't have the necessary information to inspect Python frames. Compiling with the right optimization options (or, rather, without optimization) allows gdb to inspect Python frames on clang builds just fine (or almost).
I would like to propose to set an expectation that a
--with-pydebug
build should be debuggable with the Python extension to gdb in most configurations by default (that is, without additional compiler flags or shenanigans), and attempt to guarantee this as much as possible. This should be possible on basically all platforms where gcc or clang are used, and it's already basically certain with gcc.Some details of what the proposal entails as far as I understand:
Easy one: don't imply
-Og
for debug builds on clang.-Og
on clang is the same as-O1
(but it could change in the future) and performs certain (which?) optimization passes that don't work well with debug inspection.-fno-inline
is not sufficient to prevent that, although some other option might be. Otherwise, defaulting to-O0
will work, but might take longer to compile and generate very inefficient code. If this is acceptable, it could be all there is to it :)During my testing, I saw one test failed/skipped (
test_print_after_up
) that had one more python frame than expected, i.e. it worked with one morepy-up
. I haven't investigated why, so it could be a legitimate case that tests should consider, or a bug, or just an artifact given by different compile options (maybe gcc -Og inlines something and clang -O0 doesn't). In any case, skipping might mask a potential issue that could be fixable. Which brings to point 3.How to test? test_gdb skips in many cases that are indistinguishable from failures. Knowing that you cannot read a Python frame could mean that the build doesn't include debug information, but it can also mean that the
libpython.py
extension is making incorrect assumptions. IMO it would be good to have some setups where tests are expected to pass (and ensure buildbots are covering those) and fewer or no tests are allowed to skip. This way test_gdb itself can be used to cover not only that the gdb extension works, but also that (that specific build of) CPython is debuggable. It could be an explicit flag like "this build should be debuggable", exposed viasysconfig
1.Even if the proposal to offer some kind of debuggability-by-default with
--with-pydebug
is rejected, maybe some work can be done to improve the clang situation.Has this already been discussed elsewhere?
This is a minor feature, which does not need previous discussion elsewhere
Links to previous discussion of this feature:
No response
Footnotes
test_gdb checks sysconfig for compiler flags to infer whether the build is optimized. This is on a similar track as what I'm suggesting in point 3, but the current behavior is to use this information to skip tests early. Even if the build is not optimized, tests can be skipped anyway, and failures are masked. Besides, the check looks for
-Og
which works for gcc but not for clang. ↩The text was updated successfully, but these errors were encountered: