pdb cannot find source code for frozen stdlib modules

[SnoopJ]

Bug report

Context: this issue came out of a discussion in #python on the Libera.chat IRC network, where a user wanted to peek into importlib._bootstrap with pdb while chasing a bug.

pdb is capable of stepping into function calls into frozen modules, but the list command cannot locate the source code necessary to display the source being stepped through.

# repro.py
import importlib._bootstrap

# some function call that we want to step into with pdb
importlib._bootstrap._resolve_name("os", ".", 1)

$ python3 -m pdb repro.py
> /home/snoopjedi/repro.py(2)<module>()
-> import importlib._bootstrap
(Pdb) n
> /home/snoopjedi/repro.py(5)<module>()
-> importlib._bootstrap._resolve_name("os", ".", 1)
(Pdb) s
--Call--
> <frozen importlib._bootstrap>(883)_resolve_name()
(Pdb) l
[EOF]

Note that executing source importlib._bootstrap from the frame that calls into this module does successfully locate the source, but this isn't very useful to a user of pdb.

I believe that bringing the frame's co_filename into agreement with the module's __file__ would fix this issue without changes to pdb (see #89815), but thought it would be good to track the issue with pdb in a separate ticket since that fix is more nuanced and I think I have an interim patch for the debugger.

Your environment

• CPython versions tested on: 3.9.4, 3.11.0b3
• Operating system and architecture: Ubuntu 20.04, x86_64

[jacobtylerwalls]

Likely duplicate of #60115

[SnoopJ]

Likely duplicate of #60115

Definitely related inasmuch as importlib._bootstrap is involved, @exarkun is the user I mentioned in my report. Note that the issue does happen with other frozen modules, though, so the scope of the problem is expanding in 3.11 because of #89183. Here's an example showing the same problem with os.path

# repro2.py
import os.path

os.path.basename("/foo/bar/baz.py")  # implemented in Python

With Python 3.10:

$ python3.10 -m pdb repro2.py 
> /home/snoopjedi/repos/cpython/repro2.py(2)<module>()
-> import os.path
(Pdb) n
> /home/snoopjedi/repos/cpython/repro2.py(4)<module>()
-> os.path.basename("/foo/bar/baz.py")  # implemented in Python
(Pdb) s
--Call--
> /home/snoopjedi/.pyenv/versions/3.10.0/lib/python3.10/posixpath.py(140)basename()
-> def basename(p):
(Pdb) l
135         return p[:0], p
136  
137  
138     # Return the tail (basename) part of a path, same as split(path)[1].
139  
140  -> def basename(p):
141         """Returns the final component of a pathname"""
142         p = os.fspath(p)
143         sep = _get_sep(p)
144         i = p.rfind(sep) + 1
145         return p[i:]

With Python 3.11:

$ python3.11 -m pdb repro2.py 
> /home/snoopjedi/repos/cpython/repro2.py(2)<module>()
-> import os.path
(Pdb) n
> /home/snoopjedi/repos/cpython/repro2.py(4)<module>()
-> os.path.basename("/foo/bar/baz.py")  # implemented in Python
(Pdb) s
--Call--
> <frozen posixpath>(140)basename()
(Pdb) l
[EOF]
(Pdb)

[SnoopJ]

Pinging this issue per the devguide in the hopes of getting review on the PR, which has been open for a little while. I understand if things are too busy for review because of the release issues with 3.11, though!

[ericsnowcurrently]

This should mostly be resolved in 3.10. See issue gh-65935 (and PR gh-28656). We now set __file__ for frozen stdlib modules. Could you give it a try?

[SnoopJ]

This should mostly be resolved in 3.10. See issue gh-65935 (and PR gh-28656). We now set __file__ for frozen stdlib modules. Could you give it a try?

I get the same behavior as in the initial report against the current tip of 3.10 (b60c3d2):

13:29 [snoopjedi@denton ~/repos/cpython (3.10)]
$ ./python -c "import sys; print(sys.version)"
3.10.5+ (remotes/upstream/3.10:b60c3d2f7e, Jul 21 2022, 12:34:09) [GCC 9.4.0]
13:29 [snoopjedi@denton ~/repos/cpython (3.10)]
$ ./python -m pdb repro.py 
> /home/snoopjedi/repos/cpython/repro.py(2)<module>()
-> import importlib._bootstrap
(Pdb) n
> /home/snoopjedi/repos/cpython/repro.py(5)<module>()
-> importlib._bootstrap._resolve_name("os", ".", 1)
(Pdb) s
--Call--
> <frozen importlib._bootstrap>(902)_resolve_name()
(Pdb) l
[EOF]

The __file__ attribute is reported correctly (I believe this has been the case since before I filed this ticket, actually), but pdb is relying on co_filename and #89815 still applies

[ericsnowcurrently]

Ah, thanks for checking. I missed that pdb wasn't using __file__. We should look at making sure co_filename is correct. I thought we were.

[SnoopJ]

Yea, I'm not entirely sure what all goes into fixing the co_filename, but got the impression from your comment on that issue that it may be trickier than "just" patching the problem in pdb, hence this relatively unclever stopgap patch 😅

[ericsnowcurrently]

Yeah, it may be worth adding the stopgap fix in the meantime.

[kumaraditya303]

FYI I am looking into this. I'll probably make deepfreeze generate the correct co_filename but since this would have to be backported the stopgap solutions seems worth it. @ericsnowcurrently ?

[rh314]

I'm adding this here in case anyone has to debug an older version of python and you still need pdb to step through "<frozen ...>".
Adapt it as needed according to the modules/files you need to debug.

This is an example of how to make the source lines available for <frozen importlib._bootstrap> on python3.9:
Early in your script(s), insert this:

import linecache

getline_original = linecache.getline
def getline_patched(filename, lineno, module_globals=None):
    if filename == '<frozen importlib._bootstrap>':
        filename = '/usr/lib/python3.9/importlib/_bootstrap.py'
    return getline_original(filename, lineno, module_globals)

linecache.getline = getline_patched

I also found this, but it didn't work for me; hence the code above.

[SnoopJ]

That workaround looks reasonable to me. I would suggest that users could replace the hard-coded /usr/lib/python3.9/ part with sysconfig.get_path("stdlib") to be slightly more portable across installations and Python versions, but the truly portable fix that works for all frozen files is in the associated PR 😸

[rh314]

I completely agree with you @SnoopJ, the associated PR is indeed the permanent solution. 🏆

In my situation, I had to work on a legacy system, and recompiling/changing the filesystem was not an option available to me at the time. 🥲