ipdb.set_trace() breaks inspect

[sthompson0]

If a script attempts to use inspect.getfile() on a class that was defined in a script file after set_trace() is called an exception is raised. Here's the traceback and the demo script.

I discovered that ipython is replacing sys.modules['main'] at some point (didn't get as far as pinpointing an exact line). This object ipython puts in there doesn't have attributes inspect is looking for which leads to the exception. It seems ipython could clean up after itself when set_trace() exits or copy those attributes to the replaced item.

I'd really like to get a work around so I can make a local modification until this fix goes out.

Version info
Python 2.7.10

pip freeze | grep ipython
ipython==3.2.1
pip freeze | grep ipdb
ipdb==0.8.1

Trackback

> /Users/username/folder/test.py(10)test()
      9         ipdb.set_trace()
---> 10         inspect.getfile(self.__class__)
     11

ipdb> c
E
======================================================================
ERROR: test (__main__.Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_test.py", line 10, in test
    inspect.getfile(self.__class__)
  File "/Users/username/homebrew/Cellar/python/2.7.10_2/Python.framework/Versions/2.7/lib/python2.7/inspect.py", line 408, in getfile
    raise TypeError('{!r} is a built-in class'.format(object))
TypeError: <module '__main__' (built-in)> is a built-in class

----------------------------------------------------------------------
Ran 1 test in 1.268s

FAILED (errors=1)

Script for reproduction

import inspect
import unittest
import ipdb


class Test(unittest.TestCase):
    def test(self):
        inspect.getfile(self.__class__)
        ipdb.set_trace()
        inspect.getfile(self.__class__)

def main():
    unittest.main()

if __name__ == '__main__':
    main()

[takluyver]

I can't work out what would be replacing the __main__ module in that case. The

[takluyver]

Sorry, hit the wrong button.

I can't work out what would be replacing the main module in that case. The %run magic can store and reset it, but ipdb shouldn't be calling that. Neither ipdb or pdb look like the culprit.

[sthompson0]

Step through the code in the example and you'll find it. I also found a work around that fixes the issue for me. It's pretty hacky and not exhaustive. I copy some of the attributes that inspect is looking for from the original sys.modules['main'] module into the one ipython overrides it with. The true fix is probably to restore sys.modules['main'] every time ipdb.set_trace() completes.

Here's the diff.

diff -C 5 interactiveshell.py interactiveshell-fixed.py
*** interactiveshell.py 2015-07-28 16:11:44.000000000 -0700
--- interactiveshell-fixed.py   2015-07-28 16:13:29.000000000 -0700
***************
*** 1168,1177 ****
--- 1168,1183 ----
          # shouldn't overtake the execution environment of the script they're
          # embedded in).

          # This is overridden in the InteractiveShellEmbed subclass to a no-op.
          main_name = self.user_module.__name__
+         orig_mod = sys.modules[main_name]
+         for attr_name in ['__file__', '__module__', '__package__']:
+             if hasattr(orig_mod, attr_name):
+                 # Make sure we aren't overriding something...
+                 assert not getattr(self.user_module, attr_name, False)
+                 setattr(self.user_module, attr_name, getattr(orig_mod, attr_name))
          sys.modules[main_name] = self.user_module

      def init_user_ns(self):
          """Initialize all user-visible namespaces to their minimum defaults.

[takluyver]

Thanks, I see it. I'll try to work out a good fix for that.

[sthompson0]

Any updates on this? It's been two weeks.

Thanks!

[takluyver]

Sorry, fell off the radar. I've just dug into it, and I think the easiest sensible fix would be in ipdb. I fixed it in my local copy by making the set_trace() function look like this:

def set_trace(frame=None):
    update_stdout()
    wrap_sys_excepthook()
    if frame is None:
        frame = sys._getframe().f_back
    p = Pdb(def_colors)
    p.set_trace(frame)
    p.shell.restore_sys_module_state()

The last line is the key bit - that will put sys.modules['__main__'] back to its previous value.

Perhaps we shouldn't be replacing the __main__ module when InteractiveShell is merely instantiated, but that would require much more delicate changes to address.

[sthompson0]

When's the next ipython release?

[minrk]

There isn't a schedule. When we do a major release, which we just did, we typically do a bugfix release as we accumulate new issues about a month later, which would be early September.

[sthompson0]

What's the status of this? Been waiting for a while.

[minrk]

@sthompson0 sorry, the repo explosion made IPython 4.1 take (much) longer than usual. We shipped 4.1 yesterday.

[sthompson0]

Did this fix make it in, or did it get deferred?

[takluyver]

I don't think we have made any change in IPython for this. As mentioned above, I think the easiest fix is in ipdb, which is a separate project which we don't control.

[sthompson0]

Oh, okay I didn't realize this was two different projects or I would have filed the bug over there. Thanks.

[takluyver]

No worries. It's a bit confusing, because most of the actual debugger machinery ipdb uses is inside IPython, but ipdb exists as a separate project to provide an API to use it outside IPython. So it's not obvious where to file bugs.

[sthompson0]

For future reference please point this out earlier. 👍

gotcha/ipdb#85