await keyword support in Debug Console

[brettcannon]

Discussed in microsoft/vscode-python#15842

^{Originally posted by jamesstidard April 2, 2021}
It would be nice to be able to use the await keyword in the Debug Console when sitting on a breakpoint.

The asyncio module actually has a REPL that supports this in the latest versions of Python:

$ python -m asyncio
asyncio REPL 3.8.5 (default, Aug 18 2020, 11:12:18)
[Clang 11.0.3 (clang-1103.0.32.62)] on darwin
Use "await" directly instead of "asyncio.run()".
Type "help", "copyright", "credits" or "license" for more information.
>>> import asyncio
>>> await asyncio.sleep(5, result="done")
'done'

If it's possible to replicate this in the Debug Console, I think this would be a very welcome feature for the python heads that live in asyncio.

If it's all part of the same system, it would also be nice to await watched variables in the Run & Debug Panel. This area:

[brettcannon]

Thanks for the feature request! We are going to give the community 60 days from when this issue was created to provide 5 👍 upvotes on the opening comment to gauge general interest in this idea. If there's enough upvotes then we will consider this feature request in our future planning. If there's unfortunately not enough upvotes then we will close this issue.

[brettcannon]

Thank you to everyone who upvoted this issue! Since the community showed interest in this feature request we will leave this issue open as something to consider implementing at some point in the future.

We do encourage people to continue 👍 this issue as it helps us prioritize our work based on what the community seems to want the most.

[septatrix]

Regarding the implementation it might be worth checking out what IPython did to archive top-level awaits in the REPL: https://ipython.readthedocs.io/en/stable/interactive/autoawait.html#internals

[luabud]

@int19h should we transfer this to debugpy?

[SkyeChen-28]

I see that this enhancement is being worked on. But I was wondering what other people do to help debug their code. I have a lot of variables who's values that I can only see if I await them inside a function and that's extremely inconvenient for me as I'll have to re-run my current .py file. Is there a better way to debug variables that need to be retrieved from awaiting coroutines?

[fabioz]

I was thinking how to implement this.

The major issue is that the tracing mode in which the debugger pauses is a sync function and thus we need a way to call async code from sync code (which in general is a no-no but is a requisite for this to work at all because we'll be paused at a breakpoint inside of the tracing from a call which went async -> sync).

There are 2 possible approaches I thought of here:

1. Evaluate each of the async calls in a new thread.

• Con: evaluating the code in a thread can mean that some evaluations won't work (some frameworks such as qt consider the UI thread as special and won't work properly, checks for the current thread won't match the thread paused, etc.).
• Con: this may need special support for each asyncio framework (i.e.: as we need to create a new loop in a new thread we may need to do it differently for asyncio, trio, curio).

2. Do a custom event loop for doing the async evaluation in a sync fashion.

• Pro: should work with any asyncio framework by synchronously calling the using the __await__ protocol.

I'll try a bit with approach 2 to see how far I get there.

[fabioz]

I've tried approach 2 and the basics work (i.e.: for a simple coroutine with a sleep) , but after checking the code I'm not sure this is reasonable. The basics of the AbstractEventLoop are implemented, but there are so many functions which I wasn't expecting in AbstractEventLoop such as create_connection, create_server, and lots of other things related to network, subprocess, etc.

Also, in the end there can be only 1 loop active, so, the only way I got it to work was calling set_event_loop(new_loop) as well as _set_running_loop(new_loop) and then restoring the original loop afterward (which means that if one of those functions I didn't implement was called by someone it'd fail).

Trying to create a new native loop on top of the same thread and set it with set_event_loop/_set_running_loop also didn't yield good results.

Given that, I think going with option 1 (evaluating each new async call in a new thread) will be less brittle (even with the known cons) -- only for asyncio for now.

[septatrix]

Might also be interesting to take a look at this PyCharm plugin: https://github.com/uriyyo/pycharm-evaluate-async-code

[fabioz]

@septatrix thank you for the reference... It seems it uses the same loop and monkey-patches many things to make it reentrant (https://github.com/uriyyo/pycharm-evaluate-async-code/blob/master/src/main/java/com/uriyyo/evaluate_async_code/AsyncPyDebugUtils.kt#L80) -- i.e.: with those patches run_until_complete can be called inside of run_until_complete, so, it doesn't need to create a new loop, it reuses the main loop.

I'm a bit wary of all the monkey-patching done to make that possible.

Also, from the implementation I get the impression that if you have other things scheduled to run in the original loop it could run those things, which is not something we want (so, I get the impression that this approach would be a no-go due to that as we don't want to run unrelated scheduled tasks while we wait for the task we're evaluating -- note that I haven't really tested it, but from code-reviewing that code I'd say this isn't an ideal approach).

So, I guess for now I'm still going with option 1.

p.s.: @septatrix really thanks for the link, it's always nice to know how other implementations work ;)