Auto-attach to subprocess with .zip target (named as .exe)

[nedsociety]

Before creating a new issue, please check the FAQ to see if your question is answered there.

Environment data

• debugpy version: 1.6.0
• OS and version: 21H2
• Python version (& distribution if applicable, e.g. Anaconda): Python 10.0.0
• Using VS Code or Visual Studio: VSCode

Actual behavior

In the debugging session, running a subprocess with python <shim> (where shim is an exe file in Scripts/ directory) fails with following message:

Traceback (most recent call last):
  File "d:\programs\python3-amd64\lib\tokenize.py", line 332, in find_cookie
    line_string = line.decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x90 in position 2: invalid start byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\Users\ned\.vscode\extensions\ms-python.python-2022.6.2\pythonFiles\lib\python\debugpy\_vendored\pydevd\pydevd.py", line 3385, in <module>
    main()
  File "c:\Users\ned\.vscode\extensions\ms-python.python-2022.6.2\pythonFiles\lib\python\debugpy\_vendored\pydevd\pydevd.py", line 3378, in main
    globals = debugger.run(setup['file'], None, None, is_module)
  File "c:\Users\ned\.vscode\extensions\ms-python.python-2022.6.2\pythonFiles\lib\python\debugpy\_vendored\pydevd\pydevd.py", line 2446, in run
    return self._exec(is_module, entry_point_fn, module_name, file, globals, locals)
  File "c:\Users\ned\.vscode\extensions\ms-python.python-2022.6.2\pythonFiles\lib\python\debugpy\_vendored\pydevd\pydevd.py", line 2453, in _exec
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "c:\Users\ned\.vscode\extensions\ms-python.python-2022.6.2\pythonFiles\lib\python\debugpy\_vendored\pydevd\_pydev_bundle\_pydev_execfile.py", line 14, in execfile
    stream = tokenize.open(file)  # @UndefinedVariable
  File "d:\programs\python3-amd64\lib\tokenize.py", line 396, in open
    encoding, lines = detect_encoding(buffer.readline)
  File "d:\programs\python3-amd64\lib\tokenize.py", line 373, in detect_encoding
    encoding = find_cookie(first)
  File "d:\programs\python3-amd64\lib\tokenize.py", line 337, in find_cookie
    raise SyntaxError(msg)
SyntaxError: invalid or missing encoding declaration for 'D:\\<fakepath>\\.venv\\Scripts\\xonsh.exe'

Expected behavior

At minimum, it should run the shim without error. Preferably, it should run with subprocess debugging.

Steps to reproduce:

1. Create a virtualenv with xonsh installed.

> poetry init -n
> poetry add xonsh prompt-toolkit

2. Locate the virtualenv in VSCode then start debugging with following launch.json:

{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "name": "xonsh",
            "type": "python",
            "request": "launch",
            "module": "xonsh",
            "justMyCode": true
        }
    ]
}

3. In the debugged xonsh shell, try: $ python @($(which xonsh.exe).strip())

[int19h]

To work around this, try "subProcess": false in your debug config.

[nedsociety]

Workaround works. Though I'd love to see zipfile execution being supported by pydevd.

[int19h]

Ah, I didn't realize that this is how the stub works!

So far as I can tell from Python docs, this would have worked if we used runpy.run_path here. @fabioz, is this implementation of execfile basically just legacy code for old Python 2 versions at this point?

[fabioz]

Well, execfile itself as defined is not legacy code -- there are other uses for it which (maybe runpy.run_path could cover those too, but I believe it's not a drop-in replacement in general).

In this case,

python -c "import runpy;runpy.run_path(r'C:\bin\Miniconda\envs\py_38_tests\Scripts\xonsh.exe', run_name='__main__')"

does seem to work for me, so, I guess that changing the execfile to runpy.run_path does make sense here. I'll take a look on changing that.