Support for pytest plugins that work with non-python files #16742

ssbarnea · 2021-07-21T08:31:34Z

Environment data

VS Code version: 1.58.2
Extension version (available under the Extensions sidebar): v2021.7.1050252941
OS and version: MacOS 11.4
Python version (& distribution if applicable, e.g. Anaconda): 3.9
Type of virtual environment used (N/A | venv | virtualenv | conda | ...): pyenv
Relevant/affected Python packages and their versions: XXX
Relevant/affected Python-related VS Code extensions and their versions: XXX
Value of the python.languageServer setting: XXX

[NOTE: If you suspect that your issue is related to the Microsoft Python Language Server (python.languageServer: 'Microsoft'), please download our new language server Pylance from the VS Code marketplace to see if that fixes your issue]

Expected behaviour

Test discovery to succeed, as it does also suceeds with pytest --collect-only

Actual behaviour

Popup reporting Test discovery error, see log below for details.

Steps to reproduce:

Open https://github.com/ansible-community/ansible-compat repository inside vscode.

Logs

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

The Python extension has run into an unexpected situation
while processing a pytest node during test discovery.  Please
Please open an issue at:
  https://github.com/microsoft/vscode-python/issues
and paste the following output there.

nodeid: README.md::ansible-compat::using-ansible-runtime::line_8::test_runtime
kind: ('function', False)
class: Function
name: test_runtime
fspath: /Users/ssbarnea/c/a/ansible-compat/README.md
location: ('README.md', 2, 'test_runtime')
function: <function test_runtime at 0x10861a430>
markers: []
user_properties: []
attrnames: ['_ALLOW_MARKERS', '__annotations__', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__slots__', '__str__', '__subclasshook__', '__weakref__', '_fixtureinfo', '_getobj', '_initrequest', '_nodeid', '_obj', '_prunetraceback', '_pyfuncitem', '_report_sections', '_repr_failure_py', '_request', '_store', 'add_marker', 'add_report_section', 'addfinalizer', 'cls', 'config', 'extra_keyword_matches', 'fixturenames', 'from_parent', 'fspath', 'funcargs', 'function', 'get_closest_marker', 'getmodpath', 'getparent', 'ihook', 'instance', 'iter_markers', 'iter_markers_with_node', 'keywords', 'listchain', 'listextrakeywords', 'listnames', 'location', 'module', 'name', 'nextitem', 'nodeid', 'obj', 'originalname', 'own_markers', 'parent', 'reportinfo', 'repr_failure', 'runtest', 'session', 'setup', 'teardown', 'user_properties', 'warn']

extra info:
fullname: test_runtime
testfunc: ansible-compat.using-ansible-runtime.line_8.test_runtime
parameterized:

traceback:
  File "/Users/ssbarnea/.vscode/extensions/ms-python.python-2021.7.1050252941/pythonFiles/testing_tools/run_adapter.py", line 22, in <module>
    main(tool, cmd, subargs, toolargs)
  File "/Users/ssbarnea/.vscode/extensions/ms-python.python-2021.7.1050252941/pythonFiles/testing_tools/adapter/__main__.py", line 100, in main
    parents, result = run(toolargs, **subargs)
  File "/Users/ssbarnea/.vscode/extensions/ms-python.python-2021.7.1050252941/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 30, in discover
    ec = _pytest_main(pytestargs, [_plugin])
  File "/Users/ssbarnea/.pyenv/versions/3.9.5/lib/python3.9/site-packages/_pytest/config/__init__.py", line 162, in main
    ret: Union[ExitCode, int] = config.hook.pytest_cmdline_main(
  File "/Users/ssbarnea/.local/lib/python3.9/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/Users/ssbarnea/.local/lib/python3.9/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/Users/ssbarnea/.local/lib/python3.9/site-packages/pluggy/manager.py", line 84, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/Users/ssbarnea/.local/lib/python3.9/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/Users/ssbarnea/.pyenv/versions/3.9.5/lib/python3.9/site-packages/_pytest/main.py", line 316, in pytest_cmdline_main
    return wrap_session(config, _main)
  File "/Users/ssbarnea/.pyenv/versions/3.9.5/lib/python3.9/site-packages/_pytest/main.py", line 269, in wrap_session
    session.exitstatus = doit(config, session) or 0
  File "/Users/ssbarnea/.pyenv/versions/3.9.5/lib/python3.9/site-packages/_pytest/main.py", line 322, in _main
    config.hook.pytest_collection(session=session)
  File "/Users/ssbarnea/.local/lib/python3.9/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/Users/ssbarnea/.local/lib/python3.9/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/Users/ssbarnea/.local/lib/python3.9/site-packages/pluggy/manager.py", line 84, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/Users/ssbarnea/.local/lib/python3.9/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/Users/ssbarnea/.pyenv/versions/3.9.5/lib/python3.9/site-packages/_pytest/main.py", line 333, in pytest_collection
    session.perform_collect()
  File "/Users/ssbarnea/.pyenv/versions/3.9.5/lib/python3.9/site-packages/_pytest/main.py", line 641, in perform_collect
    hook.pytest_collection_finish(session=self)
  File "/Users/ssbarnea/.local/lib/python3.9/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/Users/ssbarnea/.local/lib/python3.9/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/Users/ssbarnea/.local/lib/python3.9/site-packages/pluggy/manager.py", line 84, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/Users/ssbarnea/.local/lib/python3.9/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/Users/ssbarnea/.vscode/extensions/ms-python.python-2021.7.1050252941/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 107, in pytest_collection_finish
    test, parents = self.parse_item(item)
  File "/Users/ssbarnea/.vscode/extensions/ms-python.python-2021.7.1050252941/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 77, in parse_item
    return parse_item(item)
  File "/Users/ssbarnea/.vscode/extensions/ms-python.python-2021.7.1050252941/pythonFiles/testing_tools/adapter/pytest/_pytest_item.py", line 172, in parse_item
    raise should_never_reach_here(
  File "/Users/ssbarnea/.vscode/extensions/ms-python.python-2021.7.1050252941/pythonFiles/testing_tools/adapter/pytest/_pytest_item.py", line 132, in should_never_reach_here
    traceback.print_stack()
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/Users/ssbarnea/.pyenv/versions/3.9.5/lib/python3.9/site-packages/_pytest/main.py", line 637, in perform_collect
INTERNALERROR>     hook.pytest_collection_modifyitems(
INTERNALERROR>   File "/Users/ssbarnea/.local/lib/python3.9/site-packages/pluggy/hooks.py", line 286, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/Users/ssbarnea/.local/lib/python3.9/site-packages/pluggy/manager.py", line 93, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/Users/ssbarnea/.local/lib/python3.9/site-packages/pluggy/manager.py", line 84, in <lambda>
INTERNALERROR>     self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
INTERNALERROR>   File "/Users/ssbarnea/.local/lib/python3.9/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/Users/ssbarnea/.local/lib/python3.9/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/Users/ssbarnea/.local/lib/python3.9/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/Users/ssbarnea/.vscode/extensions/ms-python.python-2021.7.1050252941/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 92, in pytest_collection_modifyitems
INTERNALERROR>     test, parents = self.parse_item(item)
INTERNALERROR>   File "/Users/ssbarnea/.vscode/extensions/ms-python.python-2021.7.1050252941/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 77, in parse_item
INTERNALERROR>     return parse_item(item)
INTERNALERROR>   File "/Users/ssbarnea/.vscode/extensions/ms-python.python-2021.7.1050252941/pythonFiles/testing_tools/adapter/pytest/_pytest_item.py", line 172, in parse_item
INTERNALERROR>     raise should_never_reach_here(
INTERNALERROR> NotImplementedError: Unexpected pytest node (see printed output).
INTERNALERROR> 
INTERNALERROR> During handling of the above exception, another exception occurred:
INTERNALERROR> 
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/Users/ssbarnea/.pyenv/versions/3.9.5/lib/python3.9/site-packages/_pytest/main.py", line 269, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/Users/ssbarnea/.pyenv/versions/3.9.5/lib/python3.9/site-packages/_pytest/main.py", line 322, in _main
INTERNALERROR>     config.hook.pytest_collection(session=session)
INTERNALERROR>   File "/Users/ssbarnea/.local/lib/python3.9/site-packages/pluggy/hooks.py", line 286, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/Users/ssbarnea/.local/lib/python3.9/site-packages/pluggy/manager.py", line 93, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/Users/ssbarnea/.local/lib/python3.9/site-packages/pluggy/manager.py", line 84, in <lambda>
INTERNALERROR>     self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
INTERNALERROR>   File "/Users/ssbarnea/.local/lib/python3.9/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/Users/ssbarnea/.local/lib/python3.9/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/Users/ssbarnea/.local/lib/python3.9/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/Users/ssbarnea/.pyenv/versions/3.9.5/lib/python3.9/site-packages/_pytest/main.py", line 333, in pytest_collection
INTERNALERROR>     session.perform_collect()
INTERNALERROR>   File "/Users/ssbarnea/.pyenv/versions/3.9.5/lib/python3.9/site-packages/_pytest/main.py", line 641, in perform_collect
INTERNALERROR>     hook.pytest_collection_finish(session=self)
INTERNALERROR>   File "/Users/ssbarnea/.local/lib/python3.9/site-packages/pluggy/hooks.py", line 286, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/Users/ssbarnea/.local/lib/python3.9/site-packages/pluggy/manager.py", line 93, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/Users/ssbarnea/.local/lib/python3.9/site-packages/pluggy/manager.py", line 84, in <lambda>
INTERNALERROR>     self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
INTERNALERROR>   File "/Users/ssbarnea/.local/lib/python3.9/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/Users/ssbarnea/.local/lib/python3.9/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/Users/ssbarnea/.local/lib/python3.9/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/Users/ssbarnea/.vscode/extensions/ms-python.python-2021.7.1050252941/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 107, in pytest_collection_finish
INTERNALERROR>     test, parents = self.parse_item(item)
INTERNALERROR>   File "/Users/ssbarnea/.vscode/extensions/ms-python.python-2021.7.1050252941/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 77, in parse_item
INTERNALERROR>     return parse_item(item)
INTERNALERROR>   File "/Users/ssbarnea/.vscode/extensions/ms-python.python-2021.7.1050252941/pythonFiles/testing_tools/adapter/pytest/_pytest_item.py", line 172, in parse_item
INTERNALERROR>     raise should_never_reach_here(
INTERNALERROR> NotImplementedError: Unexpected pytest node (see printed output).

�[32m========================= �[0m�[32m32 tests collected�[0m�[32m in 0.10s�[0m�[32m ==========================�[0m

Traceback (most recent call last):
  File "/Users/ssbarnea/.vscode/extensions/ms-python.python-2021.7.1050252941/pythonFiles/testing_tools/run_adapter.py", line 22, in <module>
    main(tool, cmd, subargs, toolargs)
  File "/Users/ssbarnea/.vscode/extensions/ms-python.python-2021.7.1050252941/pythonFiles/testing_tools/adapter/__main__.py", line 100, in main
    parents, result = run(toolargs, **subargs)
  File "/Users/ssbarnea/.vscode/extensions/ms-python.python-2021.7.1050252941/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 44, in discover
    raise Exception("pytest discovery failed (exit code {})".format(ec))
Exception: pytest discovery failed (exit code 3)```

</p>
</details>

The text was updated successfully, but these errors were encountered:

karthiknadig · 2021-07-22T00:44:24Z

With this there are 3 separate problems:

With detections: We have a limitation in our current testing adapter that where it fails to handle non-python files (as the error suggests). We have plans on addressing these types of problems by updating the test adapter soon.
With debugging: Debugging kind of works but breaks in the wrong place: /cc @fabioz
The extension itself won't allow you to set breakpoint in the markdown. This might be more than just adding markdowns to the list of files extension support. Since we don't want python extension to activate on all markdown files.

/cc @luabud

DetachHead · 2023-10-25T01:44:00Z

non-python files seem to work now in the latest version of the python extension (v2023.18.0)

eleanorjboyd · 2023-10-30T17:30:08Z

Thank you @DetachHead! Closing this issue

ssbarnea added triage-needed Needs assignment to the proper sub-team bug Issue identified by VS Code Team member as probable bug labels Jul 21, 2021

karthiknadig self-assigned this Jul 21, 2021

karthiknadig changed the title ~~pytest: test discovery error~~ Support for python tests detected by pytest in non-python files Jul 22, 2021

karthiknadig changed the title ~~Support for python tests detected by pytest in non-python files~~ Support for pytest pluggins that work with non-python files Jul 22, 2021

karthiknadig removed triage-needed Needs assignment to the proper sub-team bug Issue identified by VS Code Team member as probable bug labels Jul 22, 2021

karthiknadig removed their assignment Jul 22, 2021

karthiknadig added area-testing meta Issue that is tracking an overall project needs proposal Need to make some design decisions bug Issue identified by VS Code Team member as probable bug labels Jul 22, 2021

brettcannon changed the title ~~Support for pytest pluggins that work with non-python files~~ Support for pytest plugins that work with non-python files Jul 22, 2021

pranavrajpal mentioned this issue Dec 23, 2021

Add initial support for pytest plugins #18217

Closed

DetachHead mentioned this issue Sep 12, 2023

option to disable tests from appearing in vscode's test view robocorp/robotframework-lsp#976

Closed

eleanorjboyd closed this as completed Oct 30, 2023

github-actions bot removed the needs proposal Need to make some design decisions label Oct 30, 2023

github-actions bot locked as resolved and limited conversation to collaborators Nov 30, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support for pytest plugins that work with non-python files #16742

Support for pytest plugins that work with non-python files #16742

ssbarnea commented Jul 21, 2021

karthiknadig commented Jul 22, 2021

DetachHead commented Oct 25, 2023

eleanorjboyd commented Oct 30, 2023

Support for pytest plugins that work with non-python files #16742

Support for pytest plugins that work with non-python files #16742

Comments

ssbarnea commented Jul 21, 2021

Environment data

Expected behaviour

Actual behaviour

Steps to reproduce:

Logs

karthiknadig commented Jul 22, 2021

DetachHead commented Oct 25, 2023

eleanorjboyd commented Oct 30, 2023