pytest discovery error

[stanezeaku]

Type: Bug

error discovering pytest tests

Extension version: 2022.14.0
VS Code version: Code 1.70.2 (Universal) (e4503b30fc78200f846c62cf8091b76ff5547662, 2022-08-16T05:36:37.829Z)
OS version: Darwin arm64 21.5.0
Modes:

System Info

Item	Value
CPUs	Apple M1 (8 x 24)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: disabled_off direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled metal: disabled_off multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok skia_renderer: enabled_on video_decode: enabled video_encode: enabled webgl: enabled webgl2: enabled
Load (avg)	2, 2, 2
Memory (System)	16.00GB (0.21GB free)
Process Argv	--crash-reporter-id 576b9ed6-bcd2-46b5-902b-64a1e2592cda
Screen Reader	no
VM	0%

A/B Experiments

vsliv368cf:30146710
vsreu685:30147344
python383:30185418
vspor879:30202332
vspor708:30202333
vspor363:30204092
vstes627:30244334
vslsvsres303:30308271
pythonvspyl392:30443607
vserr242:30382549
pythontb:30283811
vsjup518:30340749
pythonptprofiler:30281270
vshan820:30294714
vstes263:30335439
pythondataviewer:30285071
vscod805:30301674
binariesv615:30325510
bridge0708:30335490
bridge0723:30353136
cmake_vspar411cf:30557515
vsaa593:30376534
pythonvs932:30410667
cppdebug:30492333
pylanb8912:30545647
vsclangdc:30486549
c4g48928:30535728
hb751961:30553087
dsvsc012cf:30540253
azure-dev_surveyone:30548225
i497e931:30553904

[eleanorjboyd]

Hello! We are currently working on a rewrite of how we do pytest discovery/execution which will fix this issue. Here is the issue which outlines that work #17242 and the currently issue which is being worked on to prototype that functionality #19790.

[th0ger]

@eleanorjboyd Lots of recent bugs are reported in pytest discovery, and you link to a feature-request that should solve this.
For me, pytest discovery also used to work until recently. To me, this feels more like a regression.
Are you aware of any version introducing these errors?

[eleanorjboyd]

@th0ger are you using a conda environment? If so, there could be conda updates that could be affecting your dev environment. In general we have not made any changes to testing recently so there should not be any version updates that would have introduced these errors. Since pytest is also constantly pushing new versions it could be an issue with our integration as pytest makes changes so hopefully this re-write will fix the additional bugs. Thanks!

[th0ger]

@eleanorjboyd Thanks for the heads up, that makes sense. I use pipenv, and will look for recent changes in pytest/pipenv.

[lachaib]

I ran into an issue that may be same as this one, and it's really a pain.

I have a directory A, containing submodules B and C, each containing also tests
A's configuration (in pytest.ini) even though pytest configuration does specify testpaths and norecursedirs

The resolution for me was to port pytest.ini's settings to pyproject.toml as we're handling our dependencies with poetry, but still it drove me mad to find this out.

I hope this comment may help anyone waiting for this issue to be resolved.

[th0ger]

FYI, I always get pytest discovery issues in vscode if coverage reports are enabled in pyproject. Disabling coverage has fixed it each and every time.

[eleanorjboyd]

@th0ger, interesting thank you. You are saying you have to include the setting --no-cov for it to work?

[th0ger]

@eleanorjboyd no, simply removing the following parts in setup.cfg:

# Add here test requirements (semicolon/line-separated)
testing =
    # pytest-cov

[tool:pytest]
addopts =
    # --cov <package>
    # --cov-report term-missing

[arlucio]

I changed

"python.testing.pytestEnabled": true

on .vscode to false and reload VScode, and them before trying to run the tests selected the proper python interpreted by using command pallete on python: select interpreter, and it worked properly.

So was pretty much wrong interpreted selected at VScode at the time I tried to run the tests on the extension (Also, Im using a conda environment)

[eleanorjboyd]

Great glad that worked! We have also just begun rollout for a rewrite to our testing infrastructure. This might be a fix for anyone with ongoing issues in this thread so if you would like to test it and see if this bug is now resolved, here are the steps to do so!

To test you need to be on vscode insiders and then add this setting to your users settings.json "python.experiments.optInto": ["pythonTestAdapter"]. This is currently what is needed but we are in the process of switching all users to the rewrite but are doing so incrementally so if you do not have insiders, watch our release notes to get updated on when it will begin to hit stable. Thanks!

[github-actions]

Because we have not heard back with the information we requested, we are closing this issue for now. If you are able to provide the info later on, then we will be happy to re-open this issue to pick up where we left off.

Happy Coding!

[dgard1981]

Just in case it's useful to anyone, I have pyproject.toml in my test directory, and it seems that VS Code reads only some of this.

As you can see from the example below, I am outputting and LCOV report alongside the default coverage file. Both should be added to a directory called .coverage.

However, test discovery results in an error. I don't know for sure, but it seems as though test discovery is creating the default coverage file a ./.coverage, ignoring the table for [tool.coverage.run]. So every run of test discovery results in the following error.

INTERNALERROR> FileExistsError: [Errno 17] File exists: '.coverage'

To override coverage options set in in [tool.pytest.ini_options], I can add --no-cov to python.testing.pytestArgs in .vscode/settings.json

Example TOML

[tool.pytest.ini_options]
addopts = """
    -p no:cacheprovider \
    --cov proj.assets \
    --cov-report term-missing \
    --cov-report lcov:.coverage/.lcov \
    --cov-fail-under 95"""
enable_assertion_pass_hook = true
python_files = "tests/*_test.py"

[tool.coverage.run]
branch = true
data_file = ".coverage/.coverage"

[eleanorjboyd]

Hi @dgard1981! Could you open another issue? Your problem seems more specific than the above conversation and I would love to discuss more with you about what the issue might be in your case. Thanks

[eleanorjboyd]

Closing again as it has been inactive for a month but please reopen/ open a new issue if this persists. Thanks