Cannot discover with pytest in 2020-01

[CaselIT]

Environment data

• VS Code version: 1.41.1
• Extension version (available under the Extensions sidebar): 2020.1.57204 (6 January 2020)
• OS and version: windows 10
• Python version (& distribution if applicable, e.g. Anaconda): python 3.7.5
• Type of virtual environment used (N/A | venv | virtualenv | conda | ...): conda
• Relevant/affected Python packages and their versions: pytest version 5.3.2
• Jedi or Language Server? (i.e. what is "python.jediEnabled" set to; more info How to update the language server to the latest stable version #3977): Language Server

Expected behaviour

The test are discovered

Actual behaviour

Since updating to the version 2020.1 I have the following two problems when discovering the tests with pytest:

• The base environment does not have pytest installed, using a different environment with pytest installed I get the following output:

python c:\Users\Federico\.vscode\extensions\ms-python.python-2020.1.57204\pythonFiles\testing_tools\run_adapter.py discover pytest -- --rootdir c:\Users\Federico\Documents\GitHub\pytest-error -s --cache-clear
Test Discovery failed: 
Error: The system cannot find the path specified.

Traceback (most recent call last):

  File "c:\Users\Federico\.vscode\extensions\ms-python.python-2020.1.57204\pythonFiles\testing_tools\run_adapter.py", line 13, in <module>

    from testing_tools.adapter.__main__ import parse_args, main

  File "c:\Users\Federico\.vscode\extensions\ms-python.python-2020.1.57204\pythonFiles\testing_tools\adapter\__main__.py", line 9, in <module>

    from . import pytest, report

  File "c:\Users\Federico\.vscode\extensions\ms-python.python-2020.1.57204\pythonFiles\testing_tools\adapter\pytest\__init__.py", line 7, in <module>

    from ._discovery import discover

  File "c:\Users\Federico\.vscode\extensions\ms-python.python-2020.1.57204\pythonFiles\testing_tools\adapter\pytest\_discovery.py", line 8, in <module>

    import pytest

ModuleNotFoundError: No module named 'pytest'

• with pytest installed in the base environment (using the base envornment or any other with pytest) I get the following error

python c:\Users\Federico\.vscode\extensions\ms-python.python-2020.1.57204\pythonFiles\testing_tools\run_adapter.py discover pytest -- --rootdir c:\Users\Federico\Documents\GitHub\pytest-error -s --cache-clear
Test Discovery failed: 
Error: The system cannot find the path specified.

Steps to reproduce:

1. create a base environment, do no install pytest
2. create a different environmnet, install pytest
3. Clone the repo https://github.com/CaselIT/vscode-python-pytest-error
4. open in vscode and select the environment with pytest installed
5. should get the first error
6. try installing pytest in the base environment
7. should get the second error

[CaselIT]

I've installed the 2020.1 on another pc (also windows 10 with the same python configuration) and I have the same problem.
Let me know if you need other information

[phloose]

I've got a similar problem with windows 10 and pytest! In my case it cannot find the package for which the tests are, The same package configuration worked without any problems with extension version 2019.11.50794. I think it has to do with the environment from which the python command that runs the discovery script is invoked, but this is only a guess. Maybe i find time to dig deeper in the source code to see what changed between this and the prior release with regard to test discovery.

Worth to mention that this needs to be a windows specific problem because i tried it in WSL and there it works, tests are being discovered as expected. When reverting the extension to the prior version 2019.11.50794 it works again in windows 10.

[phloose]

I ran a git bisect from the recent release tag to the previous release and found that test discovery works until commit b543408 ... maybe this helps sorting out this bug

EDIT:
just saw that you are working on it in #9490 👍

[karrtikr]

@phloose Your issue might be different from this.

@CaselIT Unfortunately, I am not able to reproduce this issue with the steps you mentioned. I was using conda 4.8.1, my base environment didn't have pytest installed, but my environment within it had it installed. I selected the environment but was able to discover tests just fine.

The first issue you face is an upstream conda issue conda/conda#9571. For some reason it didn't happen in my machine.

Regarding Error: The system cannot find the path specified., I read it might happen when you have invalid paths in your PATH environment variable e.g. directories which don't exist anymore. You please check/paste the variable if you can.

• Does reverting to an older version of the extension solves the issue?
• Which conda version are you using?
• Please paste the Python output panel as mentioned in the issue template for both the cases.

If conda run is the problem, this issue should be fixed with #9490. Thanks for providing such detailed issue btw.

[CaselIT]

Thanks for the feedback. I'll try to check your suggestions.

• reverting to version 2019.11 works. Before updating it was working
• conda 4.8.0
• output of python

User belongs to experiment group 'AlwaysDisplayTestExplorer - control'
User belongs to experiment group 'ShowPlayIcon - start'
User belongs to experiment group 'ShowExtensionSurveyPrompt - control'
User belongs to experiment group 'DebugAdapterFactory - experiment'
User belongs to experiment group 'PtvsdWheels37 - experiment'
User belongs to experiment group 'AA_testing - control'
> conda --version
> pyenv root
> python3.7 -c "import sys;print(sys.executable)"
> python3.6 -c "import sys;print(sys.executable)"
> python3 -c "import sys;print(sys.executable)"
> python2 -c "import sys;print(sys.executable)"
> python -c "import sys;print(sys.executable)"
> py -3.7 -c "import sys;print(sys.executable)"
> py -3.6 -c "import sys;print(sys.executable)"
> py -3 -c "import sys;print(sys.executable)"
> py -2 -c "import sys;print(sys.executable)"
> ~\Miniconda3\envs\vscode\python.exe -c "import sys;print(sys.executable)"
> conda info --json
> conda info --json
> conda info --json
> conda env list
> conda info --json
> conda info --json
> conda info --json
> conda env list
> ~\AppData\Local\Microsoft\WindowsApps\python.exe c:\Users\cfede\.vscode\extensions\ms-python.python-2020.1.57204\pythonFiles\interpreterInfo.py
> ~\AppData\Local\Microsoft\WindowsApps\python.exe c:\Users\cfede\.vscode\extensions\ms-python.python-2020.1.57204\pythonFiles\interpreterInfo.py
> ~\AppData\Local\Microsoft\WindowsApps\python3.exe c:\Users\cfede\.vscode\extensions\ms-python.python-2020.1.57204\pythonFiles\interpreterInfo.py
> ~\AppData\Local\Microsoft\WindowsApps\python3.exe c:\Users\cfede\.vscode\extensions\ms-python.python-2020.1.57204\pythonFiles\interpreterInfo.py
Starting Microsoft Python language server.
> conda run -n vscode python -m flake8 --format=%(row)d,%(col)d,%(code).1s,%(code)s:%(text)s c:\Users\cfede\remove-me\vscode-python-pytest-error\tests\foo_test.py
cwd: c:\Users\cfede\remove-me\vscode-python-pytest-error
> conda run -n vscode python -m flake8 --format=%(row)d,%(col)d,%(code).1s,%(code)s:%(text)s c:\Users\cfede\remove-me\vscode-python-pytest-error\tests\foo_test.py
cwd: c:\Users\cfede\remove-me\vscode-python-pytest-error
> conda run -n vscode python -c "import flake8"
> conda run -n vscode python -c "import flake8"
> conda run -n vscode python -c "import flake8"
> conda run -n vscode python -c "import flake8"
> conda run -n vscode python c:\Users\cfede\.vscode\extensions\ms-python.python-2020.1.57204\pythonFiles\testing_tools\run_adapter.py discover pytest -- --rootdir c:\Users\cfede\remove-me\vscode-python-pytest-error -s --cache-clear
cwd: c:\Users\cfede\remove-me\vscode-python-pytest-error

Linter 'flake8' is not installed. Please install it or select another linter".
Error: Module 'flake8' not installed.
> conda run -n vscode python -c "import pytest"
> conda run -n vscode python -c "import pytest"

This gives me the error ModuleNotFoundError: No module named 'pytest'

conda list

# packages in environment at C:\Users\cfede\Miniconda3\envs\vscode:
#
# Name                    Version                   Build  Channel
atomicwrites              1.3.0                    pypi_0    pypi
attrs                     19.3.0                   pypi_0    pypi
ca-certificates           2019.11.27                    0
certifi                   2019.11.28               py37_0
colorama                  0.4.3                    pypi_0    pypi
entrypoints               0.3                      pypi_0    pypi
flake8                    3.7.9                    pypi_0    pypi
importlib-metadata        1.3.0                    pypi_0    pypi
mccabe                    0.6.1                    pypi_0    pypi
more-itertools            8.0.2                    pypi_0    pypi
openssl                   1.1.1d               he774522_3
packaging                 20.0                     pypi_0    pypi
pip                       19.3.1                   py37_0
pluggy                    0.13.1                   pypi_0    pypi
py                        1.8.1                    pypi_0    pypi
pycodestyle               2.5.0                    pypi_0    pypi
pyflakes                  2.1.1                    pypi_0    pypi
pyparsing                 2.4.6                    pypi_0    pypi
pytest                    5.3.2                    pypi_0    pypi
python                    3.7.6                h60c2a47_2
setuptools                44.0.0                   py37_0
six                       1.13.0                   pypi_0    pypi
sqlite                    3.30.1               he774522_0
vc                        14.1                 h0510ff6_4
vs2015_runtime            14.16.27012          hf0eaf9b_1
wcwidth                   0.1.8                    pypi_0    pypi
wheel                     0.33.6                   py37_0
wincertstore              0.2                      py37_0
zipp                      0.6.0                    pypi_0    pypi

I'll try updating conda and checking the path

[CaselIT]

I've tried updating conda to version 3.8.1 and checking for not existing paths in the path. I had a couple.

I have the same error if the base env does not have pytest installed, but after installing I can corretly dicover the tests, ie I no longer get the error Error: The system cannot find the path specified.

I've tried in another project and it seems that that the discovery run on the base env (it fails to impor some other package), so the current error probably happens because of the conda bug you linked to conda/conda#9571

I guess that the pr #9490 will fix this issue. Is the plan to release a fix to the 2020.1 or will it be in to 2020.2? (Just to know if I have to hold off updating until then)

Regarding the path, it's strange that it gives error, every other application I use, including the previous version of the extension, was working correctly.

I think this can be closed (now or when #9490 is merged)

Thanks for the help

[karrtikr]

We plan to release it on Monday. The version would start with 2020.1
Glad you got it working. Having non existing paths seemed to be the problem + using conda run seems to give rise to this issue. Previous version of the extension was not using conda run, so you didn't face the issue.
Closing this now.

[phloose]

When i use @CaselIT example repo and try to discover tests i get the following error:

python c:\Users\phloose\.vscode\extensions\ms-python.python-2020.1.57204\pythonFiles\testing_tools\run_adapter.py discover pytest -- --rootdir c:\Users\phloose\Desktop\vscode-python-pytest-error -s --cache-clear
Test Discovery failed: 
Error: ERROR conda.cli.main_run:execute(30): Subprocess for 'conda run ['python', 'c:\\Users\\phloose\\.vscode\\extensions\\ms-python.python-2020.1.57204\\pythonFiles\\testing_tools\\run_adapter.py', 'discover', 'pytest', '--', '--rootdir', 'c:\\Users\\phloose\\Desktop\\vscode-python-pytest-error', '-s', '--cache-clear']' command failed.  (See above for error)
Traceback (most recent call last):

  File "c:\Users\phloose\.vscode\extensions\ms-python.python-2020.1.57204\pythonFiles\testing_tools\run_adapter.py", line 13, in <module>

    from testing_tools.adapter.__main__ import parse_args, main

  File "c:\Users\phloose\.vscode\extensions\ms-python.python-2020.1.57204\pythonFiles\testing_tools\adapter\__main__.py", line 9, in <module>

    from . import pytest, report

  File "c:\Users\phloose\.vscode\extensions\ms-python.python-2020.1.57204\pythonFiles\testing_tools\adapter\pytest\__init__.py", line 7, in <module>

    from ._discovery import discover

  File "c:\Users\phloose\.vscode\extensions\ms-python.python-2020.1.57204\pythonFiles\testing_tools\adapter\pytest\_discovery.py", line 8, in <module>

    import pytest

ModuleNotFoundError: No module named 'pytest'

@karrtikr i think my error, despite a different message Error: ERROR conda.cli.main_run:execute(30): Subprocess for 'conda run [...] command failed' is related to the conda problem you mentioned and what i found with git bisect. When i revert b543408 and rebuild the extension the test discovery works as it was with the last release.

[CaselIT]

@phloose this problems seems to be that conda is using the base env, instead of the one you specify.

[karrtikr]

Yes, you're right. The error you face is related to the same conda issue. But @CaselIT also faced another error related to invalid paths, which you did not face.

[phloose]

@CaselIT Was assuming that! Glad we found it!

@karrtikr Yeah just realized it. When i looked through the issue list, this was the first issue that almost matched the problem i faced, so i decided against opening another issue.

[CaselIT]

Do you this that the invalid path error is conda's fault, or something in the extension?

I think is should be fixed, since it may happen that some folder that no longer exists are left in the path.

I can open an issue in the conda repo if the fault lies in conda

[karrtikr]

What're the paths that you removed? Why were they non existent? I believe it's not related to conda or the extension, but just a PEBCAK/ user error.

[CaselIT]

They were normal folders that no longer exist.
One was an older install of .dotnet.

I don't know why they where left there and honestly it is not important. They should not cause problems regardless of their presence

[karrtikr]

So they're not related to the extension or conda. It was just a bad setup on your system.

They should not cause problems regardless of their presence

But it just did :)

[CaselIT]

But it just did :)

so the problem is with conda or the extension...
It's the first time I've had any problem regarding not existing folders on the path.
Regardless of who's the blame, there is no point continuing this, if I have other problems when the extension is updated I'll open a new issue.