Test collector bug when --rootdir is not real path

[iynehz]

• a detailed description of the bug or problem you are having

pytest test collector can get wrong item nodeid when running with a non --rootdir is not real path.

Actually I am studying a strange vscode-python extension behavior that when Linux workspace directory is not real path, vscode-python's test discovery gets wrong info about test file path. Tracking it down now I believe it's caused by this bug in pytest.

• output of pip list from the virtual environment you are using

(.venv) $ pip list
Package    Version
---------- -------
attrs      21.2.0
iniconfig  1.1.1
packaging  21.0
pip        21.2.4
pluggy     0.13.1
py         1.10.0
pyparsing  2.4.7
pytest     6.2.4
setuptools 41.6.0
toml       0.10.2

• pytest and operating system versions

pytest: 6.2.4 as shown above
OS: RHEL 8 (I believe other Linux can reproduce this too)

• minimal example if possible

Our demo directory structure (on Linux),

(.venv) $ tree
.
├── mypytest
└── tests
    ├── __init__.py
    └── test_dummy.py

1 directory, 3 files

To see this issue we use a custom plugin to override pytest_collection_finish, to print the collected items nodeid to a log file.

(.venv) $ cat mypytest
#!/usr/bin/env python3

import sys
import pytest

class MyCollector:
    def __init__(self, logfile):
        self.logfile = logfile

    def pytest_collection_finish(self, session):
        with open(self.logfile, "w") as f:
            for item in session.items:
                f.write("%s\n" % dict(nodeid=item.nodeid, fspath=item.fspath))

if __name__ == "__main__":
    plugin = MyCollector("./collector.log")
    pytest.main(sys.argv[1:], [plugin])

Run test collection from the "tests" subdir: With --rootdir be the project root, if --rootdir is real path, result is good in that nodeid is 'tests/test_dummy.py' as relative to the rootdir. Buf if --rootdir is not real path, like if it's a symlink, the collected test has wrong nodeid.

# chdir to "tests"
(.venv) $ cd tests
(.venv) $ pwd
/var/tmp/pytest-issue/tests

# this is good
(.venv) $ ../mypytest --collect-only --rootdir /var/tmp/pytest-issue > /dev/null && cat collector.log
{'nodeid': 'tests/test_dummy.py::test_plus', 'fspath': local('/var/tmp/pytest-issue/tests/test_dummy.py')}

# bad if rootdir is not realpath: let's run from /var/tmp/foo which is not realpath.
# fspath is good but nodeid has not the "tests" subdir.
(.venv) $ ln -s /var/tmp/pytest-issue /var/tmp/foo
(.venv) $ cd /var/tmp/foo/tests
(.venv) $ ../mypytest --collect-only --rootdir /var/tmp/foo > /dev/null && cat collector.log
{'nodeid': 'test_dummy.py::test_plus', 'fspath': local('/var/tmp/pytest-issue/tests/test_dummy.py')}

vscode-python extension internally does something similar as above command for pytest test discovery. So when our workspace dir is not real path, we get problem with test discovery in vscode: while it detects the test files, it gets wrong idea about their path, causing individual run on a test file to fail.

[kdkavanagh]

@stphnlyd curious if you ever found a solution to this? Hitting this as well...

[azachar]

I've identified a workaround for the Python WSL debugging issue with symlinks in VSCode. By starting VSCode from the actual real location (not the symlinked one), test discover magically works again.

Upon further investigation, the issue seems linked to the Config class's invocation_params.dir

pytest/src/_pytest/main.py

Line 672 in 6fe4391

self.config.invocation_params.dir,

which uses the Path class. This setup resolves paths to the original sources of symlinks, rather than their targets. This misalignment causes complications in both VSCode and pytest.

For instance, a symlink path like /my-symlink/test-folder pointing to /mnt/somewhereElse/test-folder leads to issues. The system resolves to "hidden" file location (/mnt/somewhereElse/test-folder), but it should be targeting the symlink (/my-symlink/test-folder).