[doctests] Namespace packages (PEP 420) missing, leading to errors with relative imports

[punitchandra]

When testing packages with implicit namespace and using --doctest-module the doc test module is imported without the implicit namespace.

This causes problems as the module can gets imported with different name. Once with the proper path and once with missing the namespace.

My pytest related versions:
pytest (3.5.0)
pytest-attrib (0.1.3)
pytest-cov (2.5.1)
pytest-forked (0.2)
pytest-timeout (1.2.1)
pytest-xdist (1.22.2)

OS is Windows 7.
Please look at the example:

foo.zip

[pytestbot]

GitMate.io thinks possibly related issues are #603 (monkeypatch does not work on already-imported function), #181 (--pdb does not work when collecting), #275 (doctest does not consider usefixtures in pytest.ini), #605 (tmpdir.join("foo").write(...) doesn't work as expected.), and #478 (--pyargs does not understand namespace packages).

[nicoddemus]

Unfortunately pytest currently lacks support for namespace packages.

Related: #1927 and #478

[punitchandra]

#1927 is related to relative imports with implicit namespace. This is documented as not being supported here. This issue is with doctest and absolute imports in implicit namespace. It will help if it is clarified in the documentation that pytest does not support doctest for implicit namespace.
Also is there any plan to add this support?

[nicoddemus]

Hi @punitchandra, there are no specific plans for this feature. We have very loose plans since we have very little man-power, unfortunately.

If you want to help out though, you are more than welcome. We would love to guide you through a PR if you have the time.

[jaraco]

I believe the issue is illustrated by this exercise I just ran:

~ $ cd ~/draft
draft $ ls
draft $ mkdir pkg
draft $ touch pkg/__init__.py
draft $ cat > pkg/mod.py
"""
>>> print(__name__)
pkg.mod
"""
draft $ pip-run -q pytest -- -m pytest --doctest-modules
========================================================== test session starts ==========================================================
platform darwin -- Python 3.9.0, pytest-6.2.1, py-1.10.0, pluggy-0.13.1
rootdir: /Users/jaraco/draft
collected 1 item                                                                                                                        

pkg/mod.py .                                                                                                                      [100%]

=========================================================== 1 passed in 0.03s ===========================================================
draft $ rm pkg/__init__.py
draft $ pip-run -q pytest -- -m pytest --doctest-modules
========================================================== test session starts ==========================================================
platform darwin -- Python 3.9.0, pytest-6.2.1, py-1.10.0, pluggy-0.13.1
rootdir: /Users/jaraco/draft
collected 1 item                                                                                                                        

pkg/mod.py F                                                                                                                      [100%]

=============================================================== FAILURES ================================================================
_____________________________________________________________ [doctest] mod _____________________________________________________________
001 
002 >>> print(__name__)
Expected:
    pkg.mod
Got:
    mod

/Users/jaraco/draft/pkg/mod.py:2: DocTestFailure
======================================================== short test summary info ========================================================
FAILED pkg/mod.py::mod
=========================================================== 1 failed in 0.02s ===========================================================

[jaraco]

This issue is exacerbated when one of the modules in the namespace matches the name of a built-in module. In jaraco/jaraco.email@e302051, I converted the project to use PEP 420 namespace packages, but the tests started failing with errors in collection:

__________________ ERROR collecting jaraco/email/messages.py ___________________
/opt/hostedtoolcache/Python/3.8.7/x64/lib/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
<frozen importlib._bootstrap>:1014: in _gcd_import
    ???
<frozen importlib._bootstrap>:991: in _find_and_load
    ???
<frozen importlib._bootstrap>:973: in _find_and_load_unlocked
    ???
E   ModuleNotFoundError: No module named 'email.messages'

The module is named jaraco.email.messages, but due to the failed detection of the namespace package, it looks for email.messages, in which email resolves to the stdlib module, which has no messages.

I'd really like to see a fix for this issue.

[jaraco]

This issue is exacerbated when one of the modules in the namespace matches the name of a built-in module.

...any top level module.

[fabiannagel]

+1 on this. I spent hours looking for a solution that doesn't require me to bloat up my tests directory with empty __init__.py files but there actually seems no way around it for now.

[jaraco]

In the referenced commit, I've attempted to implement testpaths for configparser, but it doesn't work as expected. Maybe I'm not using it correctly, but it produces an error:

No module named 'src.backports'; 'src' is not a package

It probably doesn't help that both backports.configparser and the top-level tests contain a compat module. Still, it would be nice if the tests could somehow recognize that src should be on sys.path and always import the packages/modules from there.

[alicederyn]

This expectation is a common misconception about namespace packages. Namespace packages are just like normal packages in that they can (and do) contain modules.

My understanding is that a namespace package will only be installed once, so if you put a file in it, it will only end up being installed if you happen to be first to be pip installed. [edit: my understanding was wrong]

Regardless, pytest's importlib mode has logic that ignore files in namespace modules, so the original comment still stands. [Edit: this should definitely be fact checked, it's probably wrong. I read a bunch of different code in a short space of time and I probably mixed up a bunch of repos]

If a developer doesn't have to add the paths for regular packages, they shouldn't have to add them for namespace packages.

It looks as if importlib mode will become the default at some point, so developers will have to add for both.

But an even better approach would be for pytest to honor the package metadata to determine the top-level packages or simply to honor the Python heuristics for identifying top-level packages (a folder on sys.path is a package no matter what)

I suggest you look for existing issues about this for importlib, or open new ones if they do not exist, so your ideas can be discussed.

[jaraco]

I haven't forgotten about this issue. I do still intend to try the proposed fix on a jaraco* package, as that's where I'd reported it recently.

[alicederyn]

My understanding is that a namespace package will only be installed once, so if you put a file in it, it will only end up being installed if you happen to be first to be pip installed.

I am completely wrong about this! Mea culpa. Files in namespace packages appear to be fine provided they have a unique name.

[jaraco]

I've confirmed that use of import-mode=importlib (and without testpaths) fixes the issue for jaraco.email and jaraco.functools, although those projects have a minimal test suite and don't fall afoul of the known drawback of importlib mode:

One drawback however is that test modules are non-importable by each other. Also, utility modules in the tests directories are not automatically importable because the tests directory is no longer added to sys.path.

That issue is apparent when applying the same change to jaraco.abode, where modular behaviors are no longer supported. And of course, if I apply import-mode=importlib technique to projects generally, it'll create problems in projects without namespace packages.

So until import-mode=importlib is a stable approach without major drawbacks, I'm left to adopt the change selectively (as more of a workaround than a robust solution).

In any case, this solution is a decent workaround and a definite improvement for those projects that couldn't run doctests at all, so I'll donate half the bounty to the indicated charity (and since it's a charity, my employer will match).

I plan to follow #8110, #7652, #10332, #10337, #10341, #10228, and others relating to import-mode=importlib.

[jaraco]

[alicederyn]

Thank you, @jaraco ! 🙏🏳️‍🌈