pytest --doctest-modules crashes on file-level __test__ = False

[kaddkaka]

Bug report

Bug description:

First opened this issue at pytest-dev but was directed here. pytest-dev/pytest#11730

A file level __test__ = False succesfully stops pytest from collecting any test from that file, however when running pytest --doctest- modules on the same file, this error appears:

/sw/Python/Ubuntu-22.04/3.10.5/lib/python3.10/site-packages/_pytest/runner.py:341: in from_call
    result: Optional[TResult] = func()
/sw/Python/Ubuntu-22.04/3.10.5/lib/python3.10/site-packages/_pytest/runner.py:372: in <lambda>
    call = CallInfo.from_call(lambda: list(collector.collect()), "collect")
/sw/Python/Ubuntu-22.04/3.10.5/lib/python3.10/site-packages/_pytest/doctest.py:587: in collect
    for test in finder.find(module, module.__name__):
/sw/Python/Ubuntu-22.04/3.10.5/lib/python3.10/doctest.py:940: in find
    self._find(tests, obj, name, module, source_lines, globs, {})
/sw/Python/Ubuntu-22.04/3.10.5/lib/python3.10/site-packages/_pytest/doctest.py:536: in _find
    super()._find(  # type:ignore[misc]
/sw/Python/Ubuntu-22.04/3.10.5/lib/python3.10/doctest.py:1018: in _find
    for valname, val in getattr(obj, '__test__', {}).items():
E   AttributeError: 'bool' object has no attribute 'items'

It's enoough to have a file with only this content to see the behavior:

__test__ = False

Version:

• Python 3.10.5
• pytest 7.4.3

CPython versions tested on:

3.10

Operating systems tested on:

Linux

Linked PRs

• gh-113360: Fix doctest failure when __test__ is set to falsy value #113389
• gh-113360: Fix the documentation of module's attribute __test__ #113393
• [3.12] gh-113360: Fix the documentation of module's attribute __test__ (GH-113393) #113760
• [3.11] gh-113360: Fix the documentation of module's attribute __test__ (GH-113393) #113761

[terryjreedy]

The following is a self-contained reproducer for the exception above.

__test__ = False
import doctest
doctest.testmod()

doctest doc says, with 'M' a module, "If M.test exists and is truthy, it must be a dict". It says nothing about 'falsey' values, which to me, implies that they will be ignored. AFAIK, the _find code has never checked the boolean value of __test__. In 2004, why would anyone not make it a dict or mapping?

I think the doc should have always said that __test__ should be a mapping, but too late now.
<EDIT: strikeout wrong conclusion. I agree with just changing the doc.>
Instead, add a new unittest that currently fails. (This may or may not be easy -- a beginner should try this first.) Next replace

            for valname, val in getattr(obj, '__test__', {}).items():  

with

            if (tests = getattr(obj, '__test__', {})):
                for valname, val in tests.items():

and indent the suite under the 'for' statement.

[serhiy-storchaka]

In one place the documentation says:

If ``M.__test__`` exists and is truthy, it must be a dict, and each
entry maps a (string) name to a function object, class object, or string.
Function and class object docstrings found from ``M.__test__`` are searched, and
strings are treated as if they were docstrings.

So it can be a dict or a false value.

In other place it says:

   Also test examples reachable from dict ``m.__test__``, if it exists and is not
   ``None``.  ``m.__test__`` maps names (strings) to functions, classes and
   strings; function and class docstrings are searched for examples; strings are
   searched directly, as if they were docstrings.

So it can be a dict or None.

A small contradiction is that the first description implies any false value, wile the second one implies only None.

But the code implies that it should always be a dict.

            for valname, val in (getattr(obj, '__test__', None) or {}).items():

The documentation matches the behavior of the old, pre-2.4, code, which was changed in 8485b56. The last 19 years the documentation does not match implementation.

[serhiy-storchaka]

The documentation was updated. __test__ = False is not valid since Python 2.4.

[kaddkaka]

(Also mentioned and discussed on pytest: pytest-dev/pytest#11730)