fixture hooks not running in packages

[laurentsenta]

Hi there, thanks for maintaining such a great tool.

(pytest 3.0.7)

I have an unexplained behavior with conftest and fixtures,
I try to write a fixture that would depend on other fixtures optionally. If a test uses both the fixtures a and b for example, then the value of b would depend on the value of a.

I intend to use the pytest_fixture_setup hook, somehow, but:

• When I define pytest_fixture_setup in tests/mypackage/conftest.py, it is never run.
• When I define it in conftest.py (root, not in package), it is run.

This contradicts what I understood from:
https://docs.pytest.org/en/latest/writing_plugins.html#conftest-py-local-per-directory-plugins
(use conftest in the package where you need it, avoid non-package ones because imports are source of errors).

Even weirder, if you make an error on the hook parameters in tests/mypackage/conftest.py, pytest will raise an error. Fix the typo, the hook is ignored.

I pushed an empty project showing this issue:
https://github.com/lsenta/pytest-conftest-broken

Here's the output:

laurent at HALIX in ~/dev/pytest-conftest-broken on master
› tree
.
├── conftest.py
└── some_test
    ├── __init__.py
    ├── conftest.py
    └── test_x.py
laurent at HALIX in ~/dev/pytest-conftest-broken on master
› pytest -s
============================ test session starts =============================
platform darwin -- Python 3.6.0, pytest-3.0.6, py-1.4.32, pluggy-0.4.0
rootdir: /Users/laurent/dev/pytest-conftest-broken, inifile:
plugins: timeout-1.2.0
collected 2 items

some_test/test_x.py runtest_setup.package
runtest_setup.project
fixture_setup.project
.runtest_setup.package
runtest_setup.project
fixture_setup.project
.

========================== 2 passed in 0.01 seconds ==========================

the conftest is the same, so I expect to see fixture_setup.package at some point.

[nicoddemus]

At first glance your example should be running pytest_fixture_setup from the package, not sure what's the problem right now, might be indeed a bug. Needs more investigation.

[nicoddemus]

Btw. thanks a lot for the detailed report and the sample repository to boot! We really appreciate throughout reports like this one! 👍

[MarSoft]

I bumped into the same issue and did some investigation.
I added a set_trace statement to pluggy._HookCaller._add_hookimpl to break out to pdb when fixture name is pytest_fixture_setup. And here is what I see:

1. While py.test is initializing (before anything is printed to the console), _add_hookimpl gets called 4 times: for _pytest.setuponly (wrapper), _pytest.fixtures (the "standard" implementation), _pytest.setupplan and /tmp/tests/conftest.py (my implementation). All these implementations are successfully added to the _HookCaller.
2. Then py.test prints all its regular greeting stuff like test session starts, items collecting, and starts running tests from the first module (prints module name and is ready to print dot for the first test)
3. Now we break out into the debugger again! I.e. another _HookCaller object is initialized for pytest_fixture_setup hook. I have checked, this is a different object than the one that was initialized during "general" initialization sequence.
4. And now _add_hookimpl is called only 3 times: for _pytest.setuponly, _pytest.fixtures and _pytest.setupplan. No call for my implementation.

This explains @Isenta's observation regarding error being raised for incorrect parameters: an error is raised when the first _HookCaller object is populated.

[MarSoft]

Okay, looks like this is because _pytest.fixtures.FixtureDef.execute uses _pytest.main.Session.gethookproxy. Which in turn creates a subset_hook_caller excluding all the conftest.py modules.

So this looks like a bug with either FixtureDef.execute. It calls gethookproxy passing it request.node.fspath which is seemingly meant to select conftest modules relevant to the current test being run; but when fixture's scope is session, request.node is a Session instance, and its path points to the project's root.

Conclusion. The issue happens only with session-scoped fixtures; for such fixtures (regardless of where they are declared) only the root conftest.py module is respected.

Same issue with pytest_fixture_post_finalizer (fixtures.py, line 764).

[RonnyPfannschmidt]

thanks for the investigation, that gave me an idea where to look while tracing it

pytest fixtures consider fixture hooks based on the fspath of the requesting node (and gets fs hook proxies for the contests along the way

for a session scoped fixture, the requesting node is the Session, which uses the rootdir as fspath

as such contests along the way can no longer be considered

as far a i can tell this is per design and for good reasons, just very confusing without more context

[MarSoft]

@RonnyPfannschmidt, thanks for an explanation. Looks like this is indeed not a bug... Although it's inconvenient in some cases. Consider I have a project with all tests stuff being put in a separate tests package:

my_project/
    project_module/
        ...
    tests/
        conftest.py
        test_something.py
    setup.py
...

With this behaviour, I will have to pollute project's root directory with a separate conftest.py file just for these session-scoped hooks...
Could you suggest some workaround to avoid creating extra conftest.py in the project's root directory?

[RonnyPfannschmidt]

@MarSoft for now i am not aware of a known workaround, we'd have to come up with one

[MarSoft]

Just for record: the semi-workaround I use for now is to have the hook code declared in my "main" conftest (tests/conftest.py) together with all other stuff, and placing a stub conftest to the project's root:

from tests.conftest import pytest_fixture_setup

But for non-session-scoped fixtures it would probably result in this hook being called twice (once from root conftest and once from main conftest).

[Sushma-Kattinti-Bose]

I stumbled upon the same issue. I have pytest_collection_modifyitems python hook in the conftest.py (at root package level of the repo) which works like a charm - this is for pushing tests to test case management when pytest starts. We would like to avoid any contest.py files at the root level. Unfortunately, this hook is not getting evoked if I keep in another tests /conftest.py in the directory level. Any suggestions if there is a workaround, please? Thank you

[bluetech]

I think this was fixed in pytest 3.3.0 by commit a63b34c (PR #2127) which now makes pytest_fixture_setup respect the item's conftest, not only the session's.