apply markers based on fixtures/dependent fixtures

[RonnyPfannschmidt]

use case is deselecting tests based on markers that relate to used fixtures

[The-Compiler]

Can't this easily be done already?

def pytest_collection_modifyitems(items):
    for item in items:
        if 'qapp' in getattr(item, 'fixturenames', ()):
            item.add_marker('gui')

[RonnyPfannschmidt]

@The-Compiler the markers are supposed to be taken from the actual fixture function, thats potentially a indirect dependency

[kprzybyla]

Hi,

I came to the point in my project, where I want to use (custom) marks on fixtures. I read thought the documentation and I found that marks can only be applied to tests and have no effect on fixtures. Then I found this issue which I suppose is actually related to the problem that I'm struggling with.

I actually found a workaround for this issue. It turns out that this "feature" is already implemented as a side effect of pytest.param() because you can define fixture with single parameter and assign marks to that parameter. This basically creates fixture with marks applied to it.

import pytest

# The value of fixture param does not matter, we only use params to pass marks to fixture
@pytest.fixture(params=[pytest.param(0, marks=pytest.mark.local_address)])
def address():
    return '127.0.0.1', 7000

The above code could be treated as equal to below fixture definition.

import pytest

@pytest.fixture
@pytest.mark.local_address
def address():
    return '127.0.0.1', 7000

Now when we define below test case

def test(address):
    pass

Running pytest -m local_address will get us following outcome:

1 passed in 0.03 seconds

Running pytest -m "not local_address"

1 deselected in 0.01 seconds

You can also use marks like skip or xfail and chaining fixtures also works and keeps the information about marks. So my question is: is this actually intended? If yes, can be make it to work with the definition in second code snippet which could basically assume that fixture without params defined is a single param fixture with marks applied to it?

[nicoddemus]

Oh that's interesting. I would like to know @RonnyPfannschmidt's opinion here, but I would like to see marks working on fixtures; seems natural and what users expect.

[RonnyPfannschmidt]

@nicoddemus if getfixturevalue is explicitly excluded as mark source, it can be reasonably done

that specification needs to ensure a awareness of lazy/delayed fixtures
special care needs to be taken for fixtures that override old values vs fixtures that replace other fixtures
(it may be necessary to have fixtures declare replacement vs extension more explicitly)

i will take a deeper look in a few months when i'm back to active

[nicoddemus]

Great, thanks @RonnyPfannschmidt for the follow up

[jaraco]

I also was surprised when I applied this change to selectively skip tests that require the marker, but since the marker is silently ignored, it had no effect. Since all of the logic for skipping and xfailing tests through decorators is tied up in marking, it really would be nice for markers to be honored in fixtures.

[RonnyPfannschmidt]

@jaraco we still have no mechanism to correctly and reliably transfer them

i haven't yet gotten around looking into it

[jaraco]

Understood. And it makes me wonder - maybe there's another way to think about it. Instead of transferring markers, maybe pytest should honor the markers when the fixtures are invoked. I'm not sure that makes sense in the pytest model. Alternately, pytest could offer decorators that when applied to fixtures could skip or xfail their behavior. For example:

def decorator_skip_if(**skip_params):
    def decorator(func):
        @functools.wraps(func)
        def wrapper(*args, **kwargs):
            pytest.skip_if(**skip_params)
            return func(*args, **kwargs)
       return wrapper
    return decorator

@decorator_skip_if("not hasattr(foo, 'bar')")
@pytest.fixture
def my_selectively_skipped_fixture():
    ...

It's sad that it wouldn't re-use the same mechanism as markers, but wouldn't be constrained to test functions.

[RonnyPfannschmidt]

at fixture invocation doesn't make sense in the pytest model (as it removes pre-setup consideration)

the decorators would be a tricky api, harder to get structurally right than juts imperative code in the fixture function

we should document a pattern to externalize specific test requirements to session scope fixtures

[RonnyPfannschmidt]

i came to the conclusing that markers simply dont make sense on fixtures as transfer and consideration are simplytoo unclear

we amy eventually come up with a tool that brings fixtures and markers together, fixture themselfe aint

a possible hack-ish workaround would be a single element parameterset that adds markers

[gwerbin]

@nicoddemus if getfixturevalue is explicitly excluded as mark source, it can be reasonably done

that specification needs to ensure a awareness of lazy/delayed fixtures special care needs to be taken for fixtures that override old values vs fixtures that replace other fixtures (it may be necessary to have fixtures declare replacement vs extension more explicitly)

i will take a deeper look in a few months when i'm back to active

Pardon my ignorance, but what is a "mark source" in this context? I'm not a Pytest expert and I didn't find anything in the docs that seemed relevant.

[mfarrugi]

import pytest

# The value of fixture param does not matter, we only use params to pass marks to fixture
@pytest.fixture(params=[pytest.param(0, marks=pytest.mark.local_address)])
def address():
    return '127.0.0.1', 7000

Is this still the best way to accomplish propagating marks through fixtures?

This seems to do exactly what we want, is there a way to support proper syntax for this?

[RonnyPfannschmidt]

There's intentionally no proper syntax as there's a harrowing amount of edge cases surrounding fixture overrides

[mfarrugi]

So is the param hack the recommended way to do this?

[RonnyPfannschmidt]

It's more of a begrudgingly acknowledged workaround that suffers edge cases that usually don't bite but can't be ignored for a real implementation