Warn when an async function is executed without an appropriate plugin

[crusaderky]

In the following code:

async def test1():
    assert Fail

@pytest.mark.asyncio
async def test2():
    assert Fail

• test1 is silently skipped and marked as successful.
• test2 follows the same fate if pytest-asyncio is not installed.

Both use cases can be very easily triggered by distraction in real life; the sensible behaviour should be that both tests fail no matter what:

• Using the @pytest.mark.asyncio decorator without pytest-asyncio installed should cause an error
• An error should be raised if a test function returns a coroutine

[nicoddemus]

I agree that we should do something when a function returns a coroutine, but I believe a warning would be better. The warning might even suggest one of the existing async plugins, for example: pytest-asyncio, pytest-trio or pytest-tornasync.

[Zac-HD]

• Using the @pytest.mark.asyncio decorator without pytest-asyncio installed should cause an error

You can opt-in to errors on unknown marks with the --strict argument. We should help more though...

[nicoddemus]

Closing as duplicate of #2224