what to do with an AsyncIterator that raises a NotImplementedError

[iscai-msft]

Bug Report

(A clear and concise description of what the bug is.)
mypy doesn't seem to be fully a NotImplementedError in a parent class. In my parent class, I have an async def iter_x function that raises this error, and this function is actually implemented in subsequent child classes. It seems that mypy thinks the paren'ts iter function function is a coroutine because it's an async def without an await (gotten from this issue comment). As such, I get the following error: Return type "AsyncIterator[bytes]" of "iter_x" incompatible with return type "Coroutine[Any, Any, AsyncIterator[bytes]]" in supertype "X".

To Reproduce

(Write your steps here:)

1. Create an AsyncIterator async def function in a parent class that raises a NotImplementedError
2. Implement the AsyncIterator async def function in a child class. Ours does async iteration / await calls, so it has to be async def
3. Run mypy and you should get an error similar to Return type "AsyncIterator[bytes]" of "iter_x" incompatible with return type "Coroutine[Any, Any, AsyncIterator[bytes]]" in supertype "X"

Expected Behavior

(Write what you thought would happen.)
My expected behavior is that mypy would fully listen to NotImplementedError, and not make assumptions about the base method being a coroutine because of it's lack of an "await"

Actual Behavior

mypy thinks the parent class' method is a coroutine because it's an async def AsyncIterator without a yield (following this issue comment)

(Write what happened.)

Your Environment

• Mypy version used: 0.782
• Mypy command-line flags: mypy
• Mypy configuration options from mypy.ini (and other config files):
• Python version used: 3.9
• Operating system and version: Mac OS 11.4

[hauntsaninja]

The easy workaround is to add a yield after the NotImplementedError, or do if False: yield (mypy itself does that e.g. here:

mypy/mypy/stubtest.py

Line 771 in f98f782

def verify_typevarexpr(

)

[iscai-msft]

thanks for your quick response, I'll do this to get rid of my # type: ignore comments, thanks so much!

[seanofw]

I ran into this myself today. It's "fixed" in my code with the yield workaround, but is this bug being worked on?

[hauntsaninja]

It's not really a bug, since you could totally have an async function that produces an AsyncIterator when awaited. Relevant: #5385 (comment)

[seanofw]

I'm really not sure I agree that it's not a bug. If I take a valid method signature in a parent class and copy it verbatim into a child class, I would expect mypy not to emit error messages about an incompatible signature. I had this, and mypy complained loudly about it:

class Parent:
    async def something(self, args: Optional[str] = None) -> AsyncIterable[Something]:
        raise NotImplementedError("...")

class Child(Parent):
    async def something(self, args: Optional[str] = None) -> AsyncIterable[Something]:
        ...
        yield stuff

The child method is declared identically, byte-for-byte, character-for-character the same as the parent, and mypy complained about them being incompatible. You really don't think that's an issue?

[hauntsaninja]

Surprising, sure, but not a bug. The presence of yield very fundamentally changes the nature of the object at runtime. Another workaround that might make more sense to you:

class Parent:
    def something(self, args: Optional[str] = None) -> AsyncIterable[Something]:
        raise NotImplementedError("...")

[seanofw]

Urk. Yeah, I see why that works. I tend to forget just how much yield really turns a function inside-out. (I swear we'd all have been better off if call/cc had been more widely adopted outside of Scheme; at least there the continuation is explicit.) The real issue is the fact that a single keyword deep in the body of the function can implicitly change the function's type signature, but I suppose there's not much that can be done about that at this point.

Can't say I like the fact that this happens, though. It really violates intuition that if one thing appears to be x and the other thing appears to be x, mypy complains that x != x; but in this case, it really is x versus x', and mypy is right to gripe about it.