Unable to remove Optional wrapper when using Mapping[str, str] instead of dict

[ajhynes7]

Hello, I'm not sure if this is a bug, but perhaps it'll help to clarify the difference between using dict and Mapping[str, str] as annotations.

This is a common pattern in Python to avoid mutable default arguments:

def foo(x: Optional[dict] = None):
    if x is None:
        x = {}

So I tried writing a function to capture this pattern.

from typing import Optional, TypeVar

T = TypeVar("T")


def assign_if_none(obj: Optional[T], assignment: T) -> T:

    return assignment if obj is None else obj

Now you can use it like this:

def foo(x: Optional[dict] = None):
    x = assign_to_none(x, {})

But mypy raises errors when I use a type annotation with Mapping or Dict instead of dict.

Full example:

from typing import Mapping, Optional, TypeVar

T = TypeVar("T")


def assign_if_none(obj: Optional[T], assignment: T) -> T:

    return assignment if obj is None else obj


def func_1(mapping: Optional[dict] = None) -> str:

    mapping = assign_if_none(mapping, {})

    if "key" in mapping:
        return "Included"

    return "Excluded"


def func_2(mapping: Optional[Mapping[str, str]] = None) -> str:

    mapping = assign_if_none(mapping, {})

    if "key" in mapping:
        return "Included"

    return "Excluded"

func_1 has no errors, but func_2 does.

demo.py:23: error: Incompatible types in assignment (expression has type "object", variable has type "Optional[Mapping[str, str]]")
demo.py:25: error: Unsupported right operand type for in ("Optional[Mapping[str, str]]")

I tried putting reveal_type(mapping) before and after the call to assign_if_none in both functions. Here are the results:

func_1:

demo.py:13: note: Revealed type is "Union[builtins.dict[Any, Any], None]"
demo.py:15: note: Revealed type is "builtins.dict[Any, Any]"

func_2:

demo.py:25: note: Revealed type is "Union[typing.Mapping[builtins.str, builtins.str], None]"
demo.py:27: note: Revealed type is "Union[typing.Mapping[builtins.str, builtins.str], None]"

So in func_1, mypy correctly infers that the call to assign_if_none removes the Optional wrapper. But this doesn't happen in func_2.

If this isn't a bug with mypy, is there a better way to annotate the function assign_if_none so it works in func_2?

Environment

• Mypy version used: 0.910
• Python version used: 3.8.12

[erictraut]

This does look like a bug in mypy, probably in the TypeVar constraint solver logic. This type checks fine in pyright.

Interestingly, if you add a cast or an explicit type annotation for {}, the errors go away.

    mapping = assign_if_none(mapping, cast(Mapping[str, str], {}))

or

    empty: Mapping[str, str] = {}
    mapping = assign_if_none(mapping, empty)

[JelleZijlstra]

Does --allow-redefinition help here?

[hauntsaninja]

My guess would be that mypy infers {} as a dict[bottom, bottom], but dict is invariant in its key type (a common surprise for users) and so mypy joins Mapping[str, str] and dict[bottom, bottom] to object. Evidence in favour of this guess is that mapping = assign_if_none(mapping, cast(dict[str, str], {})) works as well.

[hauntsaninja]

Yeah, looks like that's it. A similar thing happens with List, but not with Sequence

[hauntsaninja]

This example got fixed in #16994