Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Odd error message when kwargs passed as **dict #5382

Open
scop opened this issue Jul 22, 2018 · 17 comments
Open

Odd error message when kwargs passed as **dict #5382

scop opened this issue Jul 22, 2018 · 17 comments
Labels
feature priority-0-high topic-calls Function calls, *args, **kwargs, defaults topic-usability

Comments

@scop
Copy link
Contributor

scop commented Jul 22, 2018

Checking this with mypy 0.620:

from concurrent.futures import ThreadPoolExecutor
import sys

opts = {'max_workers': None}
if sys.version_info >= (3, 6):
    opts['thread_name_prefix'] = 'SyncWorker'
executor = ThreadPoolExecutor(**opts)

...yields:

mypy_bug.py:6: error: Incompatible types in assignment (expression has type "str", target has type "None")
mypy_bug.py:7: error: Argument 1 to "ThreadPoolExecutor" has incompatible type "**Dict[str, None]"; expected "str"

Let's ignore the first error for now. But the second seems to have an incorrect message, specifically I think the **Dict[str, None] in it is weird. Just guessing, is it trying to assign something to the ThreadPoolExecutor's thread_name_prefix arg which is a str? Maybe the None from max_workers because passing kwargs as a dict like this might result in random/varying order of the dict items? But if that's the case, I'd expect it to say something else than **Dict[str, None] as the culprit, maybe None or int, but not the whole **dict.

Another thought is that when kwargs is passed as **dicts like this, their order is not really defined, so "Argument 1" isn't very helpful. Would be better to say the argument's name.

But maybe I'm just all off track here -- will leave the rest to someone who actually has an informed opinion :)

@scop
Copy link
Contributor Author

scop commented Jul 22, 2018

(I just tried defining opts as OrderedDict and the error went away, but anyway I think the points/suggestions above still hold.)

@radzak
Copy link

radzak commented Jul 22, 2018

@scop Hey, take a look at this SO question. It should help you understand why you're getting the first error.

The answer to this question will explain you that the type of the opts dictionary mypy inferred is Dict[str, None]. If you take a look at a stub for ThreadPoolExecutor, you can notice that the expected type for thread_name_prefix is str:

        def __init__(self, max_workers: Optional[int] = ...,
                     thread_name_prefix: str = ...) -> None: ...

but mypy thinks you're passing **Dict[str, None], which causes the incompatible type error (None is not str).

@akaptur
Copy link

akaptur commented Aug 30, 2018

I've hit the same issue with passing a dict of kwargs. I'm also on mypy 0.620, python 3.6.5. Here's a small repro:

def test(a: int = 1, b: str = 'hi', c: bool = True) -> None:
    return None

kwargs = {"a": 1, "b": "hello", "c": False}
test(**kwargs)

This code is correct at runtime, but generates the following mypy errors:

test.py:9: error: Argument 1 to "test" has incompatible type "**Dict[str, object]"; expected "int"
test.py:9: error: Argument 1 to "test" has incompatible type "**Dict[str, object]"; expected "str"
test.py:9: error: Argument 1 to "test" has incompatible type "**Dict[str, object]"; expected "bool"

I can suppress this error by either:

  • annotating kwargs as Dict[str, Any]
  • adding a #type: ignore
  • calling the function with the kwargs specified (test(a=1, b="hello", c=False))

Something that I might expect to help, but doesn't, is annotating kwargs as Dict[str, Union[str, bool, int]].

The first two ways are not really fixes, and the third is not always an option. Is there a good way to keep mypy from emitting this false positive?

@ilevkivskyi
Copy link
Member

Is there a good way to keep mypy from emitting this false positive?

class Options(TypedDict):
    a: int
    b: str
    c: bool

kwargs: Options = {"a": 1, "b": "hello", "c": False}
test(**kwargs)  # OK

Note however that this may trade false positive for false negative because of #5198

@ilevkivskyi
Copy link
Member

Also this is not really a false positive. There is no way mypy can guarantee that a key will be e.g. deleted from this dictionary (it will however prevent you from deleting keys if this is a TypedDict).

@ilevkivskyi
Copy link
Member

This happened again, so raising priority to normal. Btw we can at least infer TypedDict types for dict literals if they appear directly in ** context in function call. Finally, the error message should be more clear (explaining that a variable is of type dict with unknown keys, and maybe suggesting TypedDict).

@ilevkivskyi
Copy link
Member

This appeared three more times, so raising priority to high.

@w-p
Copy link

w-p commented Feb 9, 2020

Hit this today, inconsistently, in several places. I can confirm that adding an explicit Dict[str, Any] resolves the issue but it's still unclear why.

@stevenjackson121
Copy link

If the general problem is hard to solve. Would it be easier (and still valuable) to add support for something like:

# OK iff test(a=1,b="hello",c=False) is OK
kwargs: Final[frozendict] = frozendict({"a": 1, "b": "hello", "c": False})
test(**kwargs)  

Final[frozendict] removes the need to handle any changes to kwargs between definition and usage (or the need to infer changes arent made), and would cover all my use cases (described in more detail in #8485)

@ilevkivskyi
Copy link
Member

Supporting frozen typed dicts is also a possible solution. We can even support both things (this and the older idea of inferring typed dict types for dict literals appearing directly in a call).

Would it be easier (and still valuable) to add support for something like

The core team has very limited resources, if you want to work on this, please go ahead.

@patrick91
Copy link
Contributor

I got this today as well, using a TypedDict fixed the mypy error :)

@zachliu
Copy link

zachliu commented May 13, 2021

I got this today as well. Unfortunately using a TypedDict doesn't work in my case 😢

@type_checked_constructor(skip=True)
@dataclass
class MyOtherStuff():
    ...

@type_checked_constructor(skip=True)
@dataclass
class File():
    ...

@type_checked_constructor(skip=True)
@dataclass
class MyStuff():
    """MyStuff Dataclass"""
    kind: str
    is_ready: bool
    request: MyOtherStuff
    files: Optional[List[File]]

data = {...}

MyStuff(**data)

causes a new error
error: Expected keyword arguments, {...}, or dict(...) in TypedDict constructor

Using mypy==0.782 with Python 3.8.4

MrMino added a commit to MrMino/wheelfile that referenced this issue Jul 25, 2021
@derrix060
Copy link

Similar to #8890

@wereii
Copy link

wereii commented Jan 7, 2022

I've just hit this error with dataclasses

Example

from dataclasses import dataclass


@dataclass(slots=True, kw_only=True, frozen=True)
class TestClass:
    ID: int
    name: str
    domain: str
    secure: bool


data = {
    "ID": 1,
    "name": "example",
    "domain": "example.com",
    "secure": True,
}


TestClass(**data)

Output

> mypy test_my.py
test_my.py:20: error: Argument 1 to "TestClass" has incompatible type "**Dict[str, object]"; expected "int"
test_my.py:20: error: Argument 1 to "TestClass" has incompatible type "**Dict[str, object]"; expected "str"
test_my.py:20: error: Argument 1 to "TestClass" has incompatible type "**Dict[str, object]"; expected "bool"
Found 3 errors in 1 file (checked 1 source file)

Versions

Python 3.10.1 (main, Dec 18 2021, 23:53:45) [GCC 11.1.0]
mypy 0.930

@JelleZijlstra JelleZijlstra added the topic-calls Function calls, *args, **kwargs, defaults label Mar 19, 2022
@adamwojt
Copy link

I run into this problem today with below case:

error_kwargs = {
    "tags": {
        "foo": bar,
    },
    "contexts": {"foo": {"bar": "bar"}},
}
if error_msg in self.non_retryable_errors:
    raise NonRetryableResponseError(**error_kwargs)
else:
    raise RetryableResponseError(**error_kwargs)

I chose to just annotate error_kwargs with Dict[str, Any] as TypedDict seemed like overkill for me.

@mastercoms
Copy link

Hit this with pydantic's create_model:

fields = {
  "test": (str, "")
}
create_model("model", **fields)

@aripollak
Copy link

aripollak commented Nov 15, 2023

We can now at least check kwargs with differing types in a function definition by using Unpack with TypedDict, though the error message without that is still a little confusing. https://github.com/python/mypy/blob/master/CHANGELOG.md#mypy-17

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature priority-0-high topic-calls Function calls, *args, **kwargs, defaults topic-usability
Projects
None yet
Development

No branches or pull requests