Infer ParamSpec constraint from arguments

[ilevkivskyi]

Fixes #12278
Fixes #13191 (more tricky nested use cases with optional/keyword args still don't work, but they are quite tricky to fix and may selectively fixed later)

This unfortunately requires some special-casing, here is its summary:

• If actual argument for Callable[P, T] is non-generic and non-lambda, do not put it into inference second pass.
• If we are able to infer constraints for P without using arguments mapped to *args: P.args etc., do not add the constraint for P vs those arguments (this applies to both top-level callable constraints, and for nested callable constraints against callables that are known to have imprecise argument kinds).

(Btw TODO I added is not related to this PR, I just noticed something obviously wrong)

[ilevkivskyi]

OK, I have tried to tweak the special-casing after looking at the mypy_primer output (and updated PR description). Unfortunately, it looks like it is not a 100% improvement, we need to make trade-off. But overall I think it is a good trade-off, we are fixing a bunch of issues, while causing only few regressions.

[ilevkivskyi]

To summarize on the mypy_primer:

• anyio, steam.py, prefect: error moves from one argument to other on different line
• Expression, dicord.py, pydantic: a bunch of fixes
• pytest: used to work by accident because we use first overload for inference, and this is the overload user wants. Now it fails because of an arg_kind mismatch that is tricky to fix (without creating significant unsafety)

IMO this is a clear net positive.

[ilevkivskyi]

@JukkaL Could you please also check this one? I remember you were interested in this issue.

[JukkaL]

Nice, this fixes several ParamSpec use cases, some of which seem pretty fundamental. I think that the special casings are justified and can't think of a more general approach.

[JukkaL]

Does this work with other kinds of arguments, such as positional, or more than one argument? Maybe also test these, unless they are covered elsewhere.

[ilevkivskyi]

I think positional should work, and multiple args too. In general, for "first order" calls like apply(fn, arg1, arg2) practically everything should work. However, as I mentioned in the PR description for "higher order" calls like apply(apply, fn, arg1, ag2) some combinations of formal/actual kinds are tricky to support, and they will not work with this PR.

[ilevkivskyi]

(to clarify, those "higher order" calls currently don't work either, and this PR will partially fix it, in the sense that inferred types, will be correct, but mypy will complain about subtyping because of the inferred argument kinds)

[ilevkivskyi]

Hm, actually I think I found a small tweak in subtyping that may allow more "higher order" calls with a slight reduction in safety (that only affects imprecise inferred kinds). Let me try it.

[ilevkivskyi]

No, it looks too arbitrary. Let's wait a bit more and see which cases people will complain about. After we will gather more data, we can tune relaxed subtyping for callables with imprecise_arg_kinds=True.

[github-actions]

Diff from mypy_primer, showing the effect of this PR on open source code:

anyio (https://github.com/agronholm/anyio)
+ src/anyio/from_thread.py:395: error: Argument 1 to "submit" of "Executor" has incompatible type "Callable[[Callable[..., Awaitable[T_Retval]], VarArg(object), DefaultNamedArg(str, 'backend'), DefaultNamedArg(dict[str, Any] | None, 'backend_options')], T_Retval]"; expected "Callable[[Callable[[], Coroutine[Any, Any, None]], str, dict[str, Any] | None], T_Retval]"  [arg-type]
+ src/anyio/from_thread.py:396: error: Unused "type: ignore" comment  [unused-ignore]

Expression (https://github.com/cognitedata/Expression)
- tests/test_fn.py:15: error: Argument 1 to "TailCall" has incompatible type "int"; expected <nothing>  [arg-type]
- tests/test_fn.py:15: error: Argument 2 to "TailCall" has incompatible type "int"; expected <nothing>  [arg-type]

steam.py (https://github.com/Gobot1234/steam.py)
- steam/ext/commands/commands.py:401: error: Argument 2 to "maybe_coroutine" has incompatible type "Context[Any]"; expected "MC"  [arg-type]
+ steam/ext/commands/commands.py:401: error: Argument 1 to "maybe_coroutine" has incompatible type "CheckReturnType"; expected "Callable[[Context[Any]], MC | Awaitable[MC]]"  [arg-type]
+ steam/ext/commands/commands.py:401: note: "CheckReturnType.__call__" has type "Callable[[MC], MC]"
- steam/ext/commands/bot.py:474: error: Argument 2 to "maybe_coroutine" has incompatible type "Context[Any]"; expected "MC"  [arg-type]
+ steam/ext/commands/bot.py:474: error: Argument 1 to "maybe_coroutine" has incompatible type "CheckReturnType"; expected "Callable[[Context[Any]], MC | Awaitable[MC]]"  [arg-type]
+ steam/ext/commands/bot.py:474: note: "CheckReturnType.__call__" has type "Callable[[MC], MC]"

prefect (https://github.com/PrefectHQ/prefect)
+ src/prefect/engine.py:1131: error: Argument 1 to "create_call" has incompatible type "function"; expected "Callable[[Task[Any, Any], FlowRunContext, dict[str, Any], Iterable[PrefectFuture[Any, Any]] | None, Literal['future', 'state', 'result'], BaseTaskRunner | None], <nothing>]"  [arg-type]
- src/prefect/engine.py:1131: error: Argument 1 to "create_call" has incompatible type "function"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]"  [arg-type]
- src/prefect/engine.py:1132: error: Argument "task" to "create_call" has incompatible type "Task[Any, Any]"; expected <nothing>  [arg-type]
- src/prefect/engine.py:1134: error: Argument "parameters" to "create_call" has incompatible type "dict[str, Any]"; expected <nothing>  [arg-type]
- src/prefect/engine.py:1135: error: Argument "wait_for" to "create_call" has incompatible type "Iterable[PrefectFuture[Any, Any]] | None"; expected <nothing>  [arg-type]
- src/prefect/engine.py:1136: error: Argument "return_type" to "create_call" has incompatible type "Literal['future', 'state', 'result']"; expected <nothing>  [arg-type]
- src/prefect/engine.py:1137: error: Argument "task_runner" to "create_call" has incompatible type "BaseTaskRunner | None"; expected <nothing>  [arg-type]

pytest (https://github.com/pytest-dev/pytest)
+ src/_pytest/_py/path.py:758: error: Argument 1 has incompatible type overloaded function; expected "Callable[[str, Any, Any], TextIOWrapper]"  [arg-type]
+ src/_pytest/_py/path.py:1270: error: Argument 1 has incompatible type overloaded function; expected "Callable[[str], str]"  [arg-type]

discord.py (https://github.com/Rapptz/discord.py)
- discord/ext/commands/core.py:1279: error: Argument 2 to "maybe_coroutine" has incompatible type "Context[BotT]"; expected "Context[BotT]"  [arg-type]
- discord/ext/commands/hybrid.py:415: error: Argument 2 to "maybe_coroutine" has incompatible type "Context[Bot]"; expected "Context[BotT]"  [arg-type]

pydantic (https://github.com/samuelcolvin/pydantic)
- pydantic/v1/class_validators.py:148: error: Need type annotation for "f_cls"  [var-annotated]