Change `cache`/`lru_cache` typing to retain function signatures #12945

alwaysmpe · 2024-11-03T07:25:45Z

My attempt at fixing the cache & lru_cache typing problem. Using the newer generics syntax I get the same result in both mypy and pyright. Using the older typevar syntax (which I'm less familiar with) I get different errors. See my comment for the newer generics approach.

Using these type stubs in pyright the below code checks as expected, mypy thinks some of the type invariance should be different (update: I've pushed changes to match mypy's preference)

Update: think I've worked it out, converting to draft while making changes.

from functools import cache

class CFnCls:

    @cache
    def fn(self, arg: int) -> int:
        print("method fn called")
        return arg

    @classmethod
    @cache
    def cls_fn(cls, arg: int) -> int:
        print("class fn called")
        return arg

    @staticmethod
    @cache
    def st_fn(arg: int) -> int:
        print("static fn called")
        return arg

cfn_inst = CFnCls()
cfn_inst.fn(1)
CFnCls.st_fn(1)
CFnCls.cls_fn(1)
cfn_inst.fn(1)
CFnCls.st_fn(1)
CFnCls.cls_fn(1)
assert_type(cfn_inst.fn(1), int)
assert_type(CFnCls.st_fn(1), int)
assert_type(CFnCls.cls_fn(1), int)
CFnCls().fn.cache_clear()
CFnCls.fn.cache_clear()
CFnCls.st_fn.cache_clear()
CFnCls.cls_fn.cache_clear()
if TYPE_CHECKING:
    CFnCls().fn(1, 1) # type error - correct
    CFnCls.st_fn(1, 1) # type error - correct
    CFnCls.cls_fn(1, 1) # type error - correct

@cache
def fn(arg: int) -> int:
    return arg

@cache
def df_fn(arg: int, darg: str = "default"):
    print("default fn called")
    return darg
df_fn(1)

fn(1)
assert_type(fn(1), int)
if TYPE_CHECKING:
    fn(1, 2) # type error - correct
fn.cache_clear()


@overload
@cache
def fn_overload(arg: int) -> int:
    ...
@overload
@cache
def fn_overload(arg: str) -> str:
    ...
@cache
def fn_overload(arg: int | str) -> int | str:
    return arg

fn_overload(1)
fn_overload("1")
# behaves same as previous cache, merges overloads
assert_type(fn_overload(1), int | str)
assert_type(fn_overload("1"), int | str)
fn_overload.cache_clear()
if TYPE_CHECKING:
    fn_overload({1,2}) # type error - correct

class Unhashable:
    @override
    def __eq__(self, value: object) -> bool:
        return False

@cache
def no_cache(arg: Unhashable, arg2: int) -> None:
    pass

if TYPE_CHECKING:
    # type error in pyright - correct
    # no type error in mypy?
    no_cache(Unhashable(), 2)

Define overloads for cache/lru_cache that are constrained by the signature of the function they're called with. This assumes self as a first parameter is a function method, cls is a classmethod and everything else are either static methods or functions.

Then direct each overload to a descriptor protocol that mirrors the binding behaviour of methods/functions/classmethods.

The returned type is then a union with a callable that takes arbitrary arguments that are hashable. This means that the previous hashable requirement is retained (type checkers don't know which type was returned so calls must satisfy both, the union takes any arguments as long as they're hashable while the descriptor protocol mirrors the cached function's behaviour).

Define overloads for `cache`/`lru_cache` that are constrained by the signature of the function they're called with. This assumes `self` as a first parameter is a function method, `cls` is a classmethod and everything else are either static methods or functions. Then direct each overload to a descriptor protocol that mirrors the binding behaviour of methods/functions/classmethods. The returned type is then a union with a callable that takes arbitrary arguments that are hashable. This means that the previous hashable requirement is retained (type checkers don't know which type was returned so calls must satisfy both, the union takes any arguments as long as they're hashable while the descriptor protocol mirrors the cached function's behaviour).

alwaysmpe · 2024-11-03T07:40:08Z

Some of the errors so far:
I've used concatenate which was added in 3.10, so need to keep the old version and enable this on 3.10
I've followed pyright on type invariance and it disagrees with mypy, so need to change to follow mypy convention

alwaysmpe · 2024-11-03T09:03:53Z

I've fixed the version gate and invariance. Of the issues that primer raises, there are a lot of type errors due to incompatible parameter types which is the point of this change. There are some Signature of "..." incompatible with supertype errors, picking one at random:

manticore (https://github.com/trailofbits/manticore)
+ manticore/core/smtlib/solver.py:505: error: Signature of "can_be_true" incompatible with supertype "Solver"  [override]
+ manticore/core/smtlib/solver.py:505: note:      Superclass:
+ manticore/core/smtlib/solver.py:505: note:          def can_be_true(self, constraints: Any, expression: Any = ...) -> bool
+ manticore/core/smtlib/solver.py:505: note:      Subclass:
+ manticore/core/smtlib/solver.py:505: note:          _MethodDescriptor[SMTLIBSolver, [ConstraintSet, bool | Bool], bool] | _FnHashable[[SMTLIBSolver, ConstraintSet, bool | Bool], bool]
+ manticore/core/smtlib/solver.py:605: error: Signature of "get_all_values" incompatible with supertype "Solver"  [override]
+ manticore/core/smtlib/solver.py:605: note:      Superclass:
+ manticore/core/smtlib/solver.py:605: note:          def get_all_values(self, constraints: Any, x: Any, maxcnt: Any = ..., silent: Any = ...) -> Any
+ manticore/core/smtlib/solver.py:605: note:      Subclass:
+ manticore/core/smtlib/solver.py:605: note:          _MethodDescriptor[SMTLIBSolver, [ConstraintSet, Any, int | None, bool], Any] | _FnHashable[[SMTLIBSolver, ConstraintSet, Any, int | None, bool], Any]

It looks like the parent function is untyped so mypy infers Any, the subclass function is typed meaning more constrained than parent so breaks liskov substitution.

alwaysmpe · 2024-11-03T09:07:53Z

Looking through the primer errors, the only thing that looks like a problem is the subclass method type invariance error eg:

core (https://github.com/home-assistant/core)
+ homeassistant/util/unit_conversion.py:367: error: Signature of "converter_factory" incompatible with supertype "BaseUnitConverter"  [override]
+ homeassistant/util/unit_conversion.py:367: note:      Superclass:
+ homeassistant/util/unit_conversion.py:367: note:          _ClassmethodDescriptor[BaseUnitConverter, [str | None, str | None], Callable[[float], float]] | _ClsHashable[BaseUnitConverter, [str | None, str | None], Callable[[float], float]]
+ homeassistant/util/unit_conversion.py:367: note:      Subclass:
+ homeassistant/util/unit_conversion.py:367: note:          _ClassmethodDescriptor[SpeedConverter, [str | None, str | None], Callable[[float], float]] | _ClsHashable[SpeedConverter, [str | None, str | None], Callable[[float], float]]

But hopefully someone else knows how to fix that.

alwaysmpe · 2024-11-03T09:10:56Z

I don't yet understand why this is raising an error, it looks correct:

pip (https://github.com/pypa/pip)
+ src/pip/_internal/resolution/resolvelib/provider.py:242: error: Signature of "is_satisfied_by" incompatible with supertype "AbstractProvider"  [override]
+ src/pip/_internal/resolution/resolvelib/provider.py:242: note:      Superclass:
+ src/pip/_internal/resolution/resolvelib/provider.py:242: note:          def is_satisfied_by(self, requirement: Requirement, candidate: Candidate) -> bool
+ src/pip/_internal/resolution/resolvelib/provider.py:242: note:      Subclass:
+ src/pip/_internal/resolution/resolvelib/provider.py:242: note:          _MethodDescriptor[PipProvider, [Requirement, Candidate], bool] | _FnHashable[[PipProvider, Requirement, Candidate], bool]

Investigating

alwaysmpe · 2024-11-03T09:20:44Z

This is possibly a problem with type invariance:

psycopg (https://github.com/psycopg/psycopg)
+ psycopg/psycopg/types/enum.py:156: error: Incompatible types in assignment (expression has type "type[_BaseEnumLoader[E@register_enum]] | type[_BaseEnumLoader[E@_make_binary_loader]]", variable has type "type[_BaseEnumLoader[E@register_enum]] | type[_BaseEnumLoader[E@_make_loader]]")  [assignment]
+ psycopg/psycopg/types/enum.py:164: error: Incompatible types in assignment (expression has type "type[_BaseEnumDumper[E@register_enum]] | type[_BaseEnumDumper[E@_make_binary_dumper]]", variable has type "type[_BaseEnumDumper[E@register_enum]] | type[_BaseEnumDumper[E@_make_dumper]]")  [assignment]

alwaysmpe · 2024-11-03T09:24:26Z

At this point I think I need the opinion of someone more knowledgeable than me, but a lot of those errors seem to be new true positives which is good.

alwaysmpe · 2024-11-03T09:40:29Z

I've got no idea how to fix the below, seems to be the error in stdlib stubtest

note: unused allowlist entry functools._lru_cache_wrapper.cache_parameters
Found 1 error (checked 729 modules)

update: I've removed them, will see what that does.

alwaysmpe · 2024-11-03T09:51:56Z

I think in the case of inheritance, if both parent and child functions are cached the _ClassmethodDescriptor type parameters are invariant and I don't have a good enough understanding of invariance/covariance etc to know how to fix that.

github-actions · 2024-11-03T10:21:19Z

Diff from mypy_primer, showing the effect of this PR on open source code:

dacite (https://github.com/konradhalas/dacite)
+ dacite/cache.py:11: error: Unused "type: ignore" comment  [unused-ignore]

pylint (https://github.com/pycqa/pylint)
+ pylint/checkers/utils.py:31: error: Module "functools" has no attribute "_lru_cache_wrapper"  [attr-defined]

psycopg (https://github.com/psycopg/psycopg)
+ psycopg/psycopg/rows.py:143: error: Argument 2 to "__call__" of "_FunctionDescriptor" has incompatible type "*Generator[bytes | None, None, None]"; expected "bytes"  [arg-type]
+ psycopg/psycopg/types/enum.py:156: error: Incompatible types in assignment (expression has type "type[_BaseEnumLoader[E@register_enum]] | type[_BaseEnumLoader[E@_make_binary_loader]]", variable has type "type[_BaseEnumLoader[E@register_enum]] | type[_BaseEnumLoader[E@_make_loader]]")  [assignment]
+ psycopg/psycopg/types/enum.py:164: error: Incompatible types in assignment (expression has type "type[_BaseEnumDumper[E@register_enum]] | type[_BaseEnumDumper[E@_make_binary_dumper]]", variable has type "type[_BaseEnumDumper[E@register_enum]] | type[_BaseEnumDumper[E@_make_dumper]]")  [assignment]

prefect (https://github.com/PrefectHQ/prefect)
- src/prefect/settings/legacy.py:105: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "type[PrefectBaseSettings]"; expected "Hashable"  [arg-type]
- src/prefect/settings/legacy.py:105: note: Following member(s) of "PrefectBaseSettings" have conflicts:
- src/prefect/settings/legacy.py:105: note:     Expected:
- src/prefect/settings/legacy.py:105: note:         def __hash__() -> int
- src/prefect/settings/legacy.py:105: note:     Got:
- src/prefect/settings/legacy.py:105: note:         def __hash__(self: object) -> int
- src/prefect/settings/legacy.py:105: note:     Expected:
- src/prefect/settings/legacy.py:105: note:         def __hash__() -> int
- src/prefect/settings/legacy.py:105: note:     Got:
- src/prefect/settings/legacy.py:105: note:         def __hash__(self: object) -> int
- src/prefect/settings/legacy.py:136: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "type[PrefectBaseSettings]"; expected "Hashable"  [arg-type]
+ src/prefect/settings/legacy.py:136: error: Argument 1 to "__call__" of "_FnHashable" has incompatible type "type[PrefectBaseSettings]"; expected "Hashable"  [arg-type]

ibis (https://github.com/ibis-project/ibis)
+ ibis/backends/tests/test_temporal.py:2105: error: Argument "exclude" to "__call__" of "_FunctionDescriptor" has incompatible type "tuple[str, str]"; expected "tuple[str]"  [arg-type]
+ ibis/backends/tests/test_temporal.py:2123: error: Argument "exclude" to "__call__" of "_FunctionDescriptor" has incompatible type "tuple[str, str, str, str, str, str]"; expected "tuple[str]"  [arg-type]

mitmproxy (https://github.com/mitmproxy/mitmproxy)
+ mitmproxy/proxy/mode_specs.py:167: error: Argument 2 to "__get__" of "_ClassmethodDescriptor" has incompatible type "type[ProxyMode]"; expected "type[Never]"  [arg-type]
+ mitmproxy/proxy/mode_specs.py:167: error: Argument 2 to "__get__" of "_ClsHashable" has incompatible type "type[ProxyMode]"; expected "type[Never]"  [arg-type]
+ mitmproxy/connection.py:193: error: Argument 2 to "__get__" of "_ClassmethodDescriptor" has incompatible type "type[ProxyMode]"; expected "type[Never]"  [arg-type]
+ mitmproxy/connection.py:193: error: Argument 2 to "__get__" of "_ClsHashable" has incompatible type "type[ProxyMode]"; expected "type[Never]"  [arg-type]
+ mitmproxy/utils/asyncio_utils.py:62: error: Argument 1 to "__call__" of "_FunctionDescriptor" has incompatible type "Any | str"; expected "tuple[Any, ...] | None"  [arg-type]
+ mitmproxy/tools/console/common.py:811: error: Argument "error_message" to "__call__" of "_FunctionDescriptor" has incompatible type "str | None"; expected "str"  [arg-type]
+ mitmproxy/tools/console/common.py:848: error: Incompatible types in assignment (expression has type "_FunctionDescriptor[[NamedArg(RenderMode, 'render_mode'), NamedArg(bool, 'focused'), NamedArg(str, 'marked'), NamedArg(str | None, 'is_replay'), NamedArg(str, 'request_method'), NamedArg(str, 'request_scheme'), NamedArg(str, 'request_host'), NamedArg(str, 'request_path'), NamedArg(str, 'request_url'), NamedArg(str, 'request_http_version'), NamedArg(float, 'request_timestamp'), NamedArg(bool, 'request_is_push_promise'), NamedArg(bool, 'intercepted'), NamedArg(int | None, 'response_code'), NamedArg(str | None, 'response_reason'), NamedArg(int | None, 'response_content_length'), NamedArg(str | None, 'response_content_type'), NamedArg(float | None, 'duration'), NamedArg(str | None, 'error_message')], Any] | _FnHashable[[NamedArg(RenderMode, 'render_mode'), NamedArg(bool, 'focused'), NamedArg(str, 'marked'), NamedArg(str | None, 'is_replay'), NamedArg(str, 'request_method'), NamedArg(str, 'request_scheme'), NamedArg(str, 'request_host'), NamedArg(str, 'request_path'), NamedArg(str, 'request_url'), NamedArg(str, 'request_http_version'), NamedArg(float, 'request_timestamp'), NamedArg(bool, 'request_is_push_promise'), NamedArg(bool, 'intercepted'), NamedArg(int | None, 'response_code'), NamedArg(str | None, 'response_reason'), NamedArg(int | None, 'response_content_length'), NamedArg(str | None, 'response_content_type'), NamedArg(float | None, 'duration'), NamedArg(str | None, 'error_message')], Any]", variable has type "_FunctionDescriptor[[NamedArg(RenderMode, 'render_mode'), NamedArg(bool, 'focused'), NamedArg(str, 'marked'), NamedArg(bool, 'is_replay'), NamedArg(str, 'request_method'), NamedArg(str, 'request_scheme'), NamedArg(str, 'request_host'), NamedArg(str, 'request_path'), NamedArg(str, 'request_url'), NamedArg(str, 'request_http_version'), NamedArg(float, 'request_timestamp'), NamedArg(bool, 'request_is_push_promise'), NamedArg(bool, 'intercepted'), NamedArg(int | None, 'response_code'), NamedArg(str | None, 'response_reason'), NamedArg(int | None, 'response_content_length'), NamedArg(str | None, 'response_content_type'), NamedArg(float | None, 'duration'), NamedArg(str | None, 'error_message')], Any] | _FnHashable[[NamedArg(RenderMode, 'render_mode'), NamedArg(bool, 'focused'), NamedArg(str, 'marked'), NamedArg(bool, 'is_replay'), NamedArg(str, 'request_method'), NamedArg(str, 'request_scheme'), NamedArg(str, 'request_host'), NamedArg(str, 'request_path'), NamedArg(str, 'request_url'), NamedArg(str, 'request_http_version'), NamedArg(float, 'request_timestamp'), NamedArg(bool, 'request_is_push_promise'), NamedArg(bool, 'intercepted'), NamedArg(int | None, 'response_code'), NamedArg(str | None, 'response_reason'), NamedArg(int | None, 'response_content_length'), NamedArg(str | None, 'response_content_type'), NamedArg(float | None, 'duration'), NamedArg(str | None, 'error_message')], Any]")  [assignment]
+ mitmproxy/tools/console/common.py:853: error: Argument "is_replay" to "__call__" of "_FunctionDescriptor" has incompatible type "str | None"; expected "bool"  [arg-type]
+ mitmproxy/test/tflow.py:134: error: Argument 2 to "__get__" of "_ClassmethodDescriptor" has incompatible type "type[ProxyMode]"; expected "type[Never]"  [arg-type]
+ mitmproxy/test/tflow.py:134: error: Argument 2 to "__get__" of "_ClsHashable" has incompatible type "type[ProxyMode]"; expected "type[Never]"  [arg-type]
+ mitmproxy/test/tflow.py:235: error: Argument 2 to "__get__" of "_ClassmethodDescriptor" has incompatible type "type[ProxyMode]"; expected "type[Never]"  [arg-type]
+ mitmproxy/test/tflow.py:235: error: Argument 2 to "__get__" of "_ClsHashable" has incompatible type "type[ProxyMode]"; expected "type[Never]"  [arg-type]
+ mitmproxy/proxy/server.py:582: error: Argument 2 to "__get__" of "_ClassmethodDescriptor" has incompatible type "type[ProxyMode]"; expected "type[Never]"  [arg-type]
+ mitmproxy/proxy/server.py:582: error: Argument 2 to "__get__" of "_ClsHashable" has incompatible type "type[ProxyMode]"; expected "type[Never]"  [arg-type]
+ mitmproxy/proxy/mode_servers.py:115: error: Argument 2 to "__get__" of "_ClassmethodDescriptor" has incompatible type "type[ProxyMode]"; expected "type[Never]"  [arg-type]
+ mitmproxy/proxy/mode_servers.py:115: error: Argument 2 to "__get__" of "_ClsHashable" has incompatible type "type[ProxyMode]"; expected "type[Never]"  [arg-type]
+ mitmproxy/addons/clientplayback.py:98: error: Argument 2 to "__get__" of "_ClassmethodDescriptor" has incompatible type "type[UpstreamMode]"; expected "type[Never]"  [arg-type]
+ mitmproxy/addons/clientplayback.py:98: error: Argument 2 to "__get__" of "_ClsHashable" has incompatible type "type[UpstreamMode]"; expected "type[Never]"  [arg-type]
+ mitmproxy/addons/clientplayback.py:98: error: Need type annotation for "mode"  [var-annotated]
+ mitmproxy/addons/proxyserver.py:109: error: Argument 2 to "__get__" of "_ClassmethodDescriptor" has incompatible type "type[ProxyMode]"; expected "type[Never]"  [arg-type]
+ mitmproxy/addons/proxyserver.py:109: error: Argument 2 to "__get__" of "_ClsHashable" has incompatible type "type[ProxyMode]"; expected "type[Never]"  [arg-type]
+ mitmproxy/addons/proxyserver.py:255: error: Argument 2 to "__get__" of "_ClassmethodDescriptor" has incompatible type "type[ProxyMode]"; expected "type[Never]"  [arg-type]
+ mitmproxy/addons/proxyserver.py:255: error: Argument 2 to "__get__" of "_ClsHashable" has incompatible type "type[ProxyMode]"; expected "type[Never]"  [arg-type]
+ mitmproxy/addons/proxyserver.py:299: error: Argument 2 to "__get__" of "_ClassmethodDescriptor" has incompatible type "type[ProxyMode]"; expected "type[Never]"  [arg-type]
+ mitmproxy/addons/proxyserver.py:299: error: Argument 2 to "__get__" of "_ClsHashable" has incompatible type "type[ProxyMode]"; expected "type[Never]"  [arg-type]
+ mitmproxy/master.py:135: error: Argument 2 to "__get__" of "_ClassmethodDescriptor" has incompatible type "type[ReverseMode]"; expected "type[Never]"  [arg-type]
+ mitmproxy/master.py:135: error: Argument 2 to "__get__" of "_ClsHashable" has incompatible type "type[ReverseMode]"; expected "type[Never]"  [arg-type]
+ mitmproxy/master.py:135: error: Need type annotation for "mode"  [var-annotated]

rich (https://github.com/Textualize/rich)
+ rich/progress_bar.py:145: error: Argument 3 to "__call__" of "_FunctionDescriptor" has incompatible type "str | None"; expected "str"  [arg-type]

pip (https://github.com/pypa/pip)
+ src/pip/_internal/index/package_finder.py:902: error: Argument 1 to "__call__" of "_FunctionDescriptor" has incompatible type "str | None"; expected "str"  [arg-type]
+ src/pip/_internal/resolution/resolvelib/provider.py:242: error: Signature of "is_satisfied_by" incompatible with supertype "AbstractProvider"  [override]
+ src/pip/_internal/resolution/resolvelib/provider.py:242: note:      Superclass:
+ src/pip/_internal/resolution/resolvelib/provider.py:242: note:          def is_satisfied_by(self, requirement: Requirement, candidate: Candidate) -> bool
+ src/pip/_internal/resolution/resolvelib/provider.py:242: note:      Subclass:
+ src/pip/_internal/resolution/resolvelib/provider.py:242: note:          _MethodDescriptor[PipProvider, [Requirement, Candidate], bool] | _FnHashable[[PipProvider, Requirement, Candidate], bool]

manticore (https://github.com/trailofbits/manticore)
+ manticore/core/smtlib/solver.py:505: error: Signature of "can_be_true" incompatible with supertype "Solver"  [override]
+ manticore/core/smtlib/solver.py:505: note:      Superclass:
+ manticore/core/smtlib/solver.py:505: note:          def can_be_true(self, constraints: Any, expression: Any = ...) -> bool
+ manticore/core/smtlib/solver.py:505: note:      Subclass:
+ manticore/core/smtlib/solver.py:505: note:          _MethodDescriptor[SMTLIBSolver, [ConstraintSet, bool | Bool], bool] | _FnHashable[[SMTLIBSolver, ConstraintSet, bool | Bool], bool]
+ manticore/core/smtlib/solver.py:605: error: Signature of "get_all_values" incompatible with supertype "Solver"  [override]
+ manticore/core/smtlib/solver.py:605: note:      Superclass:
+ manticore/core/smtlib/solver.py:605: note:          def get_all_values(self, constraints: Any, x: Any, maxcnt: Any = ..., silent: Any = ...) -> Any
+ manticore/core/smtlib/solver.py:605: note:      Subclass:
+ manticore/core/smtlib/solver.py:605: note:          _MethodDescriptor[SMTLIBSolver, [ConstraintSet, Any, int | None, bool], Any] | _FnHashable[[SMTLIBSolver, ConstraintSet, Any, int | None, bool], Any]

core (https://github.com/home-assistant/core)
+ homeassistant/util/unit_conversion.py:367: error: Signature of "converter_factory" incompatible with supertype "BaseUnitConverter"  [override]
+ homeassistant/util/unit_conversion.py:367: note:      Superclass:
+ homeassistant/util/unit_conversion.py:367: note:          _ClassmethodDescriptor[BaseUnitConverter, [str | None, str | None], Callable[[float], float]] | _ClsHashable[BaseUnitConverter, [str | None, str | None], Callable[[float], float]]
+ homeassistant/util/unit_conversion.py:367: note:      Subclass:
+ homeassistant/util/unit_conversion.py:367: note:          _ClassmethodDescriptor[SpeedConverter, [str | None, str | None], Callable[[float], float]] | _ClsHashable[SpeedConverter, [str | None, str | None], Callable[[float], float]]
+ homeassistant/util/unit_conversion.py:381: error: Signature of "converter_factory_allow_none" incompatible with supertype "BaseUnitConverter"  [override]
+ homeassistant/util/unit_conversion.py:381: note:      Superclass:
+ homeassistant/util/unit_conversion.py:381: note:          _ClassmethodDescriptor[BaseUnitConverter, [str | None, str | None], Callable[[float | None], float | None]] | _ClsHashable[BaseUnitConverter, [str | None, str | None], Callable[[float | None], float | None]]
+ homeassistant/util/unit_conversion.py:381: note:      Subclass:
+ homeassistant/util/unit_conversion.py:381: note:          _ClassmethodDescriptor[SpeedConverter, [str | None, str | None], Callable[[float | None], float | None]] | _ClsHashable[SpeedConverter, [str | None, str | None], Callable[[float | None], float | None]]
+ homeassistant/util/unit_conversion.py:447: error: Signature of "converter_factory" incompatible with supertype "BaseUnitConverter"  [override]
+ homeassistant/util/unit_conversion.py:447: note:      Superclass:
+ homeassistant/util/unit_conversion.py:447: note:          _ClassmethodDescriptor[BaseUnitConverter, [str | None, str | None], Callable[[float], float]] | _ClsHashable[BaseUnitConverter, [str | None, str | None], Callable[[float], float]]
+ homeassistant/util/unit_conversion.py:447: note:      Subclass:
+ homeassistant/util/unit_conversion.py:447: note:          _ClassmethodDescriptor[TemperatureConverter, [str | None, str | None], Callable[[float], float]] | _ClsHashable[TemperatureConverter, [str | None, str | None], Callable[[float], float]]
+ homeassistant/util/unit_conversion.py:461: error: Signature of "converter_factory_allow_none" incompatible with supertype "BaseUnitConverter"  [override]
+ homeassistant/util/unit_conversion.py:461: note:      Superclass:
+ homeassistant/util/unit_conversion.py:461: note:          _ClassmethodDescriptor[BaseUnitConverter, [str | None, str | None], Callable[[float | None], float | None]] | _ClsHashable[BaseUnitConverter, [str | None, str | None], Callable[[float | None], float | None]]
+ homeassistant/util/unit_conversion.py:461: note:      Subclass:
+ homeassistant/util/unit_conversion.py:461: note:          _ClassmethodDescriptor[TemperatureConverter, [str | None, str | None], Callable[[float | None], float | None]] | _ClsHashable[TemperatureConverter, [str | None, str | None], Callable[[float | None], float | None]]
+ homeassistant/components/xiaomi_aqara/config_flow.py:216: error: Argument 1 to "__call__" of "_FunctionDescriptor" has incompatible type "str | None"; expected "str"  [arg-type]
+ homeassistant/components/profiler/__init__.py:8: error: Module "functools" has no attribute "_lru_cache_wrapper"  [attr-defined]
+ homeassistant/components/profiler/__init__.py:201: error: "object" has no attribute "__wrapped__"  [attr-defined]
+ homeassistant/components/profiler/__init__.py:202: error: "object" has no attribute "__wrapped__"  [attr-defined]
+ homeassistant/components/profiler/__init__.py:203: error: "object" has no attribute "cache_info"  [attr-defined]
+ homeassistant/components/mqtt_room/sensor.py:199: error: Argument 1 to "__call__" of "_FunctionDescriptor" has incompatible type "Any | None"; expected "str"  [arg-type]
+ homeassistant/components/esphome/update.py:63: error: Argument 2 to "__get__" of "_ClassmethodDescriptor" has incompatible type "type[DomainData]"; expected "type[Never]"  [arg-type]
+ homeassistant/components/esphome/update.py:63: error: Argument 2 to "__get__" of "_ClsHashable" has incompatible type "type[DomainData]"; expected "type[Never]"  [arg-type]
+ homeassistant/components/esphome/update.py:63: error: "Never" has no attribute "get_entry_data"  [attr-defined]
+ homeassistant/components/esphome/light.py:178: error: Argument 1 to "__call__" of "_FunctionDescriptor" has incompatible type "tuple[int, ...]"; expected "list[int]"  [arg-type]
+ homeassistant/components/esphome/light.py:187: error: Argument 1 to "__call__" of "_FunctionDescriptor" has incompatible type "tuple[int, ...]"; expected "list[int]"  [arg-type]
+ homeassistant/components/esphome/light.py:198: error: Argument 1 to "__call__" of "_FunctionDescriptor" has incompatible type "tuple[int, ...]"; expected "list[int]"  [arg-type]
+ homeassistant/components/esphome/light.py:207: error: Argument 1 to "__call__" of "_FunctionDescriptor" has incompatible type "tuple[int, ...]"; expected "list[int]"  [arg-type]
+ homeassistant/components/esphome/light.py:208: error: Argument 1 to "__call__" of "_FunctionDescriptor" has incompatible type "tuple[int, ...]"; expected "list[int]"  [arg-type]
+ homeassistant/components/esphome/light.py:213: error: Argument 1 to "__call__" of "_FunctionDescriptor" has incompatible type "tuple[int, ...]"; expected "list[int]"  [arg-type]
+ homeassistant/components/esphome/light.py:225: error: Argument 1 to "__call__" of "_FunctionDescriptor" has incompatible type "tuple[int, ...]"; expected "list[int]"  [arg-type]
+ homeassistant/components/esphome/light.py:242: error: Argument 1 to "__call__" of "_FunctionDescriptor" has incompatible type "tuple[int, ...]"; expected "list[int]"  [arg-type]
+ homeassistant/components/esphome/light.py:247: error: Argument 1 to "__call__" of "_FunctionDescriptor" has incompatible type "tuple[int, ...]"; expected "list[int]"  [arg-type]
+ homeassistant/components/esphome/light.py:261: error: Argument 1 to "__call__" of "_FunctionDescriptor" has incompatible type "tuple[int, ...]"; expected "list[int]"  [arg-type]
+ homeassistant/components/esphome/light.py:342: error: Argument 1 to "__call__" of "_FunctionDescriptor" has incompatible type "tuple[int, ...]"; expected "list[int]"  [arg-type]
+ homeassistant/components/esphome/__init__.py:61: error: Argument 2 to "__get__" of "_ClassmethodDescriptor" has incompatible type "type[DomainData]"; expected "type[Never]"  [arg-type]
+ homeassistant/components/esphome/__init__.py:61: error: Argument 2 to "__get__" of "_ClsHashable" has incompatible type "type[DomainData]"; expected "type[Never]"  [arg-type]
+ homeassistant/components/esphome/__init__.py:61: error: Need type annotation for "domain_data"  [var-annotated]
+ homeassistant/components/esphome/__init__.py:89: error: Argument 2 to "__get__" of "_ClassmethodDescriptor" has incompatible type "type[DomainData]"; expected "type[Never]"  [arg-type]
+ homeassistant/components/esphome/__init__.py:89: error: Argument 2 to "__get__" of "_ClsHashable" has incompatible type "type[DomainData]"; expected "type[Never]"  [arg-type]
+ homeassistant/components/esphome/__init__.py:89: error: "Never" has no attribute "get_or_create_store"  [attr-defined]

Daverball · 2024-11-03T10:59:34Z

stdlib/functools.pyi

 from typing_extensions import ParamSpec, Self, TypeAlias

 if sys.version_info >= (3, 9):
    from types import GenericAlias

+if sys.version_info >= (3, 10):
+    from typing import Concatenate
+


Suggested change

from typing_extensions import ParamSpec, Self, TypeAlias

if sys.version_info >= (3, 9):

from types import GenericAlias

if sys.version_info >= (3, 10):

from typing import Concatenate

from typing_extensions import Concatenate, ParamSpec, Self, TypeAlias

if sys.version_info >= (3, 9):

from types import GenericAlias

See CONTRIBUTING for which type system features are supported by typeshed and which ones need to be imported from typing_extensions rather than typing.

Thank you, I've been thinking about this (and actually reading documentation instead of blindly trying to get stuff to work). My thought at the moment is as you've suggested use the typing_extensions, also change back to _lru_cache_wrapper as a base class then have the descriptors as typing.type_check_only subclasses. I think the super/subclass issue is a correct error. And I need to add back into the stubtest_allowlist that the classes I've created are valid/to be ignored.

Simplified implementation now I've got a better understanding of the code

alwaysmpe · 2024-11-04T20:42:30Z

Creating a new PR from a new branch

alwaysmpe and others added 3 commits November 3, 2024 07:16

[pre-commit.ci] auto fixes from pre-commit.com hooks

c16b351

Fix remove overload implementation

833e40e

alwaysmpe mentioned this pull request Nov 3, 2024

Use ParamSpec for lru_cache #11662

Open

This comment has been minimized.

Sign in to view

alwaysmpe and others added 10 commits November 3, 2024 07:47

Fix add version gate for typing.Concatenate

6e22af9

Fix match mypy's expected type invariance

ab29e6d

[pre-commit.ci] auto fixes from pre-commit.com hooks

3cbb714

Fix copy/paste mistake

91f43a2

[pre-commit.ci] auto fixes from pre-commit.com hooks

54e4fe0

Fix cache should be positional-only

fd0d180

[pre-commit.ci] auto fixes from pre-commit.com hooks

3aaf0ff

Remove what I think are nolonger needed ignores

8c5c469

Fix final shouldn't have been removed

28e87fc

[pre-commit.ci] auto fixes from pre-commit.com hooks

1d1c46a

This comment has been minimized.

Sign in to view

Use type invariance recommended by pyright (which I think is correct)

d705819

Daverball reviewed Nov 3, 2024

View reviewed changes

alwaysmpe marked this pull request as draft November 3, 2024 11:09

Various improvements

6af9214

Simplified implementation now I've got a better understanding of the code

alwaysmpe closed this Nov 4, 2024

alwaysmpe mentioned this pull request Nov 4, 2024

Change cache/lru_cache typing to retain function signatures #12952

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Change `cache`/`lru_cache` typing to retain function signatures #12945

Change `cache`/`lru_cache` typing to retain function signatures #12945

alwaysmpe commented Nov 3, 2024 •

edited

Loading

alwaysmpe commented Nov 3, 2024

This comment has been minimized.

This comment has been minimized.

alwaysmpe commented Nov 3, 2024 •

edited

Loading

alwaysmpe commented Nov 3, 2024

alwaysmpe commented Nov 3, 2024

alwaysmpe commented Nov 3, 2024

alwaysmpe commented Nov 3, 2024

alwaysmpe commented Nov 3, 2024 •

edited

Loading

This comment has been minimized.

alwaysmpe commented Nov 3, 2024

This comment has been minimized.

github-actions bot commented Nov 3, 2024

Daverball Nov 3, 2024

alwaysmpe Nov 3, 2024

alwaysmpe commented Nov 4, 2024

Change cache/lru_cache typing to retain function signatures #12945

Change cache/lru_cache typing to retain function signatures #12945

Conversation

alwaysmpe commented Nov 3, 2024 • edited Loading

alwaysmpe commented Nov 3, 2024

This comment has been minimized.

This comment has been minimized.

alwaysmpe commented Nov 3, 2024 • edited Loading

alwaysmpe commented Nov 3, 2024

alwaysmpe commented Nov 3, 2024

alwaysmpe commented Nov 3, 2024

alwaysmpe commented Nov 3, 2024

alwaysmpe commented Nov 3, 2024 • edited Loading

This comment has been minimized.

alwaysmpe commented Nov 3, 2024

This comment has been minimized.

github-actions bot commented Nov 3, 2024

Daverball Nov 3, 2024

Choose a reason for hiding this comment

alwaysmpe Nov 3, 2024

Choose a reason for hiding this comment

alwaysmpe commented Nov 4, 2024

Change `cache`/`lru_cache` typing to retain function signatures #12945

Change `cache`/`lru_cache` typing to retain function signatures #12945

alwaysmpe commented Nov 3, 2024 •

edited

Loading

alwaysmpe commented Nov 3, 2024 •

edited

Loading

alwaysmpe commented Nov 3, 2024 •

edited

Loading