improve lru_cache by using ParamSpec

[asottile]

python/mypy#5107

Fixes #6987.

[asottile]

hmmm, this is close but not quite there. seems my little descriptor trick fixes a lot of situations but doesn't quite work for classmethods. also seems to trigger an internal error in mypy (though pyright is ok with it) -- mypy's matching of Callable with ParamSpec seems a little incomplete

I had originally written this idea as a WIP mypy plugin

[AlexWaygood]

Linking to previous typeshed discussion on this topic:

• functools._lru_cache_wrapper should be a descriptor class, not a callable #6347

[AlexWaygood]

I've reported the mypy crash at python/mypy#12922.

[AlexWaygood]

I've reported the mypy crash at python/mypy#12922.

The crash should be fixed in 0.970 now that python/mypy#12953 has been merged :)

[AlexWaygood]

Marking this as deferred until mypy 0.970 has been released.

[AlexWaygood]

The patch no longer causes mypy to crash on 0.971, but there's now a stubtest failure. Also, the mypy_primer report is a bit of a mixed bag.

[github-actions]

Diff from mypy_primer, showing the effect of this PR on open source code:

jax (https://github.com/google/jax)
+ jax/experimental/sharding.py:94: error: Signature of "shard_shape" incompatible with supertype "Sharding"  [override]
+ jax/experimental/sharding.py:304: error: Signature of "devices_indices_map" incompatible with supertype "Sharding"  [override]
+ jax/experimental/sharding.py:317: error: Signature of "shard_shape" incompatible with supertype "XLACompatibleSharding"  [override]
+ jax/experimental/sharding.py:317: error: Signature of "shard_shape" incompatible with supertype "Sharding"  [override]
+ jax/experimental/sharding.py:378: error: Signature of "devices_indices_map" incompatible with supertype "Sharding"  [override]

manticore (https://github.com/trailofbits/manticore)
+ manticore/core/smtlib/solver.py:505: error: Signature of "can_be_true" incompatible with supertype "Solver"

pandas (https://github.com/pandas-dev/pandas)
+ pandas/core/dtypes/cast.py:560: error: Unused "type: ignore" comment

pydantic (https://github.com/samuelcolvin/pydantic)
+ pydantic/tools.py:37: error: Unused "type: ignore" comment

jinja (https://github.com/pallets/jinja)
+ src/jinja2/environment.py:1184: error: Unused "type: ignore" comment
+ src/jinja2/environment.py:1199: error: Unused "type: ignore" comment

ibis (https://github.com/ibis-project/ibis)
+ ibis/backends/duckdb/tests/conftest.py:19: error: Too many arguments for "__call__" of "_lru_cache_wrapper_0"
+ ibis/backends/duckdb/tests/conftest.py:53: error: Signature of "connect" incompatible with supertype "BackendTest"

paroxython (https://github.com/laowantong/paroxython)
+ paroxython/assess_costs.py:85: error: Incompatible types in assignment (expression has type "_lru_cache_wrapper[int, [int], float]", variable has type "_lru_cache_wrapper_0[[int], float]")

pylint (https://github.com/pycqa/pylint)
+ pylint/checkers/classes/class_checker.py:2002: error: Returning Any from function declared to return "Union[SupportsDunderLT[Any], SupportsDunderGT[Any]]"  [no-any-return]

rich (https://github.com/Textualize/rich)
+ rich/progress_bar.py:145: error: Argument 3 to "__call__" of "_lru_cache_wrapper_0" has incompatible type "Optional[str]"; expected "str"  [arg-type]

zulip (https://github.com/zulip/zulip)
+ zerver/lib/cache.py:744: error: "_lru_cache_wrapper" expects 3 type arguments, but 1 given  [type-arg]
+ zerver/lib/cache.py:761: error: Unused "type: ignore" comment

aiohttp (https://github.com/aio-libs/aiohttp)
+ aiohttp/connector.py:935: error: Too many arguments for "__call__" of "_lru_cache_wrapper_0"  [call-arg]
+ aiohttp/connector.py:941: error: Too many arguments for "__call__" of "_lru_cache_wrapper_0"  [call-arg]
+ aiohttp/connector.py:942: error: Too many arguments for "__call__" of "_lru_cache_wrapper_0"  [call-arg]

[asottile]

this is going to go the same way as #7430 so I'm just going to retract it

[Greedquest]

Hello, has this been implemented somewhere. lru_cache and cache would really benefit from ParamSpec they seem like the textbook usecase. Right now mypy is seeing the function args as Any which means my callsites are not enforced.

[kkirsche]

I don't believe so. I know when I tried it originally, I had run into some problems getting everything to work with the various dunder (double underscore) properties. Some of my personal obligations are clearing up so I may be able to take another stab at this. I may have still misunderstood ParamSpec at that time (I wanted it to be more like typescript function parameter utility type). At the very least, giving it another shot should allow me to articulate more clearly what specific issues I ran into in case others may be able to help solve those.

[JelleZijlstra]

I believe the difficulty is in dealing with __get__ to make it work correctly as both a function and a method decorator.