@functools.wraps return type doesn't include __wrapped__ attribute

[huonw]

Since Python 3.2, the functools.wraps(f) decorator adds a __wrapped__ attribute, set to the original function f. The following (unconventional) use of wraps runs successfully and demonstrates this:

import functools

def foo():
    pass

@functools.wraps(foo)
def bar():
    pass

assert bar.__wrapped__ is foo 

However, this doesn't pass type checking, likely because the wraps function is annotated as returning a plain Callable which doesn't include anything about the __wrapped__ attribute:

typeshed/stdlib/3/functools.pyi

Line 58 in 3ae99d1

def wraps(wrapped: _AnyCallable, assigned: Sequence[str] = ..., updated: Sequence[str] = ...) -> Callable[[_T], _T]: ...

Errors from mypy https://mypy-play.net/?mypy=0.790&python=3.9&gist=8b06bab355f47dbabf1f8eb42c047a62 :

main.py:10: error: "Callable[..., Any]" has no attribute "__wrapped__"
Found 1 error in 1 file (checked 1 source file)

[srittau]

Unfortunately, I don't see a solution with the current type system. There is no way the "enhance" an existing generic type. I see two options here:

1. The status quo: return the original type. This has the disadvantage that __wraps__ does not exist in the returned type.
2. Return Any (or a subclass with __wrapped__ and __call__ attributes). Unfortunately this means we lose nearly all try checking on the call.

I prefer solution 1, since I believe this uncovers far more problems than not having __wrapped__ causes.

[huonw]

Hm, I might be misunderstanding your point about 2, but it seems like there's already very little checking on the call:

import functools

def foo():
    pass

@functools.wraps(foo)
def bar(x: int) -> None:
    pass

reveal_type(bar)

says the type is def (*Any, **Any) -> Any (whereas it's def (x: builtins.int) if the wraps line is commented out). Given this, it doesn't catch some things with incorrect types, as calls like bar() and bar("foo") are considered valid when using wraps.

As such, something like the following (along the lines of your 2), might be an improvement over the current state?

from typing import Callable, Any

_AnyCallable = Callable[..., Any]

class Ret:
    __wrapped__: _AnyCallable
    __call__: _AnyCallable


def wraps(wrapped_f: _AnyCallable) -> Callable[[_AnyCallable], Ret]:
    def inner(new_f):
        def wrapper(*args, **kwargs):
            return new_f(*args, **kwargs)
        wrapper.__wrapped__ = wrapped_f
        return wrapper
    return inner


def foo(x: str) -> bool:
    return x.startswith("f")


@wraps(foo)
def bar(x: int) -> str:
    return str(x + 1)
    
reveal_type(bar) # Ret
reveal_type(bar.__call__) # def (*Any, **Any) -> Any
reveal_type(bar.__wrapped__) # def (*Any, **Any) -> Any

(The biggest downside I can see here is the type of the wrapped function becomes Ret (or whatever the class is called) rather than direct callable, and so is slightly more opaque.)

Going more complicated, one could do something like the following, which preserve types but I think it requires variadic generics python/typing#193 to do properly; for instance, supporting 0 or 2 or 3 or ... arguments. (Collapsed to keep this comment more focused.)

from typing import TypeVar, Callable, Generic, Any

WrappedIn = TypeVar("WrappedIn")
WrappedOut = TypeVar("WrappedOut")
NewIn = TypeVar("NewIn")
NewOut = TypeVar("NewOut")

class Ret(Generic[WrappedIn, WrappedOut, NewIn, NewOut]):
    __wrapped__: Callable[[Any, WrappedIn], WrappedOut]
    __call__: Callable[[Any, NewIn], NewOut]


def wraps(wrapped_f: Callable[[WrappedIn], WrappedOut]) -> Callable[
    [Callable[[NewIn], NewOut]], 
    Ret[WrappedIn, WrappedOut, NewIn, NewOut]
]:
    def inner(new_f):
        def wrapper(*args, **kwargs):
            return new_f(*args, **kwargs)
        wrapper.__wrapped__ = wrapped_f
        return wrapper
    return inner


def foo(x: str) -> bool:
    return x.startswith("f")

    
@wraps(foo)
def baz(x: int) -> str:
    return str(x + 1)

baz(1) # ok
baz.__wrapped__("foo") # ok

baz("foo") # error: Argument 1 has incompatible type "str"; expected "int"
baz.__wrapped__(1) # error: Argument 1 has incompatible type "int"; expected "str"

---

All that said, wraps is typically used inside another function, where it'll have type annotation with the convenient Callable[...]:

def my_decorator(f: Callable[[int, float], str]) -> Callable[[int], None]:
    @wraps(f)
    def wrapper(x: int) -> None:
        print(f(x, x + 0.5))

    return wrapper

@my_decorator
def foo(x: int, y: float) -> str:
    return f"{x} {y}"

As such, this may be little value to refine.

[hauntsaninja]

You're not using typeshed master, which has #4743

[mrahtz]

@huonw I noticed you referenced python/typing#193 on variadic generics in this thread. Heads up that we've been working on a draft of a PEP for this in PEP 646. If this is something you still care about, take a read and let us know any feedback in this thread in typing-sig. Thanks!

[srittau]

I am going to close this for now, as is standard for issues labeled "inexpressible". We can revisit this when PEP 646 is implemented in supported type checkers.