Check callable argument/return types

[bergwerf]

How hard do you think it would be to fully type check a callable? I am not very familiar with all the underlying technology.

[agronholm]

It's doable but requires nontrivial amount of work. There is a particular issue around type variables where get_type_hints() does not resolve forward declarations and typeguard would have to do it manually.

[zbentley]

This would be extremely valuable for several of our use cases. Does typing_inspect support the forward resolution needed to get fully reified annotations from an arbitrary callable?

[agronholm]

Getting the annotations is not a problem. Making a compatibility comparison between them is.

[zbentley]

That makes sense. Would it be reasonable to allow users to opt-in to exact type checking? I.e. check_argument_types(strict_callables=True) would require supplied callables to be explicitly annotated with compatible types?

"Compatible" could initially be defined as "exactly matches", with enhancements possible in future releases.

So, given the signature def foo(arg: Callable[[numbers.Number, MutableMapping[str, int]], SomeUserClass]), the following behavior would be present:

• check_argument_types(strict_callables=False) would pass if callable(arg) == True, and fail otherwise.
• check_argument_types(strict_callables=True) would pass if arg was a reference to def bar(arg1: numbers.Number, arg2: MutableMapping[str, int]) -> SomeUserClass, explicitly annotated.
• Initially, check_argument_types(strict_callables=True) would fail if arg was any of these cases:

def bar(arg1: int, arg2: MutableMapping[str, int]): implicit Any return doesn't satisfy SomeUserClass: int isn't equal to numbers.Number.
def bar(arg1: numbers.Number, arg2: Dict[str, int]) -> SomeUserClass: Dict isn't equal to MutableMapping.
def bar(arg1: numbers.Number, arg2: MutableMapping[str, int]) -> SubclassOfSomeUserClass: SubclassOfSomeUserClass isn't equal to SomeUserClass.

Now, all of those cases should succeed eventually, but building a type validator is, as you pointed out, tricky.

I think a good path forward would be to start with "overly restrictive but safe and consistent" by requiring people to pass Callables whose annotations explicitly exactly match the receiver's annotation.

Then, over time, contributors can gradually improve the meta-type checker to accept more valid input annotations. For example, an easy rule to add would be to check compatibility of annotated types that have zero-argument constructors via check_type--e.g. typeguard for def bar(arg: Callable[[MutableMapping], Any]) could validate an arg reference to def bar(arg1: dict) by running check_type(t(), annotation) for each t in arg's parameter annotation types if t had a no-arguments constructor method, resulting in check_type('arg', dict(), MutableMapping) in this example.

With this approach you don't have to make a full general type-compatibility checker all at once, up front. However, it may result in more bug reports due to the artificial restrictiveness of the initial approach.

What do you think?

[zbentley]

Some prior art (which works for a surprising number of cases given how little code it is) is here: https://github.com/hyroai/gamla/blob/master/gamla/type_safety.py

[zbentley]

Some more prior art, with a bit better API, but also pretty young/not too widely used: https://github.com/bojiang/typing_utils

[agronholm]

I honestly don't have any bandwidth to work on this at all, but if somebody puts together a PR with accompanying tests, I would be willing to review it.

[zbentley]

No worries, and thanks for all the work you've done on typeguard so far, it's invaluable!

I can work on that PR, but due to the initially overly-restrictive checking to be performed it's likely that it'll yield some issue reports of folks who want the type reconciliation to be better (the idea being we could refine it over time).

Given that, would you prefer I PR here or do that work in a different library? I don't wanna put it up if it's unlikely to be accepted, and also don't want to cause you more interrupt hassle.

[agronholm]

I don't mind users getting new errors along with the new code, so long as the errors reveal actual type incompatibilities. And as I said, I would accept a PR against this project.

[zbentley]

Thanks. I will work on something but it won't be particularly soon, sorry (working on this on the side of Real Job ™️ ).