Unable to match Protocol implementation using function-level generic to Protocol interface using class-level generic

[EthanBullVulpe]

Environment data

• Language Server version: 2023.4.41
• OS and version: linux x64
• Python version (and distribution if applicable, e.g. Anaconda): 3.10 (Anaconda)
• python.analysis.indexing: true
• python.analysis.typeCheckingMode: strict

Code Snippet

from __future__ import annotations
from abc import abstractmethod
from typing import Protocol, Sequence, TypeVar

Input = TypeVar("Input", contravariant=True)
Output = TypeVar("Output", covariant=True)

T = TypeVar("T")

class Map(Protocol[Input, Output]): # the Protocol interface
    
    @abstractmethod
    def __call__(self, input: Input) -> Output:
        raise NotImplementedError
    
class GetFirst: # the Protocol implementation
    
    def __call__(self, input: Sequence[T]) -> T:
        return input[0]
    
def select(selector: Map[Sequence[int], int]) -> bool:
    raise NotImplementedError

select(GetFirst()) # This line produces a type error
    
m: Map[Sequence[int], int] = GetFirst() # This line produces a type error

Expected behavior

I think there should be no type error on lines select(GetFirst()) and m: Map[Sequence[int], int] = GetFirst(). Since GetFirst implements the Map[Sequence[T], T] Protocol based on the definition of GetFirst.__call__, and thus should match the type Map[Sequence[int], int].

Actual behavior

Pylance strict type checking gives an error on the following two lines:
select(GetFirst())
and
m: Map[Sequence[int], int] = GetFirst()
In both cases, the error is essentially the same:

Expression of type "GetFirst" cannot be assigned to declared type "Map[Sequence[int], int]"
  "GetFirst" is incompatible with protocol "Map[Sequence[int], int]"
    TypeVar "Input@Map" is contravariant
      "Sequence[int]" is incompatible with "Sequence[T@__call__]"
        TypeVar "_T_co@Sequence" is covariant
          Type "int" cannot be assigned to type "T@__call__"
    TypeVar "Output@Map" is covariant
      "object*" is incompatible with "int"

I have a feeling that the source of the issue is that in Map, Input and Output are class-level generics, while in GetFirst, T is a function-level generic, but I do not see why that should lead to a type error.

Note that mypy considers this code well-typed.

Conda Enviornment

[environment.yml.txt](https://github.com/microsoft/pylance-release/files/11348276/environment.yml.txt)

Thanks in advance.

[erictraut]

@rchiodo , would you like to transfer this to pyright since it's a core type checking issue?

[rchiodo]

Will do :)

[last-partizan]

Looks like i stumbled upon this bug as well, or at least similar one.

Here's real life example, djangoresframework uses djangoresftramework-stubs, and it has defined protocol for filter backend classes:

_Q = TypeVar("_Q", bound=Union[QuerySet[Any], MongoQuerySet[Any]])

class _FilterBackendProtocol(Protocol):
    def filter_queryset(self, request: Any, queryset: _Q, view: APIView) -> _Q: ...

But when using with untyped package django-filters, it produces error: "DjangoFilterBackend" is incompatible with protocol "_FilterBackendProtocol"

# This is from reveal_type, for the reference
Type of "DjangoFilterBackend.filter_queryset" is "(self: DjangoFilterBackend, request: Unknown, queryset: Unknown, view: Unknown) -> (Unknown | Any | QuerySet[Unknown])"

Full example here:

https://github.com/last-partizan/pyright-examples/tree/44e045de106e3d5f2fe7f079280441eca1654be2

@erictraut is that looks like the same bug, or i'd better create new issue?

[erictraut]

@last-partizan, yes, I can confirm this is the same issue.

[erictraut]

@last-partizan, upon further investigation, this isn't the same issue. The problem is that the DjangoFilterBackend method filter_queryset is unannotated, so its return type is being inferred. This return type isn't compatible with the protocol _FilterBackendProtocol. So I think pyright is correct to generate an error in this case. To fix this, you'd need to provide correct type information for the django_filters library in the form of type stubs.

[erictraut]

The bug reported at the top of this issue will be addressed in the next release.

[last-partizan]

I'm trying to figure out proper type annotation, but something is not right.

Pyright version tested:

• 1.1.308
• 1.1.314
• built from source (main)

Here's my example, stripped out of all external references.

from typing import Any, Generic, Protocol, TypeVar

T = TypeVar("T")


class QuerySet(Generic[T]):
    ...


Q = TypeVar("Q", bound=QuerySet[Any])


class FilterBackendProtocol(Protocol):
    def filter_queryset(self, qs: Q) -> Q:
        ...


class DjangoFilterBackend:
    def filter_queryset(self, qs: QuerySet[Any]) -> QuerySet[Any]:
        return qs


class APIView:
    filter_backends: list[type[FilterBackendProtocol]]


class FinalAPIView(APIView):
    filter_backends = [DjangoFilterBackend]

/home/serg/src/pyright-examples/example.py
  /home/serg/src/pyright-examples/example.py:28:24 - error: Expression of type "list[type[DjangoFilterBackend]]" cannot be assigned to declared type "list[type[FilterBackendProtocol]]"
    "DjangoFilterBackend" is incompatible with protocol "FilterBackendProtocol"
    Type "type[DjangoFilterBackend]" cannot be assigned to type "type[FilterBackendProtocol]"
      "filter_queryset" is an incompatible type
        Type "(qs: QuerySet[Any]) -> QuerySet[Any]" cannot be assigned to type "(qs: Q@filter_queryset) -> Q@filter_queryset"
          Function return type "QuerySet[Any]" is incompatible with type "Q@filter_queryset"
            Type "QuerySet[Any]" cannot be assigned to type "Q@filter_queryset" (reportGeneralTypeIssues)
1 error, 0 warnings, 0 informations

[erictraut]

@last-partizan, the problem with your most recent code sample is that your implementation of DjangoFilterBackend.filter_queryset doesn't meet the requirements of the protocol. The filter_queryset method in the protocol requires that the type of the qs parameter and the return type are identical because they share the same type variable. In your implementation, the qs parameter can be any subtype of QuerySet[Any], and the return type can be a different subtype of QuerySet[Any]. To make your implementation compatible with the protocol, you'd need to use a type variable in its signature.

class DjangoFilterBackend:
    def filter_queryset(self, qs: Q) -> Q:
        return qs

[erictraut]

This is included in pyright 1.1.315, which I just published. It will also be included in this week's insiders release of pylance.