Plugins?

[MiroFurtado]

Does there currently exist any way of extending pyright akin to the plugin system that is offered by mypy?

[erictraut]

No, not currently. I'm reluctant to add plugin support at this time for several reasons: 1) Pyright is still changing at a rapid pace, and a plugin system would create a constraint that would prevent certain changes, 2) type support within the Python language has improved significantly since mypy was originally developed, so most (although still not all) cases can be handled without custom extensions, 3) philosophically, I'd prefer to continue to push for improving the Python standard and the adoption of this standard within third-party libraries rather than encouraging library-specific non-standard behaviors.

I'm open to revisiting this in the future (say, in a year).

Out of curiosity, what use case are you interested in?

[sobolevn]

I am interested in this 🙂

We have lots of mypy plugins that are currently impossible to model with regular typing. It includes:

• function curring and typed partial
• higher kinded types
• pipe(a, a -> b, b -> c, c -> d, ...)
• better decorators support with changing types inside them

I would love to add extra typecheckers that would do the same thing as mypy, so I am looking for options 👍

[sobolevn]

@erictraut

I'm open to revisiting this in the future (say, in a year).

One year has passed, are there any news on this feature? 🙂

Some context, why I am asking about it: I have several quite complex mypy plugins and I am wondering if other type checkers could support my libraries as well. Example: https://github.com/dry-python/returns/tree/master/returns/contrib/mypy/_features

[erictraut]

We've spent some time on this. Here's what I can say at this point:

We will not be providing a proprietary plug-in model akin to the one provided by mypy. There are too many problems with such a model. It doesn't work across type checkers, maintenance of plugins is a problem, distribution and installation of plugins is a problem (since users need to know they exist and how to configure them), and there are potential security issues with plugins that are downloaded and run on users' machines.

Instead, we've started to look at ways to cover the most common reasons why mypy plugins are written today. This is what led to this proposal, which is implemented in pyright today and supports several popular libraries (pydantic, attrs, etc.) that previously required mypy plugins. Our preference is to find declarative forms to describe type semantics rather than imperative code used, for example, in mypy plugins.

If there are patterns similar to dataclass_transform that you think would have general utility across multiple libraries and could form the basis for an official language extension (in the form of a ratified PEP), we'd be interested in your thoughts.

In general, we want to encourage authors of new Python libraries (or new versions of existing Python libraries) to take into account static type checking when designing their interface contracts. Non-standard semantics and dynamic behaviors are antithetical to static type checking and work against efforts to improve the Python coding experience with tools such as language servers and type checkers.

[AaronBeaudoin]

I think there are cases where a plugin system would still be beneficial. For me the big problem is Odoo. Someone has developed a closed-source (unfortunately) VS Code extension here that is implemented on top of Pyright to support all of Odoo's mechanisms, and the side effect is that the author also recommends disabling the regular Pyright extension in Odoo projects. It would have been so much better if Pyright was simply extensible instead.

I am not a developer of Odoo itself (I only develop addons) so I can't speak into all of the technical reasons why the developers chose to design Odoo the way it is. But one thing is clear: Odoo philosophy revolves around making the system extensible, and this design goal seems to be what has led to many of the static type checker incompatibilities. If you want to understand why it would be impossible to accomplish the extensibility Odoo provides while being totally "pythonic" and supporting static type checkers, you'll have to do a bit of reading to understand what Odoo's addon system does that is not possible with Python's built-in inheritance mechanisms. Here are a few suggestions:

• https://www.odoo.com/documentation/13.0/developer/reference/addons/orm.html#inheritance-and-extension
• https://stackoverflow.com/questions/38362501/odoo-extension-and-inheritance

[felixvd]

This is what led to this proposal, which is implemented in pyright today

Since the link is dead: This is now https://peps.python.org/pep-0681/.

[Suor]

The thing is plugins are not always about extra typing and when they are it may be because some local code changes.

An example of the first is filtering pyright messages to changed lines or functions only as opposed to the whole changed files. The whole files makes migration impossible for us now because of walls of warnings for each PR.

Second situation happens when say you add a custom middleware to Django, which adds some attributes to request object, pyright doesn't know about that and compains about "non-existing" attributes.

[erictraut]

@Suor, I understand there are benefits to plugins. They also come with very significant downsides. We made the decision long ago not to support plugins in pyright, and we're going to stand by that decision. Our strategy is to continue to participate in extending the Python type system so more patterns are supported.

[dvargas92495]

Is there a way for me to introduce my own annotation, that can perform dynamic type analysis, similar to dataclass/dataclass_transform? Here's a simple example:

from dataclasses import dataclass
from typing import Type, TypeVar

_T = TypeVar("_T")


def vargas(cls: Type[_T]) -> Type[_T]:
    def init(self: _T, foo: str) -> None:
        self.foo = foo

    cls.__init__ = init
    return cls


@dataclass
class A:
    foo: str


@vargas
class B:
    pass


A(foo="")

B(foo="")

I was able to use a mypy plugin to resolve this issue for our CI system. However, most engineers in our org use VSCode, therefore pylance, therefore pyright. Which means we don't get type help during our editing experience:

To be clear, @dataclass_transform will not solve our use case, since our end goal is not a dataclass-like experience, but rather an experience where we could provide static type help for any metaprogramming our annotations will provide, the above being a simple example of one. Does pyright provide a mechanism where we could define these rules somehow? Otherwise, the development of our framework will be limited and we'll have to explore the feasibility of using mypy instead in our editors for intellisense