Allow use of entry-points like strings in mypy.ini to register plugins

[chadrik]

This addresses issue #3916. Unlike my last attempt this does not use pkg_resources.

It is currently possible for third-parties to author plugins and load them via mypy.ini, using a path to a plugin file. e.g.

[mypy]
plugins = path/plugin1.py, path/plugin2.py

This PR extends the syntax to allow plugins to be specified using entry-points-like strings:

[mypy]
plugins = path/plugin1.py, plugin2:register

Reservations have been expressed about the still-experimental nature of the plugin API. In rebuttal, I would say:

• It's already possible to load third-party plugins, it's just inconvenient
• Allowing developers to synchronize release of their plugins with their libraries will ensure compatibility and help reduce spurious bug reports.

Perhaps a compromise is to simply not advertise this feature extensively yet. It will enable projects like attrs, which are at the bleeding edge of static type checking and well aware of the unsupported risks involved, to push the tooling forward, while shouldering the responsibility of making adjustments to our plugin if the API changes.

[gvanrossum]

I think it's fine to add this. Yes, the plugin API will still change rapidly (basically each time someone writes a new plugin they have to spend some time changing the existing API), but since we already have plugin configuration in mypy.ini I don't see how this makes it worse.

(Haven't had time to review the code yet, sorry.)

[emmatyping]

I think it's fine to add this. Yes, the plugin API will still change rapidly (basically each time someone writes a new plugin they have to spend some time changing the existing API), but since we already have plugin configuration in mypy.ini I don't see how this makes it worse.

I agree. I think having a way to point mypy to a plugin has gotten significantly more important as several projects (such as attrs and numpy-stubs) will likely need this. With PEP 561 support, I only expect the number of projects wanting this to increase.

I will take a look at reviewing shortly.

[chadrik]

Great! I'll go over this a bit more thoroughly (I haven't actually tested it yet) and get all tests passing.

[ilevkivskyi]

I actually wanted to do something similar myself. Using a qualified name of plugin seems much simpler and sustainable than the full path. Didn't look at the code yet, but a quick question now: why such special syntax (I mean the colon)? I would expect just a qualified name, i.e. plugins = path/to/plugin1.py, package.plugin2 (IOW the same thing one would write in an import statement).

[emmatyping]

why such special syntax (I mean the colon)?

This is used in entry points for packaging, so I think the idea is to mimic that.

[ilevkivskyi]

This is used in entry points for packaging, so I think the idea is to mimic that.

What I would care is whether this is a common syntax for plugins in other Python projects. If not, I would stick to something simpler. Also there is a certain asymmetry with the current syntax, for the full path we always look for a function called plugin. I would expect the same for the new syntax, I would just expect a module instead of path.

[chadrik]

Btw, I support omitting the function name, as in the third example:

[mypy]
plugins = path/plugin1.py, plugin2:register, plugin3

In the last case, the registration function defaults to plugin, as it does for the first case.

[ilevkivskyi]

Btw, I support omitting the function name

OK, then I am fine with this. As I understand the module name can be qualified, like package.plugin.

Btw, maybe we can also add the possibility to override the function name with a colon syntax for the case of full path (for consistency)?

[chadrik]

Btw, maybe we can also add the possibility to override the function name with a colon syntax for the case of full path (for consistency)

I just realized that we cannot currently tell whether a plugin registered as foo.py is a file or a qualified module name (py.py is a valid python file name). Here are a few options:

1. make the entry-point required for modules: foo.py is a file and foo.py:register is a module entry-point.
2. require that paths include at least one slash: ./foo.py is a file and foo.py is a module.
3. remove support for file paths (I seriously doubt they see much real-world use)
4. choose a new key for module plugins, other than plugins
5. ignore the problem as a niche case

I will defer to you all on this.

[refi64]

Personal opinion: choice 2 would have the advantage of being explicit as to the file's location. It's immediately obvious that this is looking in the current directory, not a special lookup path of some sort or similar.

[chadrik]

choice 2 would have the advantage of being explicit as to the file's location.

It's not backward incompatible, though. If we're willing to make a breaking change, then why not option 3? In that case, if someone is currently using foo.py as a file path, they can change it to foo, and it should work when run from the same directory as the config file.

[gvanrossum]

Who would call their module "foo.py"? That would mean they have a filename "foo/py.py". That's just silly. So if it ends in ".py" (exactly) it should be treated as a file.

[chadrik]

So option 5 then ;)

[ilevkivskyi]

I am with Guido here.

[chadrik]

I added some tests.

For certain key cases related to the loading mechanism, I test both the file and module style loading, but what do you think about converting the remainder to use the module-style by default?

[ilevkivskyi]

@chadrik Thanks for adding tests! I like the PR in general, but it looks like I will not have time to review this in detail before my vacation. I hope someone else can review and merge this. I have only one comment, I would prefer to see "sandboxed" tests for installed plugins, e.g. like @ethanhs did this for PEP 561 packages testing instead of modifying sys.path.

[emmatyping]

it looks like I will not have time to review this in detail before my vacation. I hope someone else can review and merge this.

Enjoy your well earned vacation! I am happy to take this over, unless anyone else would prefer to take it (go ahead and unassign me if so).

[gvanrossum]

Thanks -- please go ahead, Ethan!

[chadrik]

I have only one comment, I would prefer to see "sandboxed" tests for installed plugins

The existing plugin tests use testcheck.py which runs in the current process. Pushing this into a subprocess will necessitate a specialized runner for plugins, which substantially increases the scope
of this PR. It feels like the right thing to do here is to make a separate issue for plugin test sandboxing.

[emmatyping]

I think ideally the plugin tests should have their own suite which is run out of process. The check suite is supposed to test the core type checking features of mypy, a plugin seems like a different unit to test. That being said, if you feel like it would be easier, I think there will plenty of time to refactor the testing in a separate PR.

[chadrik]

I completely agree with your logic, I’m just time poor and I don’t want this PR to get hung up on that issue. Let’s put it in a separate PR.

…

On Sat, Jul 28, 2018 at 4:17 PM Ethan Smith ***@***.***> wrote: I think ideally the plugin tests should have their own suite which is run out of process. The check suite is supposed to test the core type checking features of mypy, a plugin seems like a different unit to test. That being said, if you feel like it would be easier, I think there will plenty of time to refactor the testing in a separate PR. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#5358 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAD3E_o0HBcU1q8wI318Y2P1BlU5nGjSks5uLPD_gaJpZM4VQD5z> .

[JukkaL]

I usually prefer having non-trivial refactorings as separate PRs. PRs with both refactorings and new features are slower to review. They are also more likely to get merge conflicts, resulting in further delays. Sometimes it's best to do the refactoring as the first PR, but this is up to the author.

[chadrik]

It sounds like we are in agreement about creating a separate ticket for sandboxing the plugin tests. What’s left to do on this one?

…

On Mon, Jul 30, 2018 at 4:13 AM Jukka Lehtosalo ***@***.***> wrote: I usually prefer having non-trivial refactorings as separate PRs. PRs with both refactorings and new features are slower to review. They are also more likely to get merge conflicts, resulting in further delays. Sometimes it's best to do the refactoring as the first PR, but this is up to the author. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#5358 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAD3E4WBgNh-IRFMRQWf9yMM2AL2EeOUks5uLupbgaJpZM4VQD5z> .

[emmatyping]

What’s left to do on this one?

I will do a final review later today and hopefully merge.

[emmatyping]

Just two minor things. Thank you for working on this!

[emmatyping]

This comment seems out of date.

[chadrik]

true. will get this addressed once I get your answer on the second issue.

[emmatyping]

This won't work correctly on Windows if someone gives a path with e.g C:\\Foo\\bar. I'd recommend os.sep in plugin_path instead.

[chadrik]

Thanks for catching that. What I intended to do was this:

re.search(r'[\\/]', plugin_path) 

I considered os.sep, but decided against it because it's bound to the current OS. I think it's more correct to detect anything that looks like a file path, but I'll let you make the call.

[emmatyping]

That is a good point. I think your proposed solution is good.

[chadrik]

Notes addressed

[emmatyping]

@chadrik thank you!