Define conventions for stubs

[JukkaL]

The PEP should define some conventions for library stubs. Here are some things that might be relevant:

• How to declare variable types? Maybe use Undefined(...) exclusively, so that a parser that processes the stubs doesn't need to be able to access comments?
• Do we need to use __all__, or should this be implicit? What about visibility of names imported from other modules?
• Should we have a convention for recognizing a Python file as a stub file? Maybe they could optionally have a different extension, or maybe any files in directory named stubs are considered stubs automatically?

[ambv]

We should define what stubs are and why we need them.

[JukkaL]

Related idea (to avoid needing to use __all__ everywhere): python/mypy#453

[JukkaL]

Related issue with some discussion about an alternative extension (.pyi) for stubs, which is now supported by mypy: python/mypy#345

[ambv]

1. The as idea is confusing, as I mentioned in Do not export imported names in stubs unless using 'as' mypy#453
2. @gvanrossum, are stubs a part of the PEP? This is a fundamental question because if we want to support them and @JukkaL already provides a different file extension for them in mypy, maybe we should rethink if they should use Python syntax at all.

pytypedecl is a rather mature approach that solves a number of issues by abandoning Python syntax, namely:

• function overloading
• generics with arbitrary types (no need for typing.py redefining List et al.)
• nicer syntax for union types

As I understand it, stubs are useful for typing existing third-party libraries, builtins and C extensions. Maybe we should prefer to only have one obvious way to do it?

(I won't argue for that longer, just wanted to get a definitive answer here.)

If we want stubs to be Python, I believe that it should be Python with no tricks.

[ambv]

Overloading would be solved by having stubs sport a custom syntax.

#14

[JukkaL]

Mypy provides an optional, alternative file extension for stubs, but all stubs bundled with mypy still use the .py extension, and stubs use the same syntax as normal Python files. However, mypy stubs are not intended to be imported, since they would conflict with the target module as they have the same name.

I don't like the idea of having a completely different syntax for stubs. There are just a few differences that would be useful to have for stubs, including support for overloading and potentially an implicit __all__ attribute. The latter isn't really needed, but it would be convenient. Having two annotation syntaxes to learn seems like unnecessary complexity to me.

Without stubs for standard library modules and third-party modules type checking (and static analysis in general) will not be terribly effective, at least until most third-party modules get migrated to static typing (which will likely take a while, since most of them have to support Python 2). I think that having a standard format for library stubs that is compatible with this PEP would be very useful, if only to discourage every tool maker from designing their own stub format. It doesn't need to happen within this PEP, though.

[gvanrossum]

Yes.

• Let's have stubs
• Let's allow either .py or .pyi as their extension
• Let's give them the same syntax as regular Python modules
• With the exception of @overload (which is only allowed in stubs)

Regarding the implicit __all__, I don't like the as hack much. My counter-proposal is that if you want to export something that you've imported you must explicitly assign it. Example:

import typing
List = typing.List

Or if you really want to use from ... import, you can write

from typing import List
List = List

The assignment flags the desire to export the name.

Since stubs are never imported by the "real" Python interpreter (i.e. never used or seen at runtime) we don't need an implementation of this. But for various other reasons, if you do manage to import a stub module, it should in general parse and load without errors. (But using from <stub> import * may export more than you expected.)

[JukkaL]

I like the assignment proposal, as it's more explicit than the as idea.

[vlasovskikh]

Here is a link to the typeshed repo.

[JukkaL]

Some additional context: Our intention is to use typeshed to centrally collect stubs for the standard library and 3rd party modules, and we can agree on stub conventions using the typeshed issue tracker. The typeshed repo will likely be moved away from my personal github account (since no individual should own it) -- check the typeshed readme to see if this move has happened already.

[gvanrossum]

I am assuming we only need to add the import ... as ... convention to the PEP in order to close this issue. Or is there anything else?

[gvanrossum]

Tentatively closing as the "as" issue is now in the PEP.