Support for multiple versions of same library

[caulagi]

Is it possible to add multiple versions of the same library to typeshed? Say for instance, django 1.8 and 1.9 for Python 3?

[gvanrossum]

This currently isn't supported. We've considered it but haven't come up with a reasonable naming scheme. It would be somewhat problematic because each library has potentially different semantics for its version strings. Hopefully there aren't too many differences between Django 1.8 and 1.9?

[caulagi]

Hmm...that would probably make it tricky to add stubs for Django. Each release has a few incompatible changes and my sense is the stub files would also change in some way. I can check how the stub files are different for Django 1.8 and 1.9.

[caulagi]

This is how the version numbers work for Django (https://docs.djangoproject.com/en/dev/internals/release-process/)

Since version 1.0, Django’s release numbering works as follows:

• Versions are numbered in the form A.B or A.B.C.
• A.B is the feature release version number. Each version will be mostly backwards compatible with the previous release. Exceptions to this rule will be listed in the release notes.
• C is the patch release version number, which is incremented for bugfix and security releases. These releases will be 100% backwards-compatible with the previous patch release. The only exception is when a security or data loss issue can’t be fixed without breaking backwards-compatibility. If this happens, the release notes will provide detailed upgrade instructions.

[JukkaL]

We discussed this last year when working on PEP 484 but didn't reach a conclusion. I don't remember all the ideas that came up, but here are some:

• Use foo.pyi (or foo/) for the stubs for the latest release. Use foo-x.y.pyi (or foo-x.y/) for earlier releases. If stubs foo.pyi and foo-1.2.pyi exist, we'd use foo-1.2.pyi for 1.1, 1.2, 1.2.9, etc. We'd use foo.pyi for 1.3, 1.3.0 and newer.

• Support conditional module version checks within stubs. Perhaps something like this:

  if <greater_than_equal_comparison_fn>(__version__, '1.2'): 
      def dostuff(x: Union[int, str]) -> None: ... # definitions for 1.2+
  else:
      def dostuff(x: int) -> None ... # definitions for older releases
  

  The comparison function and __version__ don't have to exist at runtime since stubs won't generally be runnable anyway. This would be similar but not identical to Python version checks in stubs.

Type checking tools like mypy would also need a way of specifying the versions of dependencies so that they can use the right stubs. Perhaps tools would process pip requirements files and look up package versions from them. If some package has no given version, a tool would revert to the latest version.

[caulagi]

@JukkaL thanks! Is my understanding correct that you don't need any changes in typeshed itself with either approach?

Also, can you please explain

The comparison function and version don't have to exist at runtime since stubs won't generally be runnable anyway. This would be similar but not identical to Python version checks in stubs.

How will mypy know which definition to use?

[JukkaL]

@caulagi We'd need to update typeshed documentation to describe the approach we support, and we may need to tweak the typeshed CI script to test multiple versions of certain packages. Mypy could take a pip requirements file with version numbers as an option to determine which definitions use. For example, we could give mypy this requirements file:

mock==1.0.1
nose==1.3.0

Mypy would now assume that __version__ in stubs for mock would be 1.0.1.

[gvanrossum]

Perhaps also related: python/typing#84

[Naddiseo]

Would something similar to the naming system used in flow-typed work for python/typeshed?

Something like:

└third_party/
  ├ 2/                          # <-- Python version on top level
  | |
  | ├ django_v1.6.x/            # <-- folder containing python2 
  | | |                         #     specific stubs for django 1.6.x
  | | |
  | | └ __init__.pyi            # stubs
  | |
  | ├ django_v1.7.0-v1.7.9/     # <-- folder for python2 specific stubs
  | | |                         #     for django v1.7.0 to v1.7.9 (inclusive)
  | | |
  | | └ ...
  | |
  | └ django_v1.8.x-/           # <-- folder for python2 specific stubs
  |   |                         #     for django versions higher than
  |   |                         #     version 1.8
  |   └ ...
  ├ 2and3/                      # as it currently is, stubs for py 2 and 3
  | |
  | ├ django_v1.6.x/            # <-- folder containing python 2/3 
  | | |                         #     stubs for django 1.6.x
  | | |
  | | └ __init__.pyi            # stubs
  | |
  | ├ django_v1.7.0-v1.7.9/     # <-- folder for python 2/3 stubs
  | | |                         #     for django v1.7.0 to v1.7.9 (inclusive)
  | | |
  | | └ ...
  | |
  | └ django_v1.8.x-/           # <-- folder for python 2/3 stubs
  |   |                         #     for django versions higher than
  |   |                         #     version 1.8
  |   └ ...
  ...

and perhaps changing the python version naming to support ranges (eg v2-3.5, v3.6-) might keep it future compatible.

[gvanrossum]

How would mypy know which version of Django you want to check with? It can't look at sys.path since that may be a different Python version. It would be awkward to have to pass that on every run like the python version

[Naddiseo]

@gvanrossum, admittedly I didn't address that half of the problem because I thought it was covered in JukkaL's earlier comments; I was only trying to offer an example of prior art which is in use in JS land. To answer your question: as a user of mypy, I would expect it to look at my requirements.txt file (or setup.py) for specified versions, and use the latest available version if none were otherwise specified.

[gvanrossum]

Sorry for not re-reading pr remembering the year-old comments. :-)

This then becomes blocked on someone implementing the desired search algorithm in mypy (though we should also consider the possibility of supporting this in PyCharm and pytype).

[JelleZijlstra]

PEP 561 (https://www.python.org/dev/peps/pep-0561/) now provides a workaround for this issue: you can release and install a separate stub package for each version of the library. Supporting multiple versions directly in typeshed is still potentially useful, but most third-party libraries in typeshed are currently stable enough that this is not a big issue.

[gvanrossum]

I actually think we probably shouldn't try to support this in typeshed.

[JelleZijlstra]

I agree, so let's close this.