What to do about types from the numbers module?

[gvanrossum]

PEP 3141 introduces a numeric tower and a corresponding module numbers defining some numeric ABCs like Integral and Real. These ABCs are ignored by PEP 484 (you just have to use int or float) but some people don't know that and in their zeal to be hyper-correct use e.g. numbers.Integral in type annotations. Then type checks fail because mypy doesn't consider int a subclass of numbers.Integral.

We should either make this work or warn about numeric ABCs when used in type annotations.

[JukkaL]

I looked into this a long time ago and my conclusion was that supporting numbers.Integral etc. in a useful way would be pretty complicated. I may be able to find some old notes I wrote about this.

Also, as Integral wouldn't be compatible with int, their usefulness would be pretty limited unless they are used in a lot of places instead of int, as otherwise there would be need for a lot of int(x) conversions. But if Integral is used a lot, then interacting with modules that only use int annotations would be a pain. Basically there seems to be a small risk that supporting Integral would eventually result in two "mypy dialects" that can't seamlessly interoperate with each other: one dialect would prefer int, while the other would use Integral everywhere where possible. Also, some stubs might use one dialect and others use the other.

[gvanrossum]

That sounds about right, and similar to the deliberations I recall we had about this during the formative days of PEP 484. So maybe we can just warn about them?

[ilevkivskyi]

Just posting here an older proposal by @JukkaL so that it is not lost python/typing#272

[JukkaL]

Problems with numbers are pretty frequent. I think that mypy should give a warning if it sees numbers.Integral in an annotation.

[gvanrossum]

Or it could just treat it as an alias for int?

[JukkaL]

Making Integral an alias of int would interact really awkwardly with subclassing. Example:

class MyIntegral(numbers.Integral):   
    # Mypy would think that this would inherit all `int` methods, even
    # though that's not the case at runtime
    ...

n = MyIntegral(5)
chr(n)   # Mypy thinks that this okay even though it would fail at runtime

Another concern is that if we support this, some users will likely start using numbers.Integral everywhere, and we'd be stuck with the alias hack. If we'd implement Integral properly in the future, we'd break a lot of code.

I don't really see the benefits of having the alias, since it would usually not do the right thing. Also, I haven't seen any real-world code that would clearly benefit from Integral. But in case we'll find evidence of Integral being useful, I'd rather have it not supported at all right now so that we have the freedom to implement it in a useful way (whatever that might be) once the use case becomes clear, without backward compatibility concerns.

[ilevkivskyi]

I think most of Integral and other numbers functionality can be implemented correctly just by making them protocols in typeshed (except for Number, since this is just an empty "root" ABC).

[gvanrossum]

OK, let's not do anything for numbers right now, and let's consider making them protocols in the future.

[JukkaL]

I foresee ugliness even if we'd use protocols, because numbers relies heavily on overloading, NotImplemented and reverse methods, which are complicated. A tricky bit is making int a structural subtype of Integral, since int doesn't do the overloading isinstance magic that other Integral subclasses are supposed to use.

[JukkaL]

I still think that supporting numbers properly is not important (even if it might be technically feasible), and it's better to just give a warning if somebody uses any of the types in an annotation. Making them protocols would require changes to the stdlib, since they are defined as regular ABCs.

[ilevkivskyi]

@JukkaL I think it may be possible to make an exception for numbers ABCs, so that they will be defined as protocols in typeshed, but isinstance with them will use nominal bases only. (But I didn't think about this carefully, there may be some problems.)

[JukkaL]

@ilevkivskyi There is the problem that the official docs (https://docs.python.org/2/library/numbers.html#implementing-the-arithmetic-operations) suggest that implementations of operator methods should use checks like isinstance(other, Integral). This doesn't work at runtime for structural subtypes of Integral since Integral is a regular ABC. If mypy treated Intregral as a protocol, it would mean that static checking and runtime semantics would diverge, which would result in major confusion, in my opinion. I don't see how we can work this around.

The above is just one of the problems I am aware of with numbers. I don't know how we can make int a structural subtype of Integral. For example, apparently the __radd__ of Integral should accept Integral but int.__radd__ only accepts int.

If you want to convince me that this will work, I'd like to see a stub definition of subsets of, say, Real and Integral that only define a few methods such as __add__, __radd__, __int__ and __float__ and have these properties:

• Integral is a subprotocol of Real
• int is compatible with Integral
• A concrete Integral subclass implemented as suggested in https://docs.python.org/2/library/numbers.html#implementing-the-arithmetic-operations type checks cleanly (and is compatible with Integral, of course).
• There aren't Any types or # type: ignores (preferably).
• Various operations have the expected static types, such as MyIntegral + MyIntegral has type MyIntegral and MyIntegral + UnrelatedIntegral has type int (or Integral?).

[ilevkivskyi]

@JukkaL
Yes, this is indeed hard. I think it may be possible to use self-types to "emulate" covariant args in protocols. Although I don't have time to think about this carefully now, this looks like an interesting problem to play with at some point. Thanks!

[JukkaL]

I think that this is a complicated problem and a solution would only be situationally useful, so this doesn't seem worth the effort.