Allow specifying a default for omitted type parameters

[refi64]

Take the following class:

class Base: pass
class Derived(Base): pass

T = TypeVar('T', bound=Base)
class MyType(Generic[T]):
    pass

myvar: MyType = None  # type of `myvar` is `MyType[Any]`

My idea is to allow for some way to specify a default other than any when the parameters are omitted. For example:

class Base: pass
class Derived(Base): pass

T = TypeVar('T', bound=Base, default=Base)
class MyType(Generic[T]):
    pass

myvar: MyType = None  # type of `myvar` is `MyType[Base]`

or (I know this is invalid syntax, but it's the idea that counts):

class Base: pass
class Derived(Base): pass

T = TypeVar('T', bound=Base)
class MyType(Generic[T, default=Base]):
    pass

myvar: MyType = None  # type of `myvar` is `MyType[Base]`

[ilevkivskyi]

@kirbyfan64
Interesting idea. It would be easy to implement the first option in typing.py, but the difficult part would be to implement this in mypy.

(btw mypy will complain about you example with --strict-optional unless you remove None)

[refi64]

Another though: since classes can take arbitrary arguments:

class Base: pass
class Derived(Base): pass

T = TypeVar('T', bound=Base)
class MyType(Generic[T], T=Base):
    pass

myvar: MyType = None  # type of `myvar` is `MyType[Base]`

Then, on the typing.py side, GenericMeta could just ignore any keyword arguments.

[ratijas]

+1

Would be so nice to have a Result[T, E] with default E=MyCommonlyUsedError if omitted, and some other domain-specific error type otherwise.

[ratijas]

As far as I remember, for now we can not do type aliases like this either:

IResult[T] = Result[T, IError]
UResult[T] = Result[T, UError]

because of a syntax error.

Is there any way around until the subject is implemented?

[JelleZijlstra]

You can just write IResult = Result[T, IError], and the alias will be generic (so you can write IResult[int]).

[ratijas]

@JelleZijlstra Brilliant! Thanks!

[srittau]

Another example where this could be beneficial, from typeshed:

_T = TypeVar("_T", default=None)
def nullcontext(enter_result: _T = ...) -> ContextManager[_T]: ...

At the moment this uses an overload.

[Gobot1234]

I've written up a draft PEP for this feature if anyone in this thread is still around and wants to give it a look feedback would be appreciated https://gist.github.com/Gobot1234/8c9bfe8eb88f5ad42bf69b6f118033a7

[erictraut]

Thanks @Gobot1234 for writing up the PEP. Overall, I think it's looking good.

A few thoughts...

You mention that Foo[()] can be used as an alias to Foo when all of the type variables have defaults. Are these completely equivalent? If so, why support the former? It just introduces a second way to do the same thing — and adds complexity and ambiguity. Or are you thinking that the former would be preferred as an explicit form of Foo, and type checkers could optionally flag Foo as an undesired implicit form? Today it's common for developers to forget to add type arguments for a generic type — either out of ignorance or laziness, and the result is an unexpected behavior and "holes" in type checking. With a default value, the undesirable behaviors largely go away, but I'm still unsure whether we want to encourage the use of Foo rather than Foo[()] because the latter is more explicit than the former. I'm interested in your thoughts here.

For TypeVarTuples, Foo[()] already has a defined meaning, so this form would mean something different in the context of a TypeVarTuple.

PEP 484 is very clear that the bound type cannot be parameterized by type variables.

A type variable may specify an upper bound using bound= (note: itself cannot be parameterized by type variables).

The section "Using other TypeVar-likes as the default" implies that other TypeVars can be used in a "default". I'm strongly opposed to this. This adds significant complexity and opens up lots of questions about type variable scoping rules. It will also likely get in the way of ongoing attempts to improve syntax for type variable declaration. I strongly recommend that you change the PEP to be clear that the default cannot be generic and cannot contain any type variables. The "default" value should be a concrete type, just like "bound". We could revisit this limitation in the future if HKT's are added to the type system, but for now I think this is a really important constraint.

There's a small typo in the example you provide under the heading "Function Defaults". The type annotation for t should be DefaultIntT (i.e. it's missing the final T).

I found the phrase "a type variable appearing only once in the signature" ambiguous. Normally "signature" refers to the input and return parameters for a function, but I think you're using it here to refer only to the input parameters.

The phrase "the parameter's default" confused me on first read. I think you're referring to the default argument value associated with the parameter, not the default type associated with the parameter's TypeVar annotation. Since there are two "defaults" involved here, a few extra adjectives would help resolve the ambiguity.

The statement "If a TypeVar with a default annotates a function parameter: the parameter's default must be specified if it only shows up once in the signature" is unclear to me. Does this apply only in cases where the parameter is annotated with a "bare TypeVar"? I presume it doesn't apply when a TypeVar is used as a type argument, as in list[DefaultIntT]? What about a union, as in DefaultIntT | None?

I like that the proposed PEP is clarifying where a default argument value can be used with a parameter annotated with a "bare TypeVar". This is a case where type checkers have diverged, and it's an opportunity to get everyone on the same page.

Under the subhead "Subscription", the code sample includes the line def bar(*ts, default_int_t): .... I was confused by this because it's an unannotated function. Did you mean to include type annotations here?

Supporting defaults for TypeVarTuple and ParamSpec adds quite a bit of complexity to the PEP, and I'm not entirely convinced of the value these provide. I understand the argument for completeness, but maybe it would be better to initially add support for TypeVar only. If and when we find that there's a compelling use case for TypeVarTuple and ParamSpec, it could be added in the future. My intuition is that there will not be a compelling use case for these. Adding support for these now might also complicate ongoing efforts to introduce a simplified syntax for TypeVars. I'm not strongly opposed to including these in the PEP, but my general philosophy when it comes to language features is "keep it simple until it becomes clear that the added complexity is justified".

[Gobot1234]

Thank you for the feedback yet again Eric.

You mention that Foo[()] can be used as an alias to Foo when all of the type variables have defaults. Are these completely equivalent? If so, why support the former? It just introduces a second way to do the same thing — and adds complexity and ambiguity. Or are you thinking that the former would be preferred as an explicit form of Foo, and type checkers could optionally flag Foo as an undesired implicit form? Today it's common for developers to forget to add type arguments for a generic type — either out of ignorance or laziness, and the result is an unexpected behavior and "holes" in type checking. With a default value, the undesirable behaviors largely go away, but I'm still unsure whether we want to encourage the use of Foo rather than Foo[()] because the latter is more explicit than the former. I'm interested in your thoughts here.

For TypeVarTuples, Foo[()] already has a defined meaning, so this form would mean something different in the context of a TypeVarTuple.

Ideally I'd like the two to be strictly equivalent and neither would involve Unknowns, but as you've pointed out this wouldn't work for TypeVarTuples so, I've chosen to remove it.

PEP 484 is very clear that the bound type cannot be parameterized by type

A type variable may specify an upper bound using bound= (note: itself cannot be parameterized by type variables).

The section "Using other TypeVar-likes as the default" implies that other TypeVars can be used in a "default". I'm strongly opposed to this. This adds significant complexity and opens up lots of questions about type variable scoping rules. It will also likely get in the way of ongoing attempts to improve syntax for type variable declaration. I strongly recommend that you change the PEP to be clear that the default cannot be generic and cannot contain any type variables. The "default" value should be a concrete type, just like "bound". We could revisit this limitation in the future if HKT's are added to the type system, but for now I think this is a really important constraint.

After careful consideration it pains me to agree with you, I've removed this section from the draft. I'd really like to update this to include this feature when HKT is implemented.

There's a small typo in the example you provide under the heading "Function Defaults". The type annotation for t should be DefaultIntT (i.e. it's missing the final T).

Whoops, thank you, fixed that.

I found the phrase "a type variable appearing only once in the signature" ambiguous. Normally "signature" refers to the input and return parameters for a function, but I think you're using it here to refer only to the input parameters.

Yep, you're correct, I've changed this.

The phrase "the parameter's default" confused me on first read. I think you're referring to the default argument value associated with the parameter, not the default type associated with the parameter's TypeVar annotation. Since there are two "defaults" involved here, a few extra adjectives would help resolve the ambiguity.

Done.

The statement "If a TypeVar with a default annotates a function parameter: the parameter's default must be specified if it only shows up once in the signature" is unclear to me. Does this apply only in cases where the parameter is annotated with a "bare TypeVar"? I presume it doesn't apply when a TypeVar is used as a type argument, as in list[DefaultIntT]? What about a union, as in DefaultIntT | None?

I've changed the wording to "Defaults for parameters aren't required if other parameters annotated with the same TypeVar already have defaults". So this should apply to all the previously mentioned types, bare TypeVars, type arguments and unions.

Under the subhead "Subscription", the code sample includes the line def bar(*ts, default_int_t): .... I was confused by this because it's an unannotated function. Did you mean to include type annotations here?

No the annotations aren't important, it's just to show the current behaviour in Python. I've attempted to make this slightly clearer, so thank you for pointing this out.

Supporting defaults for TypeVarTuple and ParamSpec adds quite a bit of complexity to the PEP, and I'm not entirely convinced of the value these provide. I understand the argument for completeness, but maybe it would be better to initially add support for TypeVar only. If and when we find that there's a compelling use case for TypeVarTuple and ParamSpec, it could be added in the future. My intuition is that there will not be a compelling use case for these. Adding support for these now might also complicate ongoing efforts to introduce a simplified syntax for TypeVars. I'm not strongly opposed to including these in the PEP, but my general philosophy when it comes to language features is "keep it simple until it becomes clear that the added complexity is justified".

Ok I've removed this section as well.

I'll publish the changes tomorrow/today

[mscuthbert]

This would be so valuable for a project of mine (music21) where our main container (Stream) holds any type of subclass of Music21Object (notes, chords, keys, tempos, etc.) in 99% of the cases, but specifying that all elements are of a certain class is extremely valuable in some cases. Right now, mypy and other type checkers are asking people to specify Stream[Music21Object]() every time, even though Music21Object is already the bound for the type.

Hope that this doesn't get stalled so far that it's not in 3.12. :-)

[gvanrossum]

Hm, I'm not actually aware of a PEP for this, so maybe it will miss 3.12, alas. It's not a syntactic feature, so presumably it could be backported using typing_extensions easily. Nevertheless, there would have to be a PEP first. (Edit: and that PEP would have to be accepted.)

[JelleZijlstra]

This is PEP 696.

[gvanrossum]

Whoops. Never mind me.

[erictraut]

This has been specified in PEP 696 which was recently approved.