Make ExitStack, AbstractContextManager and AsyncAbstractContextManager generic in return type of __exit__

[Daverball]

See https://discuss.python.org/t/add-an-else-clause-to-with-statements-to-detect-early-termination/38031 for related discussion.

[Daverball]

Quite a few mypy primer hits here. This change probably does not make sense until after PEP696 support.

[srittau]

See #11422 for the type var generics feature tracker.

[Daverball]

This is really encouraging, PEP-696 truly shines here. I'll check if there are any other places in typeshed that use AbstractContextManager/AbstractAsyncContextManager that should have their second parameter specified.

---

I fixed the obvious ones, the other ones are probably fine using the default as a fallback for now (or maybe we should just set them all to bool | None, so type checkers without full PEP-696 support still have a good UX).

[srittau]

TypeVar defaults are now available in typeshed.

[Daverball]

One of the mypy_primer hits is irrelevant, since it is the same unrelated error with different types and the other one looks like an improvement to me.

There are a couple of caveats to this change however:

• Right now I'm explicitly specifying the second parameter in subclasses to AbstractContextManager/AbstractAsyncContextManager in the sdlib as long as they override __exit__ with a different value than the default for clarity. Technically this shouldn't be necessary and we could keep the surface area of the change smaller. Or we could go the opposite way and make sure we always specify the second parameter everywhere in the stdlib. Either of those options would probably be more manageable for maintainers of type checkers that don't yet support PEP-696, compared to the mish mash we have now.
• typing.ContextManager and typing.AsyncContextManager are just aliases for the ABCs that changed, but this means the Python docs are no longer accurate. Technically it is a harmless mismatch, since you can omit the parameter, that is missing from the docs, but we should probably still consider updating the docs, if we accept this change.

[Daverball]

https://discuss.python.org/t/passing-only-one-typevar-of-two-when-using-defaults/49134 clued me into an issue, since at runtime typing.ContextManager/typing.AsyncContextManager inherit from Generic/Protocol they will emit runtime errors if you try to use the second argument, this is in contrast to any __class_getitem__ methods added to builtins and ABC, which don't validate the number of arguments.

[Daverball]

The new primer hit seems fine to me. I think the only reason that previously didn't error is because AbstractContextManager[Any] was in the MRO, which used to be an alias for ContextManager[Any], so mypy took a shortcut and didn't attempt to structurally match, because the simpler subclassing check already told it everything is fine (due to Any). This could actually be considered a mypy bug. I'm not sure how I feel about subclasses that override all occurrences of a TypeVar, that was previously bound to Any with a concrete type still behaving as if the type was Any in some scenarios... Although I guess that's just the price you pay if you omit a type parameter when subclassing (or set it to Any)

[github-actions]

Diff from mypy_primer, showing the effect of this PR on open source code:

werkzeug (https://github.com/pallets/werkzeug)
+ src/werkzeug/debug/__init__.py:368: error: Unused "type: ignore" comment  [unused-ignore]

mongo-python-driver (https://github.com/mongodb/mongo-python-driver)
+ pymongo/__init__.py:176: error: Incompatible return value type (got "_TimeoutContext", expected "ContextManager[None]")  [return-value]
+ pymongo/__init__.py:176: note: Following member(s) of "_TimeoutContext" have conflicts:
+ pymongo/__init__.py:176: note:     Expected:
+ pymongo/__init__.py:176: note:         def __enter__(self) -> None
+ pymongo/__init__.py:176: note:     Got:
+ pymongo/__init__.py:176: note:         def __enter__(self) -> _TimeoutContext

[srittau]

LGTM, the new primer hits are promising. I'll leave it open a bit, in case another maintainer wants to chime in.