gh-114053: Fix another edge case involving get_type_hints, PEP 695 and PEP 563

[AlexWaygood]

This PR fixes a bug in #118009 that can be seen in the following REPL demo, where get_type_hints under from __future__ import annotations semantics won't detect if a variable defined in a type-parameter namespace has been overidden in a local namespace:

Python 3.14.0a0 (heads/getannotations-nameerror:ec4848f2c8b, Jun  8 2024, 15:28:33) [Clang 15.0.0 (clang-1500.3.9.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> class Foo[T]:
...     T = int
...     x: T
...     
>>> Foo.__annotations__
{'x': <class 'int'>}
>>> from __future__ import annotations
>>> class Foo[T]:
...     T = int
...     x: T
...     
>>> from typing import get_type_hints
>>> get_type_hints(Foo)
{'x': T}

The cause of the bug is that get_type_hints does this weird swapping of globals and locals, and the logic added in #118009 doesn't account for that -- it injects the type parameters into the dictionary called localns (which is actually the globals) when it should be injecting the type parameters into globalns (which is actually the locals).

I actually added a test for this exact edge case in #118009, which should have caught the bug. But there was a bug in the test, because the test was trying to be too clever, and I got mixed up between ^ and & when you're using sets. Had I not tried to write such a clever test, I would have caught the bug.

• Issue: typing.get_type_hints fails when passed a class with PEP 695 type parameters and PEP 563 is enabled #114053

[JelleZijlstra]

Can you add a test where T also exists in the global scope, ensuring that the type param is picked up over the global?

[AlexWaygood]

Can you add a test where T also exists in the global scope, ensuring that the type param is picked up over the global?

Thanks, good catch -- that test failed. I added it and fixed it.

[AlexWaygood]

Okay, this still doesn't get things right in all cases :(

>>> from __future__ import annotations
>>> from typing import get_type_hints
>>> Eggs, Spam = int, str
>>> def f[Eggs, **Spam](x: Eggs, y: Spam): pass
... 
>>> get_type_hints(f)
{'x': <class 'int'>, 'y': <class 'str'>}

[AlexWaygood]

Okay, this still doesn't get things right in all cases :(

>>> from __future__ import annotations
>>> from typing import get_type_hints
>>> Eggs, Spam = int, str
>>> def f[Eggs, **Spam](x: Eggs, y: Spam): pass
... 
>>> get_type_hints(f)
{'x': <class 'int'>, 'y': <class 'str'>}

Should have been fixed in 2fbcd7b.

[JelleZijlstra]

Why do we need the __forward_is_class__ condition? I don't understand why class-scoped forwardrefs are special here.

[AlexWaygood]

For functions, names in type_params should always take highest priority, and this is implemented by inserting the type params into the local namespace (which is here called globalns) and popping them from the global namespace (which is here called localns). For classes, we only want to give type_params highest priority if they haven't been overridden in the local scope, however.

[JelleZijlstra]

Why do we need to do this? Shouldn't the local namespace override the type params?

Test case:

class X[T]:
    T = int
    x: T

[AlexWaygood]

I already have that test case included in this PR, and it fails unless we have this block of code here. The local namespace shoudl indeed override the type param, but the locals here are found in the globalns, and the globals here are found in the localns variable. This highly confusing situation is because the globals and locals are swapped higher up in the call stack here, for backwards compatibility reasons if you believe the comment:

cpython/Lib/typing.py

Lines 2419 to 2426 in a3711af

	if localns is None and globalns is None:
	# This is surprising, but required. Before Python 3.10,
	# get_type_hints only evaluated the globalns of
	# a class. To maintain backwards compatibility, we reverse
	# the globalns and localns order so that eval() looks into
	# *base_globals* first rather than *base_locals*.
	# This only affects ForwardRefs.
	base_globals, base_locals = base_locals, base_globals

[JelleZijlstra]

I saw that comment, maybe we should find a way to get rid of that compatibility hack.

[AlexWaygood]

I don't really see a way round it, short of deprecating get_type_hints altogether. Which... I'm not 100% against, but it would be quite disruptive.

[JelleZijlstra]

There could be other approaches, like adding new keyword arguments. Will have to think about it more.

[picnixz]

If you decide to add keywords arguments or deprecate things, please tell me in advance so that I can be reactive on Sphinx' side if needed.

[AlexWaygood]

@JelleZijlstra, are you okay with this going in? I'd love to get this into the next 3.13 beta if possible (but at the same time, the bug also exists in Python 3.12, so it wouldn't be a disaster if it waited until beta 4).

I agree that the implementation is kinda weird but I'm not sure there's a better way of doing it until we've sorted out the strange globals/locals swapping (which I think would probably be harder to backport to 3.12).

[miss-islington-app]

Thanks @AlexWaygood for the PR 🌮🎉.. I'm working now to backport this PR to: 3.12, 3.13.
🐍🍒⛏🤖

[bedevere-app]

GH-121003 is a backport of this pull request to the 3.13 branch.

[bedevere-app]

GH-121004 is a backport of this pull request to the 3.12 branch.