[red-knot] Add `typing.Any` as a spelling for the Any type #14742

dcreager · 2024-12-02T19:50:23Z

We already had a representation for the Any type, which we would use e.g. for expressions without type annotations. We now recognize typing.Any as a way to refer to this type explicitly. Like other special forms, this is tracked correctly through aliasing, and isn't confused with local definitions that happen to have the same name.

Closes #14544

We already had a representation for the Any type, which we would use e.g. for expressions without type annotations. We now recognize `typing.Any` as a way to refer to this type explicitly. Like other special forms, this is tracked correctly through aliasing, and isn't confused with local definitions that happen to have the same name.

crates/red_knot_vendored/vendor/typeshed/stdlib/typing.pyi

github-actions · 2024-12-02T20:37:55Z

`ruff-ecosystem` results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

✅ ecosystem check detected no linter changes.

crates/red_knot_python_semantic/resources/mdtest/annotations/any.md

crates/red_knot_python_semantic/src/types.rs

crates/red_knot_vendored/vendor/typeshed/stdlib/typing.pyi

This goes back to the actual definition of `typing.Any`.

carljm

This is excellent, thank you!!

A few suggested changes on non-obvious subtleties.

crates/red_knot_python_semantic/src/types.rs

crates/red_knot_python_semantic/src/types/infer.rs

carljm · 2024-12-02T22:34:57Z

crates/red_knot_python_semantic/src/types/mro.rs

@@ -377,6 +377,7 @@ impl<'db> ClassBase<'db> {
                | KnownInstanceType::Union
                | KnownInstanceType::NoReturn
                | KnownInstanceType::Never
+                | KnownInstanceType::Any


It's under-specified, and perhaps unfortunate, but the typing spec does actually permit inheriting from Any: https://typing.readthedocs.io/en/latest/spec/special-types.html#any

And this is used in typeshed (e.g. for unittest.mock.Mock): https://github.com/python/typeshed/blob/main/stdlib/unittest/mock.pyi#L109

This is already supported by the ClassBase enum: we should return Some(Self::Any) instead of None for KnownInstanceType::Any.

I've added some tests for subclasses; see below

crates/red_knot_python_semantic/src/types.rs

crates/red_knot_python_semantic/src/types/infer.rs

crates/red_knot_python_semantic/src/types.rs

dcreager · 2024-12-03T19:57:41Z

crates/red_knot_python_semantic/src/types/mro.rs

@@ -377,6 +377,7 @@ impl<'db> ClassBase<'db> {
                | KnownInstanceType::Union
                | KnownInstanceType::NoReturn
                | KnownInstanceType::Never
+                | KnownInstanceType::Any


I've added some tests for subclasses; see below

crates/red_knot_python_semantic/resources/mdtest/annotations/any.md

dcreager · 2024-12-03T20:00:27Z

crates/red_knot_python_semantic/resources/mdtest/annotations/any.md

+x: Subclass = 1
+y: int = Subclass()


I think I've updated the inference logic correctly to treat instances of Subclass as if they were Any instead of Instance(Subclass). These lines both got assignment errors before the change.

I don't think we want to make that change; Subclass should still have the type Instance(Subclass), not the type Any. Our representation of the class Subclass will have Any in its method resolution order (MRO), which will impact how we treat it in some cases. For example, it should be assignable to any other non-final type, since it might be a subclass of that type, as mentioned below, and accessing an unknown attribute on it should give an attribute type of Any (the unknown base type might have that attribute, with some unknown type). But these are cases that we will implement in e.g. Type::is_assignable_to and Type::member, not by making the subclass of Any equivalent to Any.

carljm

This is quite impressive for your first PR! Nice work.

Unfortunately I wasn't clear about the semantics of inheriting Any (because I wasn't actually expecting you to go ahead and implement those semantics in this PR!), so I think we'll need to revert a couple of the changes here. I would suggest leaving the semantics of inheriting Any to a separate PR.

crates/red_knot_python_semantic/resources/mdtest/annotations/any.md

carljm · 2024-12-03T20:33:17Z

crates/red_knot_python_semantic/resources/mdtest/annotations/any.md

+# Since Subclass is a subclass of Any, it is assignable to and from any other type, just like Any.
+x: Subclass = 1
+y: int = Subclass()


I think this comment, and the line x: Subclass = 1, are not quite right; that latter assignment should be an error. Subclass is not equivalent to Any; it's a distinct class, which has an unknown base class. There is no "materialization" (using the term from the typing spec, to mean "replacing Any with a concrete static type") of the unknown base class that could possibly result in 1 being an instance of Subclass.

The second test line here is correct; since Subclass inherits from an unknown type, that unknown type could be (could materialize to) int or a subclass of int, therefore we should allow the assignment y: int = Subclass().

You can see that both pyright and mypy agree with this, via their online playgrounds.

Just for future reference, not because it's relevant to this PR: one place where I think mypy and pyright get it wrong is that they also allow the assignment x: Fin = Sub() in this code sample. Because Fin is a final class, it can have no subclasses, therefore the unknown base class of Sub can't materialize to Fin, therefore I don't think that assignment should be allowed. But this is for future: we don't even have support for final classes in red-knot at all yet.

carljm · 2024-12-03T20:38:06Z

crates/red_knot_python_semantic/resources/mdtest/annotations/any.md

+x: Subclass = 1
+y: int = Subclass()


I don't think we want to make that change; Subclass should still have the type Instance(Subclass), not the type Any. Our representation of the class Subclass will have Any in its method resolution order (MRO), which will impact how we treat it in some cases. For example, it should be assignable to any other non-final type, since it might be a subclass of that type, as mentioned below, and accessing an unknown attribute on it should give an attribute type of Any (the unknown base type might have that attribute, with some unknown type). But these are cases that we will implement in e.g. Type::is_assignable_to and Type::member, not by making the subclass of Any equivalent to Any.

crates/red_knot_python_semantic/src/types.rs

* main: [`ruff`] Extend unnecessary-regular-expression to non-literal strings (`RUF055`) (#14679) Minor followups to RUF052 (#14755) [red-knot] Property tests (#14178) [red-knot] `is_subtype_of` fix for `KnownInstance` types (#14750) Improve docs for flake8-use-pathlib rules (#14741) [`ruff`] Implemented `used-dummy-variable` (`RUF052`) (#14611) [red-knot] Simplify tuples containing `Never` (#14744) Possible fix for flaky file watching test (#14543) [`flake8-import-conventions`] Improve syntax check for aliases supplied in configuration for `unconventional-import-alias (ICN001)` (#14745) [red-knot] Deeper understanding of `LiteralString` (#14649) red-knot: support narrowing for bool(E) (#14668) [`refurb`] Handle non-finite decimals in `verbose-decimal-constructor (FURB157)` (#14596) [red-knot] Re-enable linter corpus tests (#14736)

carljm

Looks excellent! Two very small wording nits. Feel free to merge, and congrats on landing your first red-knot PR!

crates/red_knot_python_semantic/resources/mdtest/annotations/any.md

* main: [red-knot] Test: Hashable/Sized => A/B (#14769) [`flake8-type-checking`] Expands TC006 docs to better explain itself (#14749) [`pycodestyle`] Handle f-strings properly for `invalid-escape-sequence (W605)` (#14748) [red-knot] Add fuzzer to catch panics for invalid syntax (#14678) Check `AIR001` from builtin or providers `operators` module (#14631) [airflow]: extend removed names (AIR302) (#14734)

dcreager force-pushed the dcreager/typing-any branch from 4f9c140 to 59e9670 Compare December 2, 2024 19:51

dcreager commented Dec 2, 2024

View reviewed changes

crates/red_knot_vendored/vendor/typeshed/stdlib/typing.pyi Outdated Show resolved Hide resolved

Reflow commentary in Any mdtest

bb2e7d5

carljm reviewed Dec 2, 2024

View reviewed changes

crates/red_knot_python_semantic/resources/mdtest/annotations/any.md Show resolved Hide resolved

crates/red_knot_python_semantic/src/types.rs Outdated Show resolved Hide resolved

crates/red_knot_vendored/vendor/typeshed/stdlib/typing.pyi Outdated Show resolved Hide resolved

dcreager added 3 commits December 2, 2024 15:56

Revert modified vendored typeshed

0bde1ad

This goes back to the actual definition of `typing.Any`.

Verify correct Any via an assignable check

0f56794

Make Any detection slightly more like other detection

f36e533

dcreager marked this pull request as ready for review December 2, 2024 21:11

dcreager requested review from MichaReiser, AlexWaygood and sharkdp as code owners December 2, 2024 21:11

AlexWaygood added the red-knot Multi-file analysis & type inference label Dec 2, 2024

carljm reviewed Dec 2, 2024

View reviewed changes

dcreager added 3 commits December 3, 2024 09:01

Remove unnecessary comments

62f432f

Don't need to look for typing_extensions.Any

0199648

Handle subclasses of Any correctly

a2dd5a8

dcreager commented Dec 3, 2024

View reviewed changes

carljm reviewed Dec 3, 2024

View reviewed changes

dcreager added 5 commits December 3, 2024 16:24

Revert incorrect handling of Any as superclass

4c03505

Use impl parameter type

b733211

Include basic mro test

d5b2d3b

Satisfy the linter

0ed09b6

carljm mentioned this pull request Dec 3, 2024

[red-knot] Gradual forms do not participate in equivalence/subtyping #14758

Merged

carljm approved these changes Dec 3, 2024

View reviewed changes

crates/red_knot_python_semantic/resources/mdtest/annotations/any.md Outdated Show resolved Hide resolved

crates/red_knot_python_semantic/resources/mdtest/annotations/any.md Show resolved Hide resolved

dcreager added 2 commits December 4, 2024 09:47

Refine comments in mdtest

3cf67d5

dcreager merged commit 8b23086 into main Dec 4, 2024
21 checks passed

dcreager deleted the dcreager/typing-any branch December 4, 2024 14:56

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[red-knot] Add `typing.Any` as a spelling for the Any type #14742

[red-knot] Add `typing.Any` as a spelling for the Any type #14742

dcreager commented Dec 2, 2024 •

edited

Loading

github-actions bot commented Dec 2, 2024 •

edited

Loading

carljm left a comment •

edited

Loading

carljm Dec 2, 2024

dcreager Dec 3, 2024

dcreager Dec 3, 2024

dcreager Dec 3, 2024

carljm Dec 3, 2024

carljm left a comment

carljm Dec 3, 2024

carljm Dec 3, 2024

carljm Dec 3, 2024

carljm left a comment

[red-knot] Add typing.Any as a spelling for the Any type #14742

[red-knot] Add typing.Any as a spelling for the Any type #14742

Conversation

dcreager commented Dec 2, 2024 • edited Loading

github-actions bot commented Dec 2, 2024 • edited Loading

ruff-ecosystem results

Linter (stable)

Linter (preview)

carljm left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

carljm left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

carljm left a comment

Choose a reason for hiding this comment

[red-knot] Add `typing.Any` as a spelling for the Any type #14742

[red-knot] Add `typing.Any` as a spelling for the Any type #14742

dcreager commented Dec 2, 2024 •

edited

Loading

github-actions bot commented Dec 2, 2024 •

edited

Loading

`ruff-ecosystem` results

carljm left a comment •

edited

Loading