Python 3.10 tracking issue

[freundTech]

Here's an issue for tracking the current state of mypy on CPython 3.10

CPython version: 3.10.0
mypy version: latest master (e6b91bd)

Dependencies:

• typed_ast is currently broken. Fixed on master, but no release to pypi yet (Allow compilation on python 3.10.0a6 typed_ast#155)
• typed_ast again broken in 3.10.0a7. PR Fix compilation on python 3.10.0a7 typed_ast#158
• lxml is currently broken due to a cython bug. The bug has been fixed in cython 0.29.22, but lxml hasn't been built with the new version yet (https://bugs.launchpad.net/lxml/+bug/1904900)
• virtualenv uses distutils, which is deprecated and will be removed in python 3.12. Mypy test-depends on virtualenv<20, and it looks like only virtualenv >=20 is still in active development. (Virtualenv pinned to <20 #10407)
• pytest breaks on python 3.10. This is already fixed upstream, but mypy pins pytest to <6.2, because of python 3.5 support

Tests:

• Some tests from FineGrainedSuite, FineGrainedCacheSuite and PythonCmdlineSuite` are broken, because cpython parser error messages changed. (Fix a bunch of tests broken by python 3.10 #10404)
  • Some occurrences of error: invalid syntax are now error: expected ':'
  • Some occurrences of error: unexpected EOF while parsing are now error: '(' was never closed
• Tests from SamplesSuite break with mypy/typeshed/stdlib/types.pyi:334: error: Name '_NotImplementedType' is not defined (Small fixes for Python 3.10 typeshed#5044)
• Enable CI for python 3.10 (Enable CI to use Python 3.10 #11294)

Features

• Pattern matching
  • Pattern matching is not yet supported (Support for python 3.10 match statement #10191)
  • Exhaustiveness checking for pattern matching (Reachability/exhaustiveness checking for match statements #12010)
  • Pattern matching in mypyc (Compiling code with match statements fails with an exception mypyc/mypyc#911)
• Support explicit TypeAlias (Support PEP 613 #9404)
• Support PEP 604 syntax for runtime use cases (Support PEP 604 syntax for runtime use cases #9880)
  • PEP 604 for TypeVar bounds new union syntax (types.Union) doesn't work on generic bounds #10949
• Support for dataclasses' kw_only ([dataclasses plugin] Support kw_only=True #10867)
• Support PEP 612 Parameter Specification Variables (PEP 612 tracking issue #8645)

Breaking changes

• mypyc is broken, because _PyObject_HasAttrId has been removed from CPython (bpo-41991: Remove _PyObject_HasAttrId cpython#22629). PR Fix mypyc failing to compile on CPython 3.10.0a6 #10202.
• mypy/sitepkgs.py imports distutils, which is deprecated and will be removed in python 3.12 (Only import distutils if it is needed #10203).

Misc

• Build Python 3.10 wheels
• Add Python 3.10 classifier

[hauntsaninja]

Here's another important part to 3.10 support: #9880

[erikkemperman]

Not sure if this ought to be mentioned here, but test-requirements.txt pins the pytest dependency to <6.2.0, because they dropped support for Python 3.5 with their 6.2 release.

However, the tests currently fail with the Python 3.10 beta due to this bug. I've submitted a PR to backport the fix to pytest 6.1.x but it was rejected because they have a policy of not backporting to versions prior to the latest minor release.

So I guess, assuming dropping Python 3.5 support is still not an option for mypy, that the dependencies would somehow have to be split in such a way that pytest 6.1.x is used with Python 3.5 and the latest pytest can be used for Python 3.6 and up. This particular bug was fixed in pytest 6.2.4, for what it's worth.

[domdfcoding]

Could you use PEP 508 markers in the requirements file to specify the older pytest for 3.5 and a newer version for 3.6 and later?

Something like pytest<6.2.0; python_version < "3.6"

[erikkemperman]

Perhaps. I'm not sure what the best way to do this "split" would be, to be honest, I just wanted to signal here that something may have to be done about this.

[freundTech]

Mypy is in the process of dropping python 3.5 support (See #9950) and CI runs on python 3.5 have already been disabled (#10345), so if python 3.5 support is dropped before python 3.10 is released the pin on pytest could just be removed entirely.

CC: @JukkaL

[cdce8p]

Python 3.10 also adds TypeAlias (PEP 613): #9404

[erikkemperman]

@freundTech Ah, sorry, I had not seen that. Yes, in that case it can just be set to >= 6.2.4 (to include the fix for this particular issue). Thanks!

[cdce8p]

3.10 also adds KW_ONLY args to dataclasses: #10865, open PR: #10867
https://docs.python.org/3.10/library/dataclasses.html#dataclasses.KW_ONLY

[domdfcoding]

I think the _NotImplementedType issue might be sorted with Python 3.10.0rc1. I'm no longer seeing it in my own code where mypy used to flag it, and running the mypy testsuite I can't see any failures related to it.

[hauntsaninja]

Likely because python/typeshed#5044 made it onto mypy master

[cdce8p]

Another feature: PEP 612 (Parameter Specification Variables) #8645

[JukkaL]

Now that Python 3.10 is out, we should enable CI using the release version. A few tests seem to be failing, however, and may need to be fixed first. Anybody want to help with this (#11294)?

Once CI is up, we should start building 3.10 wheels in https://github.com/mypyc/mypy_mypyc-wheels.

[cpontvieux-systra]

Could a preliminary version for python 3.10 be released?

• MR Support for python 3.10 match statement #10191 seems to be ok (just need a rebase/conflict resolve)
• PEP 612 could be addressed in a seperate issue

What do you think?

[Torxed]

I'd like to get this out there, and maybe this is just venting some frustration. But I find it strange that Python 3.10 was released and considered stable by the Python foundation when the associated tool chain doesn't support it. This breaks quite a few things without a clear heads up and finding information on this topic is scattered at best.

I wouldn't mind hacky CI pipelines pulling master or some feature branch, but it's not clear which branch might solve the overall issues so pulling a branch is hard. And the issues appears to have been known for some time so I'm probably repeating known facts. Or I'm very oblivious and I'm missing a simple configuration (other then downgrading) in which case apologizes.

But the roadmap on mypy-lang.org is a dead link: https://github.com/python/mypy/blob/master/ROADMAP.md.
I couldn't find my information on the FAQ either about Python 3.10. Neither does the twitter account mention it. The closest thing I could get on public sources was the blog which mentions partial support for 3.10 as if it's features that are partially supported - not runtime.

I really hope I'm overreacting and that there's a parameter I'm missing and all is well.
But this feels like more of a runtime issue than a flag missing. I'd love to help figure something out to get it running, and in my particular case it's #11851 that I'm looking in to first of all.

[andychu]

I'm also eagerly awaiting MyPy support for match, but saying it breaks something doesn't make sense. It's better to have the feature in Python 3.10 than not have it, and type checking has always been outside the core.

If you need your code to be 100% statically typed, then you can't use match yet, but that's not a breakage.

[Torxed]

I'm also eagerly awaiting MyPy support for match, but saying it breaks something doesn't make sense. It's better to have the feature in Python 3.10 than not have it, and type checking has always been outside the core.

If you need your code to be 100% statically typed, then you can't use match yet, but that's not a breakage.

The breakage in my case was "simple", the pyproject.toml configuration dropped support (?) for exclude = ["path", "file.py"] syntax. For some reason it needs to be a string. While previous versions have supported it. I couldn't find any information on this breaking change, but again, simple fix. But it was frustrating nevertheless.

I welcome match too, but that was besides my original point :)

Edit: Looks like it's being brought back in #11828

[juliancoffee]

Should we check pattern matching as complete as #10191 is merged?

[freundTech]

I just checked off #10191, but pattern matching also needs #12010, so I added that to the list.

[emmatyping]

It looks like all that is left here is PEP 612 support, which can be tracked in #8645, and mypyc does not support match statements, which I think is an issue better tracked on the mypyc issue tracker. I suggest we close this issue.