Fix all mypy issues

[Avasam]

Summary of changes

Progress towards #2345
I'm not familiar with tox or this project's CI setup, could I get a hand to ensure that mypy errors are blocking on the CI now that they're all fixed with base configuration?

Changes:

• Fixed all mypy issues with base configurations
• Updated mypy.ini to fit the current state of type-checking in this project:
  • Added comments to explain configuration decisions
  • Explicitly target oldest supported Python version
  • Excluded vendored modules, the build folder, dynamic "extern" modules and a file that was causing "duplicate module name" issues with mypy

mypy . --python-version=3.8|3.11 --platform=linux|win32 results:

A handful of separate PRs have been created and split from this one following review discussion:

• Update Python < 3.8 (3.2 to 3.7) obsolete code and comments #4096
• Remove possibly redundant importlib_machinery code #4097
• Extract AbstractSandbox._mk_* methods #4099 or Call __func__ for AbstractSandbox's staticmethods #4227
• Mark interfaces as Protocols and add missing self argument #4144
• Bump packaging version in tests to 23.2 #4231
• Remove IDE-specific folder from .gitignore #4232
• Bump pytest-mypy jaraco/skeleton#96
• posixpath.commonpath supports Iterable parameter python/typeshed#11310
• shlex.split: allow TextIO and deprecate None python/typeshed#11451
• Improve typing of sysconfig.get_config_var(s) python/typeshed#11454
• Accept None in _XfailMarkDecorator's conditionparam and expose default value pytest-dev/pytest#11565
• Use False, instead of None as default for _XfailMarkDecorator's condition param and update doc pytest-dev/pytest#11600

Pull Request Checklist

• Changes have tests (see my comment/question about enforcing this in the CI)
• News fragment added in newsfragments/. (should not affect the end user yet)
  (See documentation for details)

[Avasam]

@jaraco (hopefully pinging the right person, you're the one who answered in #2345)

Could I get a review on this? And a hand with the CI checks (as mentioned, I'm not familiar with tox, but I know how to add standalone PR type-checking jobs as GitHub actions)

[jaraco]

(closed accidentally)

[jaraco]

Thanks for the contrib! I've got some serious concerns about the checks for vendored modules, and some other suggestions about the approach, but for the most part it looks good.

* Use `pyproject.toml` instead of `mypy.ini` to merge tooling configurations (it also makes [exclude](https://mypy.readthedocs.io/en/stable/config_file.html#confval-exclude) configuration a bit cleaner as it supports arrays in `.toml` format)
  * If you still prefer having a standalone `mypy.ini`, I'll move it back and adapt the `exclude` config.

I do wish to retain the mypy settings in mypy.ini for two reasons:

• The skeleton for this project is maintained upstream, which defines the mypy settings in its own file. Moving the settings to a different file makes it difficult to compare and manage the differences with settings inherited from upstream.
• Piling all project settings in pyproject.toml is a bad idea in general. I've been meaning to write a blog post about it, but the tl;dr is that because there's no ordering or namespacing, the settings lack structure and consistency (compared with mypy which always appears after docs/ and before newsfragments/ and is clearly distinct from towncrier or tox or pytest settings.

[Avasam]

Btw, with over 40 commits and lots of "oops" commits. Is this gonna be squashed merged? Or should I squash my git history?

[abravalheri]

Btw, with over 40 commits and lots of "oops" commits. Is this gonna be squashed merged? Or should I squash my git history?

If you are more comfortable that way, I think you can do is a git rebase -i and organise the commit history in a way that makes sense to you. Then just update the PR with a push-force... Otherwise, if you tell me that it should be squashed as a whole, I am also happy to oblige.

FYI, I recently merged a bugfix contributed by a member of that community that was having problems with building. That affects primarily build_meta, but I see that the file is not changed in this PR, so hopefully it will not cause too much trouble 🤞.

[Avasam]

I'm comfortable and experienced with interactive rebases, but for this PR you can just squash as a whole.

With typeshed I've taken the habit of not force-pushing between reviews, as it is the policy there, since GitHub makes it harder to view changes since last review if the history is force-pushed, since it's based on commit history (unlike DevOps for example, where update history is separate from commits)

[abravalheri]

Thank you very much @Avasam! That was a heroic piece of work.

---

I have a doubt:

How are you running mypy? When I try to run mypy --python-version=3.8 --platform=linux ., I still get back 3 errors, which I suppose should be ignored as configured in mypy.ini:

$ mypy --python-version=3.8 --platform=linux .
setuptools/command/build_clib.py:6: error: Cannot find implementation or library stub for module named "distutils._modified"  [import-not-found]
setuptools/command/build_clib.py:6: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
setuptools/config/_validate_pyproject/formats.py:180: error: Cannot find implementation or library stub for module named "trove_classifiers"  [import-not-found]
setuptools/command/build_ext.py:19: error: Cannot find implementation or library stub for module named "Cython.Distutils.build_ext"  [import-not-found]
Found 3 errors in 3 files (checked 146 source files)

I can see that at least setuptools/_distutils and setuptools/config/_validate_pyproject are excluded in mypy.ini.
(I have installed mypy==1.8.0, mypy-extensions==1.0.0, typing_extensions==4.9.0)

Is this something that is supposed to happen?

[Avasam]

@abravalheri Thank you too for your support!

The last two (trove_classifiers and Cython) are just missing installs, although I think Cython isn't installed in CI. For distutils._modified I don't remember why it could differ, maybe because I have an editable install of setuptools 🤷‍♂️

Since I figured out how to enable mypy in CI, I have a very small follow-up PR that does just that an appropriately ignores "missing-imports" that happens on CI with a comment. #4257