gh-104050: Run mypy on `clinic.py` in CI #104421

AlexWaygood · 2023-05-12T12:38:59Z

Add a basic mypy check to clinic.py in CI (and add some initial type annotations, so that the mypy check passes).

The check will run:

On pushes to main
On pull requests that touch files in the Tools/clinic directory, and pull requests that touch the workflow file itself
When contributors manually request a workflow on their fork, via workflow_dispatch.

When it does run, mypy will only check files inside Tools/clinic/. All other files will be excluded from the mypy check in CI.

The mypy configuration is also deliberately lax, to begin with: only a few of the stricter settings are enabled. This will allow us to make use of "gradual typing": mypy will only check functions inside Tools/clinic/ that we've added annotations to. It will assume all other functions are to be ignored by mypy, for now; we will therefore be able to gradually increase the level of type annotations inside this directory.

Issue: Add type annotations to clinic.py #104050

- ditch no-return - use callable from collections.abc - use namedtuple from typing

AlexWaygood · 2023-05-12T12:47:12Z

Tools/clinic/mypy.ini

I usually put my mypy config in a pyproject.toml file these days, but a mypy.ini file feels like it makes more "sense" unless we're planning on adding other linters/formatters to check clinic.py in the future

.github/workflows/mypy-clinic.yml

Tools/clinic/clinic.py

.github/workflows/mypy-clinic.yml

Tools/clinic/clinic.py

Tools/clinic/cpp.py

.github/workflows/mypy-clinic.yml

Tools/clinic/clinic.py

.github/workflows/mypy-clinic.yml

Co-authored-by: Nikita Sobolev <[email protected]>

Co-authored-by: Hugo van Kemenade <[email protected]>

.github/workflows/mypy.yml

Tools/clinic/requirements-dev.txt

Co-authored-by: Hugo van Kemenade <[email protected]>

AlexWaygood · 2023-05-12T14:13:23Z

Tools/clinic/requirements-dev.txt

@@ -0,0 +1,2 @@
+# Requirements file for external linters and checks we run on Tools/clinic/ in CI
+mypy==1.2.0


We're deliberately pinning to an older version here, in order to test that dependabot updates are working on this file, as per @hugovk's suggestion in #104421 (comment)

hugovk

CI changes look good 👍

Tools/clinic/cpp.py

AlexWaygood · 2023-05-12T15:52:45Z

(The test_zoneinfo failure is obviously unrelated.)

Edit: As is the test_threading failure.

Tools/clinic/mypy.ini

This reverts commit 9fcd87f.

erlend-aasland

LGTM. Thanks!

AlexWaygood · 2023-05-15T08:27:55Z

LGTM. Thanks!

Very happy to help!

* main: (29 commits) pythongh-101819: Fix _io clinic input for unused base class method stubs (python#104418) pythongh-101819: Isolate `_io` (python#101948) Bump mypy from 1.2.0 to 1.3.0 in /Tools/clinic (python#104501) pythongh-104494: Update certain Tkinter pack/place tests for Tk 8.7 errors (python#104495) pythongh-104050: Run mypy on `clinic.py` in CI (python#104421) pythongh-104490: Consistently define phony make targets (python#104491) pythongh-67056: document that registering/unregistering an atexit func from within an atexit func is undefined (python#104473) pythongh-104487: PYTHON_FOR_REGEN must be minimum Python 3.10 (python#104488) pythongh-101282: move BOLT config after PGO (pythongh-104493) pythongh-104469 Convert _testcapi/float.c to use AC (pythongh-104470) pythongh-104456: Fix ref leak in _ctypes.COMError (python#104457) pythongh-98539: Make _SSLTransportProtocol.abort() safe to call when closed (python#104474) pythongh-104337: Clarify random.gammavariate doc entry (python#104410) Minor improvements to typing docs (python#104465) pythongh-87092: avoid gcc warning on uninitialized struct field in assemble.c (python#104460) pythonGH-71383: IDLE - Document testing subsets of modules (python#104463) pythongh-104454: Fix refleak in AttributeError_reduce (python#104455) pythongh-75710: IDLE - add docstrings and comments to editor module (python#104446) pythongh-91896: Revert some very noisy DeprecationWarnings for `ByteString` (python#104424) Add a mention of PYTHONBREAKPOINT to breakpoint() docs (python#104430) ...

* main: (204 commits) pythongh-101819: Fix _io clinic input for unused base class method stubs (python#104418) pythongh-101819: Isolate `_io` (python#101948) Bump mypy from 1.2.0 to 1.3.0 in /Tools/clinic (python#104501) pythongh-104494: Update certain Tkinter pack/place tests for Tk 8.7 errors (python#104495) pythongh-104050: Run mypy on `clinic.py` in CI (python#104421) pythongh-104490: Consistently define phony make targets (python#104491) pythongh-67056: document that registering/unregistering an atexit func from within an atexit func is undefined (python#104473) pythongh-104487: PYTHON_FOR_REGEN must be minimum Python 3.10 (python#104488) pythongh-101282: move BOLT config after PGO (pythongh-104493) pythongh-104469 Convert _testcapi/float.c to use AC (pythongh-104470) pythongh-104456: Fix ref leak in _ctypes.COMError (python#104457) pythongh-98539: Make _SSLTransportProtocol.abort() safe to call when closed (python#104474) pythongh-104337: Clarify random.gammavariate doc entry (python#104410) Minor improvements to typing docs (python#104465) pythongh-87092: avoid gcc warning on uninitialized struct field in assemble.c (python#104460) pythonGH-71383: IDLE - Document testing subsets of modules (python#104463) pythongh-104454: Fix refleak in AttributeError_reduce (python#104455) pythongh-75710: IDLE - add docstrings and comments to editor module (python#104446) pythongh-91896: Revert some very noisy DeprecationWarnings for `ByteString` (python#104424) Add a mention of PYTHONBREAKPOINT to breakpoint() docs (python#104430) ...

erlend-aasland · 2023-05-16T11:32:17Z

Tools/clinic/clinic.py

@@ -2558,15 +2570,15 @@ class CConverter(metaclass=CConverterAutoRegister):
    """

    # The C name to use for this variable.
-    name = None
+    name: str | None = None


So, these str | None are now starting to cause frustrations, as I gradually add typing to other parts of clinic.py. Since there are no guards anywhere, I obviously get errors and warnings about str plus None, etc. We could change the defaults to the empty string, but that can also create a lot of havoc.

erlend-aasland and others added 6 commits May 1, 2023 18:37

Slightly annotate cpp.py

ca8532b

Slightly annotate clinic.py

d2ec2ef

Address review

801af38

- ditch no-return - use callable from collections.abc - use namedtuple from typing

Add basic mypy workflow to CI

00183ff

Make the type check pass

4bdd409

Tweak workflow file

632843b

AlexWaygood requested a review from erlend-aasland May 12, 2023 12:38

AlexWaygood requested review from ezio-melotti and hugovk as code owners May 12, 2023 12:39

bedevere-bot added the awaiting core review label May 12, 2023

bedevere-bot mentioned this pull request May 12, 2023

Add type annotations to clinic.py #104050

Closed

AlexWaygood added the skip news label May 12, 2023

AlexWaygood commented May 12, 2023

View reviewed changes

.github/workflows/mypy-clinic.yml Outdated Show resolved Hide resolved

improve workflow file

f7e7a25

JelleZijlstra reviewed May 12, 2023

View reviewed changes

Tools/clinic/clinic.py Outdated Show resolved Hide resolved

sobolevn reviewed May 12, 2023

View reviewed changes

hugovk reviewed May 12, 2023

View reviewed changes

.github/workflows/mypy-clinic.yml Outdated Show resolved Hide resolved

.github/workflows/mypy-clinic.yml Outdated Show resolved Hide resolved

Tools/clinic/clinic.py Show resolved Hide resolved

.github/workflows/mypy-clinic.yml Outdated Show resolved Hide resolved

AlexWaygood and others added 3 commits May 12, 2023 14:18

Add some more return annotations

510e08b

Co-authored-by: Nikita Sobolev <[email protected]>

simpler glob

773c2ba

Co-authored-by: Hugo van Kemenade <[email protected]>

Address various reviews!

6c7ecb9

AlexWaygood requested review from JelleZijlstra and hugovk May 12, 2023 13:37

hugovk reviewed May 12, 2023

View reviewed changes

.github/workflows/mypy.yml Outdated Show resolved Hide resolved

.github/workflows/mypy.yml Outdated Show resolved Hide resolved

Tools/clinic/requirements-dev.txt Outdated Show resolved Hide resolved

more from Hugo

717ee47

Co-authored-by: Hugo van Kemenade <[email protected]>

AlexWaygood commented May 12, 2023

View reviewed changes

Fix ini syntax

62669d1

AlexWaygood mentioned this pull request May 12, 2023

GH-86275: Implementation of hypothesis stubs for property-based tests, with zoneinfo tests #22863

Merged

hugovk approved these changes May 12, 2023

View reviewed changes

bedevere-bot removed the awaiting core review label May 12, 2023

bedevere-bot added the awaiting merge label May 12, 2023

AlexWaygood commented May 12, 2023

View reviewed changes

Tools/clinic/cpp.py Outdated Show resolved Hide resolved

Unused import

ddd0843

Merge branch 'main' into clinic-mypy-check

b7158db

JelleZijlstra reviewed May 13, 2023

View reviewed changes

Tools/clinic/mypy.ini Show resolved Hide resolved

AlexWaygood added 2 commits May 13, 2023 12:19

Restore compatibility with Python 3.8

9fcd87f

Revert "Restore compatibility with Python 3.8"

010f0dd

This reverts commit 9fcd87f.

erlend-aasland approved these changes May 15, 2023

View reviewed changes

AlexWaygood enabled auto-merge (squash) May 15, 2023 08:29

AlexWaygood merged commit 9d41f83 into python:main May 15, 2023

bedevere-bot removed the awaiting merge label May 15, 2023

sobolevn mentioned this pull request May 15, 2023

Run mypy on cases_generator #104504

Closed

AlexWaygood deleted the clinic-mypy-check branch May 15, 2023 10:22

erlend-aasland reviewed May 16, 2023

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

gh-104050: Run mypy on `clinic.py` in CI #104421

gh-104050: Run mypy on `clinic.py` in CI #104421

AlexWaygood commented May 12, 2023 •

edited

Loading

AlexWaygood May 12, 2023

AlexWaygood May 12, 2023

hugovk left a comment

AlexWaygood commented May 12, 2023 •

edited

Loading

erlend-aasland left a comment

AlexWaygood commented May 15, 2023

erlend-aasland May 16, 2023 •

edited

Loading

		@@ -0,0 +1,2 @@
		# Requirements file for external linters and checks we run on Tools/clinic/ in CI
		mypy==1.2.0

gh-104050: Run mypy on clinic.py in CI #104421

gh-104050: Run mypy on clinic.py in CI #104421

Conversation

AlexWaygood commented May 12, 2023 • edited Loading

AlexWaygood May 12, 2023

Choose a reason for hiding this comment

AlexWaygood May 12, 2023

Choose a reason for hiding this comment

hugovk left a comment

Choose a reason for hiding this comment

AlexWaygood commented May 12, 2023 • edited Loading

erlend-aasland left a comment

Choose a reason for hiding this comment

AlexWaygood commented May 15, 2023

erlend-aasland May 16, 2023 • edited Loading

Choose a reason for hiding this comment

gh-104050: Run mypy on `clinic.py` in CI #104421

gh-104050: Run mypy on `clinic.py` in CI #104421

AlexWaygood commented May 12, 2023 •

edited

Loading

AlexWaygood commented May 12, 2023 •

edited

Loading

erlend-aasland May 16, 2023 •

edited

Loading