Add type annotations to clinic.py

[erlend-aasland]

Feature or enhancement

Gradually add type hints to clinic.py.

Pitch

Adding type hints to clinic.py will make it easier to debug, fix bugs, do code review, add new features, and triage. IMO, the churn is worth the effort in the long run.

Previous discussion

Since adding typing to the CPython code base has been a slightly controversial topic (obviously controversial for Lib/), I created a topic on Discourse and asked for feedback. So far, I have not received any discouragements :)

Linked PRs

• gh-104050: Run mypy on clinic.py in CI #104421
• gh-104050: Add basic typing to CConverter in clinic.py #104538
• gh-104050: Don't star-import 'types' in Argument Clinic #104543
• gh-104050: Add more annotations to Tools/clinic.py #104544
• gh-104050: Add typing to Argument Clinic converters #104547
• gh-104050: Add type hints to Argument Clinic converter keywords #104588
• gh-104050: Add type annotations to sentinels in Argument Clinic #104589
• gh-104050: Improve some typing around defaults and sentinel values #104626
• gh-104050: Add more type annotations to Argument Clinic #104628
• gh-104050: Add more type annotations to Argument Clinic #104631
• gh-104050: Argument clinic: Add annotations to the LandMine class #104648
• gh-104050: Add basic type hints to Argument Clinic DSL parser #104704
• gh-104050: Add basic type hints to Argument Clinic clinic class #104705
• gh-104050: Annotate Argument Clinic return converters #104706
• gh-104050: Add more type hints to Argument Clinic DSLParser() #106343
• gh-104050: Add more type hints to Argument Clinic DSLParser() #106354
• gh-104050: Annotate Argument Clinic DSLParser attributes #106357
• gh-104050: Annotate more Argument Clinic DSLParser state methods #106376
• gh-104050: Annotate Argument Clinic parameter permutation helpers #106431
• gh-104050: Annotate toplevel functions in clinic.py #106435
• gh-104050: Partially annotate Argument Clinic CLanguage class #106437
• gh-104050: Argument Clinic: Annotate the Block class #106519
• gh-104050: Argument clinic: Annotate the Destination class #106655
• gh-104050: Argument Clinic: Annotate nested function parser_body() in the CLanguage class #106699
• gh-104050: Argument Clinic: Annotate output_templates() #106732
• gh-104050: Argument Clinic: Annotate BlockParser #106750
• gh-104050: Argument Clinic: Annotate Clinic.parse() #106760
• gh-104050: Improve Argument Clinic type annotation coverage #106810
• gh-104050: Argument Clinic: Annotate CLanguage.render_option_group_parsing() #106929
• gh-104050: Argument Clinic: Increase CConverter typing coverage #106932
• gh-104050: Argument Clinic: Annotate the IndentStack class #106934
• gh-104050: Argument Clinic: Annotate the BufferSeries class #106935
• gh-104050: Argument Clinic: Increase typing coverage #107074
• gh-104050: Argument Clinic: annotate main() #107192
• gh-104050: Argument clinic: annotate format_docstring() #107200
• gh-104050: Argument clinic: misc improvements to type annotation coverage #107206
• gh-104050: Argument Clinic: Annotate CRenderData.__init__ #107207
• gh-104050: Argument clinic: improve typing around adding C converters #107209
• gh-104050: Argument clinic: more misc typing coverage improvements #107210
• gh-104050: Argument clinic: annotate post_parsing() and cleanup() #107225
• gh-104050: Argument clinic: more misc typing improvements #107264
• gh-104050: Argument clinic: improve typing around parse_arg() methods #107288
• gh-104050: Argument clinic: Complete get_destination_buffer annotations #107293
• gh-104050: Argument clinic: Annotate str_converter_key() #107294
• gh-104050: Argument clinic: complete type annotations #107399
• gh-104050: Argument clinic: enable mypy's --warn-return-any setting #107405

[erlend-aasland]

Since I've been ignorant to typing until now, I'd like to give this a shot myself; it'll be a nice opportunity to learn some new tricks.

EDIT:
Don't take this too serious; I don't want to block anybody from contributing. Please fire up PRs if you want (I appreciate if you tag me, though!)

[sobolevn]

@erlend-aasland feel free to ask for help/reviews :)

[hauntsaninja]

A good place to start is by switching the instance fields in CConverter to use variable annotations. This is the cause of most of the existing mypy complaints.

As you add types, the type checker will become more useful. Once you've gotten a number of types in, I recommend upgrading to mypy --check-untyped-defs, and then possibly mypy --strict.

I'm biased, but I think mypy docs are a generally useful reference: https://mypy.readthedocs.io/en/latest/cheat_sheet_py3.html

[erlend-aasland]

As recommended by @AlexWaygood, we should probably start by adding a minimal type check to the CI. (We really don't want to increase CI minutes too much.) In the beginning, perhaps an allow-list is the most manageable path forward. When that list starts to grow, we can consider switching to using a deny-list instead. Thoughts? PRs welcome :)

[AlexWaygood]

Here's a PR to add a mypy check in CI, checking only the Tools/clinic directory: #104421. The check completed in 20 seconds (and will only run on PRs that touch files in Tools/clinic/), so I doubt it will increase our use of CI minutes too much!

[erlend-aasland]

We should perhaps consider using protocols (for example, converters comes to mind).

[AlexWaygood]

44 merged PRs later, and clinic is now 100% fully typed! 🚀

[erlend-aasland]

A big thanks to everyone involved! 🎉❤️