Argument Clinic: split out global stateless helpers and constants from clinic.py

[erlend-aasland]

Feature or enhancement

clinic.py has a lot of globals in various forms. Getting rid of the globals will make it possible to split out clinic.py in multiple files (#113299). Combined, this will impact readability and maintainability of Argument Clinic in a positive way.

Most globals can be easily dealt with. We can structure them into two groups:

• global constants that are not mutated
• stateless helper functions and classes

It can make sense split out some of these in separate files (for example a libclinic package, as suggested in #113309). For some globals, it makes more sense to embed them into one of the existing classes (for example the global version variable clearly belongs in DSLParser).

Suggesting to start with the following:

• move stateless text accumulator helpers into a Tools/clinic/libclinic/accumulators.py file
• move stateless text formatting helpers into a Tools/clinic/libclinic/formatters.py file
• move version into DSLParser and version helper into a Tools/clinic/libclinic/version.py file

Linked PRs

• gh-113317: Argument Clinic: tear out internal text accumulator APIs #113402
• gh-113317: Remove TextAccumulator type alias from clinic.py #113413
• gh-113317: Clean up Argument Clinic global namespace #113414
• gh-113317: Move more formatting helpers into libclinic #113438
• gh-113317: Rework Argument Clinic cpp.py error handling #113525
• gh-113317: Move global utility functions into libclinic #113986
• gh-113317: Move FormatCounterFormatter into libclinic #114066
• gh-113317: Pass clinic to AC parse_arg() methods #114319
• gh-113317: AC converter: Use add_include() in bad_argument() #114324
• gh-113317: Don't use global clinic instance in bad_argument() #114330
• gh-113317: Remove Argument Clinic global variable 'clinic' #114752
• gh-113317, Argument Clinic: Add BlockParser.fail() method #115369
• gh-113317, Argument Clinic: Add DSLParser.fail() method #115370
• gh-113317, Argument Clinic: Add Clinic.fail() method #115371
• gh-113317: Argument Clinic: don't use global state in warn() and fail() #115510
• gh-113317: Argument Clinic: don't use fail() in CLI #115513
• gh-113317: Argument Clinic: remove global clinic instance #115517
• gh-113317: Argument Clinic: move linear_format into libclinic #115518
• gh-113317: Argument Clinic: move C/Py identifier helpers into libclinic #115520
• gh-113317: Argument Clinic: inline required_type_for_self_for_parser() in self converter #115522
• gh-113317: Move warn() and fail() to libclinic.utils #116770
• gh-113317: Argument Clinic: Add libclinic.function #116807
• gh-113317: Add libclinic.block_parser and libclinic.language modules #116819
• gh-113317: Add libclinic.converter and libclinic.crenderdata modules #116821
• gh-113317: Move Argument Clinic converters to sub-modules #116836
• gh-113317: Change how Argument Clinic lists converters #116853
• gh-113317: Argument Clinic: Add libclinic.converters module #117315
• gh-113317: Argument Clinic: Add libclinic.return_converters #117451
• gh-113317: Argument Clinic: Add libclinic.clanguage #117455
• gh-113317: Finish splitting Argument Clinic into sub-files #117513
• gh-113317: Rename libclinic to argclinic #117533
• gh-113317: Remove unused INVALID constant in Argument Clinic #117624
• gh-113317: Add Codegen class to Argument Clinic #117626
• gh-113317: Add ParseArgsCodeGen class #117707

[erlend-aasland]

Proof-of-concept branch for version: Argument-Clinic#28

[erlend-aasland]

In #113270 (comment), the need for a refactoring of the docstring generation code arose. If we split out formatting helpers and text accumulators into a libclinic, we can also split out the docstring rendering (format_docstring()).

[erlend-aasland]

Follow-up item: when libclinic has been established, we can start partitioning the test suite (Lib/test/test_clinic.py) as well.

[erlend-aasland]

FTR, the global version variable was removed with PR #113341.

[erlend-aasland]

Moving the text accumulator APIs (TextAccumulator, _TextAccumulator, text_accumulator, and _text_accumulator) into a libclinic would imply exposing underscore prefixed methods in __all__; that feels weird. Suggested first steps:

• Clean up the accumulator APIs: gh-113317: Argument Clinic: tear out internal text accumulator APIs #113402
• Move (remaining) accumulators into libclinic
• Move stateless formatting helpers into libclinic
• Move constants into libclinic

I did some experimenting in Argument-Clinic#30.

[erlend-aasland]

We should put some thought into how we structure the rest of libclinic. Victor hashed out a proof-of-concept libclinic in #113309 #113300. We can use that PR as inspiration. We might want to do some more yak-shaving before we make such a grand transition:

• rework the error handling so warn and fail no longer depend on global variables
• refactor global configuration to be part of the application (converters, extensions/languages, destinations?, etc.), instead of hard-coded in the various classes (and as globals)

[vstinner]

@erlend-aasland: Can we close this issue now?

[erlend-aasland]

Definitely!