Refactor Argument Clinic support code into libclinic

[vstinner]

Argument Clinic was added 10 years ago. The script clinic.py grown from around 2500 lines to around 6700 lines.

I propose to split this large file into sub-files to ease the maintenance of Argument Clinic.

Linked PRs

• [DRAFT] gh-113299: Split clinic.py into sub-files #113300
• gh-113299: Create libclinic package #113309
• gh-113299: Move cpp.py into libclinic #113526
• gh-113299: Move Argument Clinic CLI into libclinic #115542

[erlend-aasland]

This has also been discussed before. See #104683 (comment).

[erlend-aasland]

I think it is a really good idea to split clinic.py into multiple files. The C preprocessor stuff is already separated out (cpp.py). We should do this in iterations; not everything in one big PR. Perhaps it is easiest to start with the converters; it's a large chunk of clinic.py, and I think it would be pretty easy to separate it from the rest. Separating out the application is also a good start.

Moreover, doing this work in chunks/iterations, instead of everything in one go, will minimise the impact on ongoing Argument Clinic work.

[AlexWaygood]

+1 to splitting clinic.py up, and +1 to doing it incrementally rather than in one big PR.

Previously I've held off from doing something like this because clinic.py's use of global variables means that lots of parts of the file are tightly coupled to other parts in surprising and unfortunate ways. In #113300, warn() and fail() still use a global clinic variable in utils.py, but that variable is now altered from code in other files, making it even harder to understand what's going on. That feels worse, and it feels like we're avoiding solving the actual problem here, which is that clinic.py just shouldn't really be using global variables like this in the first place.

But maybe it's acceptable for now, if splitting the code up into several files is a higher priority :) Getting rid of the global variables is hard!

[erlend-aasland]

IMO, getting rid of the global variables should be a prerequisite to splitting things up. We don't need to rush this; clinic.py has been a single file since it started out in life. Argument Clinic is already hard to comprehend, so any refactoring, including splitting up should make it more readable, not the other way around. Readability is IMO the highest priority.

[erlend-aasland]

Since last week, we've done a lot of yak-shaving (and there is more to be done). A bare bones libclinic has been established, containing most of the stateless formatting code. The next steps will be to move ClinicError into libclinic/errors.py, and rename cpp.py to libclinic/cpp.py.

[erlend-aasland]

libclinic/errors.py was established via #113525.

[erlend-aasland]

Superseded by #113317