Better release notes generation and automated release process

[Pierre-Sassoulas]

Current problem

• We have to cherry-pick manually [release process] Add a github actions to backport fix automatically #7776
• contributor-txt pollutes the contributor file with a header because it does not detect the existing header fixed in Use relative paths in create_contributor_list.py #7656
• The contributor needs to be sorted manually because contributor-txt does not handle it
• The release notes are generated as rst but github want markdown
• The new release note file is not included in any toctree
• Only the first link is generated when there is two or more issues referenced in a fragment
• if false-positive is used instead of false_positive the changelog check is green but the result is not automatically added to the release note see Fix 2.15 changelog #7369 [changelog] Create separate entries for reimported/shadowed-import #7903
• The release note generated from patch create a duplicated title Release branch 2.15.1 #7393 fixed in Fix duplicate section erorr in sphinx for release notes of patch version #7589
• The release date is not automatically added in the documentation (0fba80c)
• Consider automating the deduplication of changelog fragments during patch releases (ref Consider automating the deduplication of changelog fragments during patch releases #9301)

Desired solution

Markdown creation so it can be used in both context, other issue automatically fixed

Additional context

Noticed during the release of 2.15.0.

[DudeNr33]

Markdown is not really directly supported by towncrier.
But people have found hacks for a workaround.
Alternatively, we could use pandoc or a similar tool to convert it.

For the link generation: This will also require some scripting. Currently we use the issue_format option:
https://github.com/PyCQA/pylint/blob/24eaec26f701cf8adddf27ffa8859e0bb5074a89/towncrier.toml#L6
The {issue} placeholder is simply the first part of the message fragment.
Maybe some pre-commit hook to convert simple #1234 into actual GitHub links would be better?

[Pierre-Sassoulas]

For github we could do with only "#{issue}". I was thinking of using .md in order to not have to generate another document but this is done very unfrequentely so launching a script to generate github's text is reasonable.

[DudeNr33]

Sorry for the delay. I looked into the issue with the duplicated labels in the release notes.

The problem comes from the sphinx.ext.autosectionlabel extension. Comparing our generated releasenotes with other projects like pytest and twisted that use towncrier, it is normal to have duplicated titles in the resulting document.

The options we have to resolve this are:

• Add autosectionlabel_maxdepth = 2 to doc/conf.py, which limits the autogeneration of labels to the first two levels (see docs)

OR

• Use single_file=false in towncrier.toml, which would generate separate files for each release (see docs).

If the first option is OK for you, I would prefer that one.

[Pierre-Sassoulas]

The first option work, I tested that on the release branch for 2.15.4 (on one cherry picked commit only, but I don't thik more test is required). Merge request is #7589

[Pierre-Sassoulas]

The new version of contributor-txt handle sorting reasonably well while keeping comments intact.

[Pierre-Sassoulas]

I'm keeping the issue opened but I unassigned myself because with the frequence of release we have we can live with the current shortcoming. It would be different if we want to continuously release fixes or features.