✨ Add dependencies section to release notes tool #10043

willie-yao · 2024-01-23T19:07:52Z

What this PR does / why we need it:
This PR adds a dependencies section to the release notes generator. It uses the kubernetes release notes tool: https://github.com/kubernetes/release/blob/master/cmd/release-notes/README.md

Example of what is printed:

## Dependencies

### Added
- cloud.google.com/go/dataproc/v2: v2.2.3
- github.com/ProtonMail/go-crypto: [7d5c6f0](https://github.com/ProtonMail/go-crypto/tree/7d5c6f0)

### Changed
- cloud.google.com/go/accessapproval: v1.6.0 → v1.7.4
- cloud.google.com/go/accesscontextmanager: v1.7.0 → v1.8.4

This block is added aft er the Others section of the release notes.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #9854

/area release

fabriziopandini · 2024-01-24T10:35:22Z

@willie-yao nice!
for sake of documentation it will be great if you can add a preview of how release notes will look like after this change

cahillsf · 2024-01-24T21:36:10Z

/lgtm 🔥

sbueringer

Very nice. Thank you this will be very useful!

In turn we should probably get rid of the "bump" entries for regular Go dependencies in our release notes

(at least as a manual step)

hack/tools/release/notes/print.go

sbueringer · 2024-01-26T07:37:29Z

hack/tools/release/notes/process.go

@@ -132,8 +132,13 @@ func (g prEntriesProcessor) generateNoteEntry(p *pr) *notesEntry {
 		// Release trigger PRs from previous releases are not included in the release notes
 		return nil
 	case strings.HasPrefix(entry.title, ":seedling:"), strings.HasPrefix(entry.title, "🌱"):
+		title := removePrefixes(entry.title, []string{":seedling:", "🌱"})
+		// Skip PRs bump dependencies as they will be listed separately.
+		if area == "Dependency" {


We are flagging a lot more than just dependabot go dependency bumps with area/dependency (e.g. Go, Trivy, ... bumps, but I think even more).

I think dropping all area/dependency PRs is too much, we might lose something important there.

Can we drop all PRs from dependabot instead (filter on author:app/dependabot )? (we would additionally drop GitHub actions bumps, but I think that is fine)

(also works better with other repositories that don't set the label but are also using dependabot like CAPV and controller-runtime)

I'm trying to filter the GitHub API response for the author but I'm a bit confused on how the code works here. How do we go from returning the collection here to the prs []githubPR object here? I tried adding an author field to the githubPR type, but I'm a bit confused where the existing fields (Number, Title, Labels) are populated. pageRead just seems to give me the entire PR page. @g-gaston do you know if there's a way to add an author field to the current flow?

Sorry I don't fully understand the question. If the search api returns an author field for pr objects (I'm not sure if it does, you would need to check the docs or just run a query yourself and check the output), then adding it to the githubPR struct will make that available when calling listMergedPRs.

If you want to make it available to the processor (and the printer), you'll need to add the field to the pr struct as well: https://github.com/kubernetes-sigs/cluster-api/blob/main/hack/tools/release/notes/list.go#L117-L121

What I would do here is add the author information to the pr domain object so you have it available in the processor (this is the one responsible from processing each pr and deciding which kind of note entry it should be). In your case it seems you want the processor to discard PRs based on the author, so they should no be converted to notes entries. Beware this will affect the total number of commits printed at the beginning of the notes. If you don't want this to happen, then maybe slightly changing the responsibilities of processor and printer is a better idea.

Can we drop all PRs from dependabot instead (filter on author:app/dependabot )?

There still seems to be a good amount of dependencies bump PRs that are not created by dependabot. Should we just leave those as is?

I think it's better if we sort through manual bump PRs manually compared to dropping all dependency PRs from the release notes.

We even might just leave them in there. Usually we only create manual bumps if either dependabot can't bump something or if we have to bump manually because we have to change more.

Maybe let's just go with dropping PRs from author dependabot and leaving everything else in the release notes.

sbueringer · 2024-01-26T07:39:16Z

hack/tools/release/notes/print.go

+		repoURL, p.fromTag, p.toTag,
+	)
+	if err != nil {
+		fmt.Printf("Error getting dependencies: %v", err)


I would add a new error return parameter and propagate it up (with a bit of error wrapping).

Then we can fail properly

g-gaston

Could you run the integration tests and make sure you update the golden files to match the new output?

hack/tools/go.mod

g-gaston · 2024-01-26T20:38:15Z

hack/tools/release/notes/print.go

@@ -24,6 +24,8 @@ import (
 	"sort"
 	"strings"

+	"k8s.io/release/pkg/notes"


curious why didn't get added to the go.mod?

hack/tools/release/notes/generator.go

hack/tools/release/notes/print.go

willie-yao · 2024-01-27T00:07:18Z

In regards to the broken links: I think there is an issue with the kubernetes dependency tool right now that messes up the links when a tag includes a "/". We faced this issue when creating release notes for the latest CAPZ release. We might need to hold this PR until that issue is fixed.

sbueringer · 2024-01-29T10:02:31Z

@willie-yao Let's please try to figure out if the markdownlinkchecker can skip this test folder (see .markdownlinkcheck.json)

hack/tools/release/notes/process.go

g-gaston · 2024-01-29T16:22:11Z

hack/tools/release/notes/test/golden/v1.5.0.md

+- Dependency: Bump cert-manager to v1.13.0 (#9413)
+- Dependency: Bump cert-manager to v1.13.1 (#9507)
+- Dependency: Bump cert-manager to v1.13.2 (#9658)
+- Dependency: Bump controller runtime to v1.15.3 (#9624)
+- Dependency: Bump controller-runtime to v0.15.1 (#9127)
+- Dependency: Bump corefile-migration library to v1.0.21 (#9309)


I curious to see why there are new PRs appearing here.
I was expecting to only see two types of diff: PRs being omitted (authored by dependabot) in this section and the new deps section

Looks like the tool is listing merged PRs up to the date of the latest commit on the release branch. It looks like there are PRs still being cherry-picked into v1.5.0 which is why these PRs show up. Is this something we should worry about? In a normal release cycle, we'll be generating the release notes at the correct day, but if we retroactively generate them, we might be getting some new commits that aren't part of the release.

I'm thinking maybe we can add a flag for a specific end date, or we can just manually remove the newer PRs since I don't think we'd be retroactively generating release notes in the normal release flow.

Umm I see
I suspect there is bug in how the tests are setup then
Because they are supposed to use "static" config so the output doesn't change even if we merge more commits in the original branch.
We might be missing a to ref or the ref might be incorrect

I used the command RELEASE_TAG=v1.5.0 make release-notes to generate the golden files so I don't think it's just a bug in the test. I think it looks for the latest commit in the branch rather than when the tag is created: https://github.com/willie-yao/cluster-api/blob/release-notes-dependencies/hack/tools/release/notes/list.go#L65

I see that it is being defaulted to heads/branch instead of tags/tagname here. Should we make tags/ the default if RELEASE_TAG is specified? I don't see why a user would need to read from heads/branch if they are generating release notes for a specific release tag.

hack/tools/release/notes/print.go

g-gaston

This is great. Thanks a lot for pushing this all the way @willie-yao, I know it took a lot with those bugs in the deps module

/lgtm

k8s-ci-robot · 2024-02-28T16:23:57Z

LGTM label has been added.

Git tree hash: 58d5d9864aba007d8578ee387010d23df78b7cce

willie-yao · 2024-02-28T17:25:37Z

Squashed and rebased!

g-gaston · 2024-02-28T18:38:08Z

/lgtm
not sure if @willie-yao wants someone elses review
but i think @cahillsf, you should be able to approve?

k8s-ci-robot · 2024-02-28T18:38:13Z

LGTM label has been added.

Git tree hash: 75bab9edff1b64d19cfbed1e2cc7bcade414f10c

cahillsf · 2024-02-29T23:41:56Z

dont think i can approve since there are changes in hack/tools (not under release dir)

/lgtm

cahillsf · 2024-03-06T18:48:32Z

/lgtm

k8s-ci-robot · 2024-03-06T18:48:37Z

LGTM label has been added.

Git tree hash: c06d4bf2d84c9795f9801f78f38371705598ddcc

fabriziopandini · 2024-03-06T20:20:32Z

/lgtm
/approve

/hold cancel

k8s-ci-robot · 2024-03-06T20:21:07Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: fabriziopandini

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [fabriziopandini]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot requested review from cahillsf and richardcase January 23, 2024 19:08

willie-yao force-pushed the release-notes-dependencies branch from 36e3588 to 8509142 Compare January 23, 2024 19:12

k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 23, 2024

willie-yao force-pushed the release-notes-dependencies branch 4 times, most recently from e444bab to 326a0c3 Compare January 23, 2024 21:44

sbueringer reviewed Jan 25, 2024

View reviewed changes

hack/tools/release/notes/print.go Outdated Show resolved Hide resolved

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 25, 2024

willie-yao force-pushed the release-notes-dependencies branch 2 times, most recently from 324e4b5 to 525d254 Compare January 25, 2024 23:16

k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 25, 2024

sbueringer reviewed Jan 26, 2024

View reviewed changes

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 26, 2024

willie-yao force-pushed the release-notes-dependencies branch from 525d254 to 433585b Compare January 26, 2024 18:58

g-gaston reviewed Jan 26, 2024

View reviewed changes

k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 26, 2024

willie-yao force-pushed the release-notes-dependencies branch from eae6f46 to 179a08f Compare January 26, 2024 23:33

k8s-ci-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jan 26, 2024

g-gaston reviewed Jan 29, 2024

View reviewed changes

willie-yao force-pushed the release-notes-dependencies branch from 928f44f to 2c71c7d Compare February 27, 2024 23:19

g-gaston reviewed Feb 28, 2024

View reviewed changes

k8s-ci-robot assigned g-gaston Feb 28, 2024

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Feb 28, 2024

willie-yao force-pushed the release-notes-dependencies branch from 2c71c7d to 7c388ad Compare February 28, 2024 17:25

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Feb 28, 2024

k8s-ci-robot requested a review from g-gaston February 28, 2024 17:25

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Feb 28, 2024

k8s-ci-robot assigned cahillsf Feb 29, 2024

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 5, 2024

Print dependencies in release notes

3777030

willie-yao force-pushed the release-notes-dependencies branch from 7c388ad to 3777030 Compare March 6, 2024 18:46

k8s-ci-robot removed lgtm "Looks good to me", indicates that a PR is ready to be merged. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Mar 6, 2024

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Mar 6, 2024

k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Mar 6, 2024

k8s-ci-robot assigned fabriziopandini Mar 6, 2024

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 6, 2024

k8s-ci-robot merged commit 6b87089 into kubernetes-sigs:main Mar 6, 2024
21 checks passed

k8s-ci-robot added this to the v1.7 milestone Mar 6, 2024

willie-yao mentioned this pull request Apr 16, 2024

🚀 Release v1.7.0 #10440

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

✨ Add dependencies section to release notes tool #10043

✨ Add dependencies section to release notes tool #10043

willie-yao commented Jan 23, 2024 •

edited

Loading

fabriziopandini commented Jan 24, 2024

cahillsf commented Jan 24, 2024

sbueringer left a comment •

edited

Loading

sbueringer Jan 26, 2024 •

edited

Loading

sbueringer Jan 26, 2024

willie-yao Jan 26, 2024

g-gaston Jan 26, 2024

willie-yao Jan 26, 2024

sbueringer Jan 29, 2024

sbueringer Jan 26, 2024

g-gaston left a comment

g-gaston Jan 26, 2024

willie-yao commented Jan 27, 2024

sbueringer commented Jan 29, 2024 •

edited

Loading

g-gaston Jan 29, 2024

willie-yao Jan 30, 2024

g-gaston Jan 30, 2024

willie-yao Jan 31, 2024

willie-yao Jan 31, 2024

g-gaston left a comment

k8s-ci-robot commented Feb 28, 2024

willie-yao commented Feb 28, 2024

g-gaston commented Feb 28, 2024

k8s-ci-robot commented Feb 28, 2024

cahillsf commented Feb 29, 2024

cahillsf commented Mar 6, 2024

k8s-ci-robot commented Mar 6, 2024

fabriziopandini commented Mar 6, 2024

k8s-ci-robot commented Mar 6, 2024

✨ Add dependencies section to release notes tool #10043

✨ Add dependencies section to release notes tool #10043

Conversation

willie-yao commented Jan 23, 2024 • edited Loading

fabriziopandini commented Jan 24, 2024

cahillsf commented Jan 24, 2024

sbueringer left a comment • edited Loading

Choose a reason for hiding this comment

sbueringer Jan 26, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

g-gaston left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

willie-yao commented Jan 27, 2024

sbueringer commented Jan 29, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

g-gaston left a comment

Choose a reason for hiding this comment

k8s-ci-robot commented Feb 28, 2024

willie-yao commented Feb 28, 2024

g-gaston commented Feb 28, 2024

k8s-ci-robot commented Feb 28, 2024

cahillsf commented Feb 29, 2024

cahillsf commented Mar 6, 2024

k8s-ci-robot commented Mar 6, 2024

fabriziopandini commented Mar 6, 2024

k8s-ci-robot commented Mar 6, 2024

willie-yao commented Jan 23, 2024 •

edited

Loading

sbueringer left a comment •

edited

Loading

sbueringer Jan 26, 2024 •

edited

Loading

sbueringer commented Jan 29, 2024 •

edited

Loading