Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🌱 Make release notes generation concurrent #8817

Merged

Conversation

g-gaston
Copy link
Contributor

@g-gaston g-gaston commented Jun 7, 2023

What this PR does / why we need it:
Now that we retrieve the area labels from the PRs, the release notes generation can take up to 2 minutes for big releases like 1.5. Although in general this is not a problem since it should be a one time process, it does slow down the comms release team when iterating in the notes, adding missing labels, etc. It also makes development of the script painful when testing it manually.

The default is 10 concurrent routines, but this can be changed with command flag.

The default value of 10 doesn't seem to run into rate limiting problems. I had to bump it to a 100 and run the command multiple times in a row to be able to get rate limited.

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Jun 7, 2023
@k8s-ci-robot k8s-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Jun 7, 2023
@g-gaston
Copy link
Contributor Author

g-gaston commented Jun 7, 2023

/hold
Builds on top of #8780, so waiting for that one to be merged first

@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 7, 2023
@g-gaston g-gaston changed the title Make release notes generation concurrent 🌱 Make release notes generation concurrent Jun 7, 2023
@g-gaston
Copy link
Contributor Author

g-gaston commented Jun 7, 2023

/area release

@k8s-ci-robot k8s-ci-robot added the area/release Issues or PRs related to releasing label Jun 7, 2023
@g-gaston g-gaston force-pushed the release-notes-concurrency branch from a51a1e4 to 1892cfd Compare June 8, 2023 13:54
@g-gaston
Copy link
Contributor Author

g-gaston commented Jun 8, 2023

rebased after #8780 merged
/unhold

@k8s-ci-robot k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 8, 2023
@johannesfrey
Copy link
Contributor

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 12, 2023
@k8s-ci-robot
Copy link
Contributor

LGTM label has been added.

Git tree hash: b4d2f25aa8e74c88f890791ead315886544ee4be

@stmcginnis
Copy link
Contributor

/lgtm

Copy link
Member

@nawazkh nawazkh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an amazing update @g-gaston ! Thanks 🚀
Just a minor nit, can we add any more comments around results, jobs and go routines for helping anyone reading this the first time?

Copy link
Member

@fabriziopandini fabriziopandini left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working at this!

hack/tools/release/notes.go Outdated Show resolved Hide resolved
hack/tools/release/notes.go Outdated Show resolved Hide resolved
hack/tools/release/notes.go Outdated Show resolved Hide resolved
hack/tools/release/notes.go Outdated Show resolved Hide resolved
hack/tools/release/notes.go Outdated Show resolved Hide resolved
@k8s-ci-robot k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 13, 2023
@g-gaston g-gaston requested a review from fabriziopandini June 13, 2023 19:25
@fabriziopandini
Copy link
Member

/lgtm
/retest

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 14, 2023
@k8s-ci-robot
Copy link
Contributor

LGTM label has been added.

Git tree hash: a03a782499a6ce4cc394c9d53a9397dabe45dd64

Copy link
Contributor

@killianmuldoon killianmuldoon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for taking this on - nice speedup 🙂

/lgmt
/approve

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 14, 2023
@killianmuldoon
Copy link
Contributor

/check-cla

@killianmuldoon
Copy link
Contributor

@g-gaston You might want to force-push on top of this to see if that kicks easyCLA into action.

@g-gaston g-gaston force-pushed the release-notes-concurrency branch from 24cb6b7 to fd00dfd Compare June 14, 2023 13:06
@k8s-ci-robot k8s-ci-robot added area/clusterctl Issues or PRs related to clusterctl needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jun 14, 2023
Now that we retrieve the area labels from the PRs, the release notes
generation can take up to 2 minutes for big releases like 1.5. Although
in general this is not a problem since it should be a one time process,
it does slow down the comms release team when iterating in the notes,
adding missing labels, etc. It also makes development of the script
painful when testing it manually.

The default is 10 concurrent routines, but this can be changed with
command flag.

The default value of 10 doesn't seem to run into rate limiting problems.
I had to bump it to a 100 and run the command multiple times in a row to
be able to get rate limited.
@g-gaston g-gaston force-pushed the release-notes-concurrency branch from fd00dfd to c472b28 Compare June 14, 2023 13:06
@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed lgtm "Looks good to me", indicates that a PR is ready to be merged. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Jun 14, 2023
@g-gaston
Copy link
Contributor Author

@g-gaston You might want to force-push on top of this to see if that kicks easyCLA into action.

@killianmuldoon squashed and rebased :)

@fabriziopandini
Copy link
Member

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 14, 2023
@k8s-ci-robot
Copy link
Contributor

LGTM label has been added.

Git tree hash: 3b885f0fa93934f926aa7a011e29f4770788c666

@killianmuldoon
Copy link
Contributor

Thanks

/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: killianmuldoon

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot merged commit 677a182 into kubernetes-sigs:main Jun 14, 2023
@k8s-ci-robot k8s-ci-robot added this to the v1.5 milestone Jun 14, 2023
@killianmuldoon killianmuldoon removed the area/clusterctl Issues or PRs related to clusterctl label Jun 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/release Issues or PRs related to releasing cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants