[FEA] changelog generator #599

pxLi · 2020-08-21T08:35:20Z

example stdout:

.github/workflows/changelog/changelog --token=<GITHUB_PERSONAL_ACCESS_TOKEN>
# STDOUT
Generating changelog ...
Processing pull requests ...
Processing issues ...
Done.
Merged Pull Requests w/o Project:
branch-0.1 #181 Remove INCOMPAT for NormalizeNanAndZero, KnownFloatingPointNormalized https://github.com/NVIDIA/spark-rapids/pull/181
branch-0.1 #191 WindowExec handle different spark distributions https://github.com/NVIDIA/spark-rapids/pull/191
branch-0.1 #195 Support public release for plugin jar https://github.com/NVIDIA/spark-rapids/pull/195
branch-0.1 #206 Change groupID to 'com.nvidia' in IT scripts https://github.com/NVIDIA/spark-rapids/pull/206
branch-0.1 #208 Remove unneeded comment from pom.xml https://github.com/NVIDIA/spark-rapids/pull/208

pxLi · 2020-08-21T08:35:49Z

build

pxLi · 2020-08-21T08:51:25Z

build

tgravescs · 2020-08-21T13:08:37Z

.github/workflows/changelog/changelog

+
+e.g.
+    cd spark-rapids/
+    .github/workflows/changelog/changelog --token=<GITHUB_PERSONAL_ACCESS_TOKEN> --base_refs=branch-0.1,branch-0.2,branch-0.3


might be nice to add change_log is generated for each of the the branches specified.
I assume the assumption here is that we only do 1 release on each branch?

Yes we are making the assumption that we do one release per branch.

@tgravescs yes, plz take a look at the CHANGELOG.md, this is the rendered markdown preview. Since we separate prs&issues by the release info (github projects), so I try to map the pr&issue back to projects info. And plz let me know if anything else you want

my comment was saying we should update the description of this script to tell user what base_ref really does.

@tgravescs sure thing. I will add a detailed description in upcoming commits

.github/workflows/changelog/changelog

pxLi · 2020-08-25T06:59:02Z

build

pxLi · 2020-08-25T10:20:03Z

build

pxLi · 2020-08-26T01:27:10Z

build

pxLi · 2020-08-26T05:12:19Z

build

pxLi · 2020-08-26T05:20:16Z

so I added detailed desc in the script. Also there is nan auto-doc-gen workflow within this PR, so the workflow will be triggered each time a PR got merged and create a doc PR for review. If the workflow got triggered multiple times by multiple merged PR, all new commits will be append to the same opened doc PR. please let me know if anything else you want adjust thx! @sameerz @tgravescs @revans2

jlowe · 2020-08-26T13:04:47Z

the workflow will be triggered each time a PR got merged and create a doc PR for review

Please, let's not double the number of PRs. IMO we only need a changelog when we release, so we only need to run the changelog generator as part of the release process.

sameerz · 2020-08-26T21:54:37Z

Can we generate the CHANGELOG.md and auto-merge without doing a build? That way we could generate on some regular cadence (nightly?).

jlowe · 2020-08-26T22:16:54Z

Can we generate the CHANGELOG.md and auto-merge without doing a build? That way we could generate on some regular cadence (nightly?).

I am not a fan of this approach. The changelog results are highly sensitive to proper labeling and assigning to projects in github, and as the first cut showed, this is far from perfect. We need a tool to generate a changelog in preparation for a release, and I believe that changelog is going to need someone to review it for accuracy. Automerging the tool's output without review is asking for trouble like saying some issue is fixed in the release when it was instead closed as won't fix, etc.

pxLi · 2020-08-27T01:11:05Z

the workflow will be triggered each time a PR got merged and create a doc PR for review

Please, let's not double the number of PRs. IMO we only need a changelog when we release, so we only need to run the changelog generator as part of the release process.

@jlowe @sameerz actually unlike automerge, the changelog workflow won't merge the PR by itself, and it will keep just one opened doc-gen PR for review, new updates will be committed to the same PR. So team-reviews will be responsible to review the auto-gen doc and merge the PR.

cudf ask to submit PR with changelog update in their commit, but I am also agree with that we could generate the change log when we really do the release.

pxLi · 2020-08-28T02:08:57Z

build

pxLi · 2020-08-28T02:11:58Z

so I just pushed commits to disable the workflow by rename .github/workflows/{changelog.yml => changelog.yml.disabled}, we could easily turn this on if things changed in the future,
also I removed generated CHANGELOG.md file in this PR, so people could just use the tool to generate changelog when doing release.

jlowe

we could easily turn this on if things changed in the future

I personally would never want to double our commit load and clutter our commit history with a bunch of changelog-only PRs. If for some reason we need a changelog update on every PR merged, I'd rather see a hook that does the changelog update as part of the PR commit rather than a separate PR that's auto-merged. It's simply a lot less traffic and noise. In that scenario we're not doing a full changelog generation but a targeted changelog edit which is a very different workflow and script.

This looks OK to me other than I don't think we'll ever use the workflow setup, so I'm dubious on the utility of checking in dead code. Also it's a bit odd to "hide" the script we expect someone to manually run as part of building the release in the .github/workflows directory. I'd rather see this script under a top-level dev/ or scripts/ directory (or maybe even next to the documentation on how to release which should be under the docs/dev/ directory), but that's not a blocker for me. Curious what others think.

pxLi · 2020-08-31T01:16:26Z

build

pxLi · 2020-08-31T01:20:48Z

we could easily turn this on if things changed in the future

I personally would never want to double our commit load and clutter our commit history with a bunch of changelog-only PRs. If for some reason we need a changelog update on every PR merged, I'd rather see a hook that does the changelog update as part of the PR commit rather than a separate PR that's auto-merged. It's simply a lot less traffic and noise. In that scenario we're not doing a full changelog generation but a targeted changelog edit which is a very different workflow and script.

This looks OK to me other than I don't think we'll ever use the workflow setup, so I'm dubious on the utility of checking in dead code. Also it's a bit odd to "hide" the script we expect someone to manually run as part of building the release in the .github/workflows directory. I'd rather see this script under a top-level dev/ or scripts/ directory (or maybe even next to the documentation on how to release which should be under the docs/dev/ directory), but that's not a blocker for me. Curious what others think.

Ok, I removed the workflow-related code and move the script to the scripts/ folder.
Let's make it simple to keep it just as a script for people to use.

We could add a post-commit hook in the future if someone really need some automation 👍

jlowe

Can the name of this script be changed to something like generate-changelog? Otherwise it looks like it could be a changelog document for the scripts directory.

scripts/changelog

pxLi · 2020-08-31T15:57:01Z

build

scripts/generate-changelog

Signed-off-by: Peixin Li <[email protected]>

pxLi · 2020-09-01T00:22:48Z

build

pxLi · 2020-09-01T00:24:26Z

build

pxLi · 2020-09-01T00:42:22Z

build

* changelog generator Signed-off-by: Peixin Li <[email protected]>

[auto-merge] bot-auto-merge-branch-22.10 to branch-22.12 [skip ci] [bot]

tgravescs reviewed Aug 21, 2020

View reviewed changes

revans2 reviewed Aug 21, 2020

View reviewed changes

.github/workflows/changelog/changelog Outdated Show resolved Hide resolved

sameerz added the build Related to CI / CD or cleanly building label Aug 22, 2020

pxLi force-pushed the fea-573-changelog branch from 7cb34ff to c84ea42 Compare August 24, 2020 09:04

pxLi changed the title ~~[WIP] changelog generator~~ [FEA] changelog generator Aug 25, 2020

pxLi force-pushed the fea-573-changelog branch from 98e8758 to 2f4f3c4 Compare August 26, 2020 05:11

pxLi force-pushed the fea-573-changelog branch from 2f4f3c4 to 0733122 Compare August 28, 2020 02:06

jlowe previously approved these changes Aug 28, 2020

View reviewed changes

pxLi dismissed jlowe’s stale review via bd7cfe9 August 31, 2020 01:15

pxLi force-pushed the fea-573-changelog branch from ffc4ac3 to bd7cfe9 Compare August 31, 2020 01:15

jlowe reviewed Aug 31, 2020

View reviewed changes

scripts/changelog Outdated Show resolved Hide resolved

jlowe reviewed Aug 31, 2020

View reviewed changes

scripts/generate-changelog Outdated Show resolved Hide resolved

pxLi added 2 commits September 1, 2020 08:14

changelog generator

86272fb

Signed-off-by: Peixin Li <[email protected]>

only filter out labels for issue

f2c9198

pxLi added 10 commits September 1, 2020 08:14

add workflow on github actions

40c57d7

complete workflow and add desc to script

be321a6

switch pull_request to pull_request_target

b6daf6e

switch back to use default GITHUB_TOKEN

485ed9c

add github.actor as author and committer

83801a8

disable the workflow for now

a635bb7

clean generated changelog.md

95fcc59

modify the PR to tool only

f593d00

refactor to make this can be imported

2a6e828

pass token as arg

e0b5cc1

pxLi force-pushed the fea-573-changelog branch from d29c536 to e0b5cc1 Compare September 1, 2020 00:23

jlowe approved these changes Sep 1, 2020

View reviewed changes

pxLi merged commit 78f2519 into NVIDIA:branch-0.2 Sep 1, 2020

jlowe added the feature request New feature or request label Sep 1, 2020

nartal1 pushed a commit to nartal1/spark-rapids that referenced this pull request Jun 9, 2021

[FEA] changelog generator (NVIDIA#599)

7f1ee66

* changelog generator Signed-off-by: Peixin Li <[email protected]>

nartal1 pushed a commit to nartal1/spark-rapids that referenced this pull request Jun 9, 2021

[FEA] changelog generator (NVIDIA#599)

105fdc9

* changelog generator Signed-off-by: Peixin Li <[email protected]>

tgravescs pushed a commit to tgravescs/spark-rapids that referenced this pull request Nov 30, 2023

Merge pull request NVIDIA#599 from NVIDIA/bot-auto-merge-branch-22.10

41cce54

[auto-merge] bot-auto-merge-branch-22.10 to branch-22.12 [skip ci] [bot]

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[FEA] changelog generator #599

[FEA] changelog generator #599

pxLi commented Aug 21, 2020 •

edited

Loading

pxLi commented Aug 21, 2020

pxLi commented Aug 21, 2020

tgravescs Aug 21, 2020

sameerz Aug 22, 2020

pxLi Aug 23, 2020

tgravescs Aug 24, 2020

pxLi Aug 25, 2020

pxLi commented Aug 25, 2020

pxLi commented Aug 25, 2020

pxLi commented Aug 26, 2020

pxLi commented Aug 26, 2020

pxLi commented Aug 26, 2020

jlowe commented Aug 26, 2020

sameerz commented Aug 26, 2020

jlowe commented Aug 26, 2020

pxLi commented Aug 27, 2020

pxLi commented Aug 28, 2020

pxLi commented Aug 28, 2020

jlowe left a comment

pxLi commented Aug 31, 2020

pxLi commented Aug 31, 2020

jlowe left a comment

pxLi commented Aug 31, 2020

pxLi commented Sep 1, 2020

pxLi commented Sep 1, 2020

pxLi commented Sep 1, 2020

[FEA] changelog generator #599

[FEA] changelog generator #599

Conversation

pxLi commented Aug 21, 2020 • edited Loading

pxLi commented Aug 21, 2020

pxLi commented Aug 21, 2020

tgravescs Aug 21, 2020

Choose a reason for hiding this comment

sameerz Aug 22, 2020

Choose a reason for hiding this comment

pxLi Aug 23, 2020

Choose a reason for hiding this comment

tgravescs Aug 24, 2020

Choose a reason for hiding this comment

pxLi Aug 25, 2020

Choose a reason for hiding this comment

pxLi commented Aug 25, 2020

pxLi commented Aug 25, 2020

pxLi commented Aug 26, 2020

pxLi commented Aug 26, 2020

pxLi commented Aug 26, 2020

jlowe commented Aug 26, 2020

sameerz commented Aug 26, 2020

jlowe commented Aug 26, 2020

pxLi commented Aug 27, 2020

pxLi commented Aug 28, 2020

pxLi commented Aug 28, 2020

jlowe left a comment

Choose a reason for hiding this comment

pxLi commented Aug 31, 2020

pxLi commented Aug 31, 2020

jlowe left a comment

Choose a reason for hiding this comment

pxLi commented Aug 31, 2020

pxLi commented Sep 1, 2020

pxLi commented Sep 1, 2020

pxLi commented Sep 1, 2020

pxLi commented Aug 21, 2020 •

edited

Loading