Set up automation to comment on PRs w/ links to downstream jobs #21795

nickboldt · 2022-10-31T14:56:38Z

Is your enhancement related to a problem? Please describe

Purpose here is to automate the flow of information so that when a PR merge / commit happens upstream, that SHA / branch can be more easily associated w/ a downstream DS 3.x build.

Repos affected include:

che-incubator/*
eclipse-che/*
redhat-developer/devspaces (registries)
redhat-developer/devspaces-theia

For these repos, we will need a secondary webhook setup that can watch for config changes but NOT changes already synced from upstream, eg., a config change like redhat-developer/devspaces-images#353

redhat-developer/devspaces-images
redhat-developer/devspaces-chectl

Describe the solution you'd like

Downstream Jenkins job could...

check out upstream code
find pull request #
use GH api to comment on the PR with links to the job URL (console + changes); see also https://kb.novaordis.com/index.php/Jenkins_currentBuild
when the job triggers its downstream process, could then pass the upstream PR URL to the next job, and add more comments as each build completes.
you would then end up with 2-4 comments on the PR for each build, ending with a link to the quay image (and possibly the IIBs associated w/ the build too)

Describe alternatives you've considered

GHA that watches for changes in quay?

Additional context

Discussed during this current sprint's retrospective last week

nickboldt · 2022-11-16T04:05:12Z

Initial prototype: jenkinsfile to check out a project from a branch, switch to a tag if applicable, and then using that SHA, comment on any applicable pull request.

https://gist.github.com/nickboldt/efb8d1b3a60c079e46b5b7aab4412e22

Tested with: SHA = 58d8b4407ab5d2b9a696236202308d92d3e25340
and comment: redhat-developer/devspaces#848 (comment)

TODO:

switch to GH API instead of GH CLI
wrap code in a groovy method and add to util2.groovy
comment should include:
- version of DS being built (if 3.x, useful to parse https://github.com/redhat-developer/devspaces/blob/devspaces-3-rhel-8/dependencies/job-config.json#L3 to identify that this was for 3.4 (or 3.5))
- link to build console
- link to build changes / git info
- currentBuild.description when job is done
pass SHA PR_Comment_URL to downstream job so it too can comment w/ console and currentBuild.description
repeat....
then roll out this change to all the jobs!

Phase 2:

extract JIRA IDs from the PR / commits, and comment on JIRAs w/ links to builds

Change-Id: Iad6e4c9def1d1340e97c1059c5d02e40ad40eaa2 Signed-off-by: Nick Boldt <[email protected]>

nickboldt · 2022-11-17T14:43:41Z

new methods are added to util2.groovy via this PR: https://github.com/redhat-developer/devspaces/pull/850/files

commentOnPullRequestBuildLinks
commentOnPullRequestBuildDescription

Plus utilities:

commentOnPullRequest(String comments_url, String message)
commentOnPullRequest(String ownerRepo, String SHA, String message)
defaultPullRequestComment
prepareHTMLStringForJSON

DONE:

switch to GH API instead of GH CLI
wrap code in a groovy method and add to util2.groovy
comment should include:
- version of DS being built (if 3.x, useful to parse https://github.com/redhat-developer/devspaces/blob/devspaces-3-rhel-8/dependencies/job-config.json#L3 to identify that this was for 3.4 (or 3.5))
- link to build console
- link to build changes / git info
- currentBuild.description when job is done
pass SHA PR_Comment_URL to downstream job so it too can comment w/ console and currentBuild.description

TODO:

use new methods in various jobs
update job to pass PR_Comment_URL to downstream jobs
update downstream jobs check if PR_Comment_URL already set; use if found else generate a new one if not

Phase 2:

extract JIRA IDs from the PR / commits, and comment on JIRAs w/ links to builds

Change-Id: Iad6e4c9def1d1340e97c1059c5d02e40ad40eaa2 Signed-off-by: Nick Boldt <[email protected]>

…850) Change-Id: Iad6e4c9def1d1340e97c1059c5d02e40ad40eaa2 Signed-off-by: Nick Boldt <[email protected]> Signed-off-by: Nick Boldt <[email protected]>

nickboldt · 2022-11-18T14:26:17Z

Enabling PR comments for template, sync-to-downstream, get-sources-rhpkg, and push-latest jobs for 3.x:

https://gitlab.cee.redhat.com/codeready-workspaces/crw-jenkins/-/merge_requests/1409

Affected (13):

code
configbump
dashboard
devfileregistry
idea
imagepuller
machineexec
operator-bundle
operator
plugin reg
server
trafik
udi

TODO:

dsc 3.x (triggered by sync-to-downstream if operator-bundle build)
apply changes to 3.3 jobs

Other jobs to consider enabling:

copyIIBsToQuay
update-digests
theia-akamai
theia-sources

…rigger so we can see what happens in eclipse-che/che#21795 Change-Id: I803297818611a30938f0f7ad15e33a39d73a817b Signed-off-by: Nick Boldt <[email protected]>

…eclipse-che/che#21795 Change-Id: I405ed5bba2a57be36f968f2de8c4c1c555ebb714 Signed-off-by: Nick Boldt <[email protected]>

…eclipse-che/che#21795 [devfile reg] Change-Id: I51469f618ffcef37d1e684ad6bef9eb1d598e665 Signed-off-by: Nick Boldt <[email protected]>

…eclipse-che/che#21795 [devfile reg] (#851) Change-Id: I51469f618ffcef37d1e684ad6bef9eb1d598e665 Signed-off-by: Nick Boldt <[email protected]> Signed-off-by: Nick Boldt <[email protected]>

…eclipse-che/che#21795 [plugin reg] Change-Id: I59a88d6f55d54f7eda87b0b91e5baec15c3e9252 Signed-off-by: Nick Boldt <[email protected]>

…eclipse-che/che#21795 [plugin reg] (#852) Change-Id: I59a88d6f55d54f7eda87b0b91e5baec15c3e9252 Signed-off-by: Nick Boldt <[email protected]> Signed-off-by: Nick Boldt <[email protected]>

nickboldt · 2022-11-18T14:33:56Z

Testing comments on PRs from Jenkins in these two PRs:

no-op change to cause a build to trigger so... redhat-developer/devspaces#852 [plugin reg]
no-op change to cause a build to trigger so... redhat-developer/devspaces#851 [devfile reg]

…appens in eclipse-che/che#21795 [plugin reg] (#852)" This reverts commit 4cfad4c.

…appens in eclipse-che/che#21795 [plugin reg] (#852)" (#853) This reverts commit 4cfad4c.

…eclipse-che/che#21795 [plugin reg] Change-Id: I59a88d6f55d54f7eda87b0b91e5baec15c3e9252 Signed-off-by: Nick Boldt <[email protected]> Signed-off-by: Nick Boldt <[email protected]>

…eclipse-che/che#21795 [plugin reg] (#856) Change-Id: I59a88d6f55d54f7eda87b0b91e5baec15c3e9252 Signed-off-by: Nick Boldt <[email protected]> Signed-off-by: Nick Boldt <[email protected]> Signed-off-by: Nick Boldt <[email protected]>

nickboldt · 2022-11-18T15:35:23Z

Didn't work, need full 40-char SHAs:

https://gitlab.cee.redhat.com/codeready-workspaces/crw-jenkins/-/merge_requests/1410

And need to check .merge_commit_sha:

redhat-developer/devspaces@fe653f6

Trying again...

redhat-developer/devspaces#856

…eclipse-che/che#21795 [plugin reg] This reverts commit c6007f8.

…eclipse-che/che#21795 [plugin reg] (#857) This reverts commit c6007f8.

nickboldt · 2022-11-18T15:58:27Z

Worked for plugin reg comment, but didn't work for sync-to-downstream.

Fix: https://gitlab.cee.redhat.com/codeready-workspaces/crw-jenkins/-/merge_requests/1414/diffs

Testing:

https://main-jenkins-csb-crwqe.apps.ocp-c1.prod.psi.redhat.com/job/DS_CI/job/pluginregistry_3.x/125/ --> redhat-developer/devspaces#857

…eclipse-che/che#21795 [plugin reg] This reverts commit 9f3c97f.

…eclipse-che/che#21795 [plugin reg] (#858) This reverts commit 9f3c97f.

…eclipse-che/che#21795 [plugin reg] (#857) This reverts commit c6007f8.

…eclipse-che/che#21795 [plugin reg] (#857) (#859) This reverts commit c6007f8.

nickboldt · 2022-11-18T23:18:45Z

Working for the test example eclipse-che/che-machine-exec#225

However the links to the job need to be regex-replaced so they are absolute links, not relative ones.

TODO:

fix links in comments to be absolute refs to Jenkins, not relative paths that are 404'd on github.com
add logic to copyIIBsToQuay so the list of IIBs appears in the PR
add logic to dsc 3.x job (triggered by sync-to-downstream if operator-bundle build)
add logic & webhook to operator-bundle job ?

Then:

backport changes to 3.3 jobs (if necessary?)

For these 2 repos, we will need a secondary webhook setup that can watch for config changes but NOT changes already synced from upstream, eg., a config change like redhat-developer/devspaces-images#353

redhat-developer/devspaces-images
redhat-developer/devspaces-chectl

Other jobs to consider enabling:

update-digests (to connect bundle builds to the builds that triggered them)
theia-sources, theia-akamai (deprecated but still useful?) [Artem says don't bother for now] :D

nickboldt · 2022-11-21T17:14:56Z

Improved links to Jenkins so they all work now.
added status badges and indented build status so it's easier to see start/end comments

nickboldt · 2022-11-21T22:20:54Z

add logic to copyIIBsToQuay so the list of IIBs appears in the PR
add logic to dsc 3.x job (triggered by sync-to-downstream if operator-bundle build)
---> https://gitlab.cee.redhat.com/codeready-workspaces/crw-jenkins/-/merge_requests/1425
--> https://gitlab.cee.redhat.com/codeready-workspaces/crw-jenkins/-/merge_requests/1426

Testing with: https://main-jenkins-csb-crwqe.apps.ocp-c1.prod.psi.redhat.com/job/DS_CI/job/push-latest-container-to-quay_3.x/1099/

Results pushed to: eclipse-che/che-machine-exec#229 ... but also to eclipse-che/che-operator#1564 (comment)

so we need to tweak the template_3.x job so that it checks if comments_url is set BEFORE just searching for the latest commit from the current source repo, instead of this:

            // comment on the PR with links to this job
            comments_url=util.commentOnPullRequestBuildLinks(SOURCE_REPO,util.getLastCommitSHA(SOURCE_DIR,40))
            println("Computed PR comment URL: " + comments_url)
            comments_url=comments_url.replaceAll("#.+","")

nickboldt · 2022-11-22T02:10:41Z

Fix: https://gitlab.cee.redhat.com/codeready-workspaces/crw-jenkins/-/merge_requests/1427 + https://gitlab.cee.redhat.com/codeready-workspaces/crw-jenkins/-/merge_requests/1428 + https://gitlab.cee.redhat.com/codeready-workspaces/crw-jenkins/-/merge_requests/1429

Have verified that traefik rebuilds (because they're keyed to a tag, not a branch) will no longer put comments on PRs, per redhat-developer/devspaces-images#357 and traefik/traefik#9515 (comment)

Building: https://main-jenkins-csb-crwqe.apps.ocp-c1.prod.psi.redhat.com/job/DS_CI/job/push-latest-container-to-quay_3.x/1103/console

Results should be here: eclipse-che/che-machine-exec#229 (comment) ... but not eclipse-che/che-operator#1564 ... so there's still a bug causing existing comments_url values to be overwritten :(

DONE:

add comments_url logic to dsc, operator-bundle, update-digests

For these 2 repos, we will need a secondary webhook setup that can watch for config changes but NOT changes already synced from upstream, eg., a config change like redhat-developer/devspaces-images#353

redhat-developer/devspaces-images, eg., for sync*.sh changes in operator or operator-bundle
redhat-developer/devspaces-chectl

==> moved to https://issues.redhat.com/browse/CRW-3550

ensure copyIIBs job comments 2x on the PR (start w/ links and end w/ status)

ensure that operator-bundle when triggered from another PR, uses the upstream project PR (dashboard, devfile reg, machine exec, server) and not just the latest che-operator PR. comments_url values should not be computed if they're already set by the referring job -- YES, see chore: Removing deprecated devfile endpoints. Updating the 2.2 schema to the latest version che-server#386

nickboldt · 2022-11-22T20:28:41Z

this PR eclipse-che/che-machine-exec#230 shows a nearly perfect build record:

machineexec -> sync-to-down -> get-sources-rhpkg
push-latest
copyIIBs (job failed, unrelated reasons)

Then a second build cycle started for operator-bundle:

operator-bundle (missing the initial comment, but the status at the end appears)
sync-to-down (missing the initial comment, but the status at the end appears)
get-sources-rhpkg (missing the initial comment, but the status at the end appears)
dsc (both link and description messages)

nickboldt · 2022-11-22T22:10:24Z

even better build record: eclipse-che/che-server#386 (dsc job failed, copyIIB successful 2x!)

add missing notifications to template-based jobs and update-digests -- https://gitlab.cee.redhat.com/codeready-workspaces/crw-jenkins/-/merge_requests/1433
- merged, waiting for job-configurator respin, then
- merged no-op change to test GH PR commit automation... che-machine-exec#231
- see what happens in https://main-jenkins-csb-crwqe.apps.ocp-c1.prod.psi.redhat.com/job/DS_CI/job/machineexec_3.x/55/ ... all jobs logged EXCEPT 2nd comment from https://main-jenkins-csb-crwqe.apps.ocp-c1.prod.psi.redhat.com/job/DS_CI/job/push-latest-container-to-quay_3.x/1119/console because it failed ... so unless I can figure out a way to report when the job fails, this is as good as it's going to get :D

DONE:

backport changes to 3.3 jobs - https://gitlab.cee.redhat.com/codeready-workspaces/crw-jenkins/-/merge_requests/1435 and feat: gh #21795 backport new methods from... redhat-developer/devspaces#862

nickboldt · 2022-11-23T14:44:55Z

Followup enhancements:

CRW-3581 enable github to jira integration (openshift-ci)
CRW-3601 in PR comments, switch from comments API to review-comments so we can update 'job started' to 'job done' messages

nickboldt added the kind/enhancement A feature request - must adhere to the feature request template. label Oct 31, 2022

nickboldt changed the title ~~Set up GH Actions to comment on upstream and downstream PRs~~ Set up automation to comment on PRs w/ links to downstream jobs Oct 31, 2022

che-bot added the status/need-triage An issue that needs to be prioritized by the curator responsible for the triage. See https://github. label Oct 31, 2022

nickboldt added severity/P2 Has a minor but important impact to the usage or development of the system. area/ci CI build and releases, PR testing, & whitelabel/productization issues labels Oct 31, 2022

dkwon17 removed the status/need-triage An issue that needs to be prioritized by the curator responsible for the triage. See https://github. label Nov 1, 2022

nickboldt mentioned this issue Nov 1, 2022

Sprint 227 #21785

Closed

73 tasks

nickboldt added a commit to redhat-developer/devspaces that referenced this issue Nov 17, 2022

chore: add methods for PR comment automation (eclipse-che/che#21795)

7bcb35d

Change-Id: Iad6e4c9def1d1340e97c1059c5d02e40ad40eaa2 Signed-off-by: Nick Boldt <[email protected]>

nickboldt mentioned this issue Nov 17, 2022

chore: add methods for PR comment automation... redhat-developer/devspaces#850

Merged

9 tasks

devstudio-release mentioned this issue Nov 17, 2022

fix typos and improve quiet/verbose... redhat-developer/devspaces#848

Merged

9 tasks

nickboldt added a commit to redhat-developer/devspaces that referenced this issue Nov 17, 2022

chore: add methods for PR comment automation (eclipse-che/che#21795)

4c1fa8e

Change-Id: Iad6e4c9def1d1340e97c1059c5d02e40ad40eaa2 Signed-off-by: Nick Boldt <[email protected]>

nickboldt added a commit to redhat-developer/devspaces that referenced this issue Nov 17, 2022

chore: add methods for PR comment automation (eclipse-che/che#21795)

9a9030d

Change-Id: Iad6e4c9def1d1340e97c1059c5d02e40ad40eaa2 Signed-off-by: Nick Boldt <[email protected]>

ibuziuk added the sprint/next label Nov 17, 2022

ibuziuk assigned nickboldt Nov 17, 2022

ibuziuk added sprint/current and removed sprint/next labels Nov 17, 2022

nickboldt mentioned this issue Nov 18, 2022

no-op change to cause a build to trigger so... redhat-developer/devspaces#851

Merged

9 tasks

nickboldt mentioned this issue Nov 18, 2022

no-op change to cause a build to trigger so... redhat-developer/devspaces#852

Merged

9 tasks

nickboldt added a commit to redhat-developer/devspaces that referenced this issue Nov 18, 2022

Revert "no-op change to cause a build to trigger so we can see what h…

fdc5dde

…appens in eclipse-che/che#21795 [plugin reg] (#852)" This reverts commit 4cfad4c.

nickboldt mentioned this issue Nov 18, 2022

Revert "no-op change to cause a build to... redhat-developer/devspaces#853

Merged

9 tasks

nickboldt added a commit to redhat-developer/devspaces that referenced this issue Nov 18, 2022

Revert "no-op change to cause a build to trigger so we can see what h…

6b4ac6a

…appens in eclipse-che/che#21795 [plugin reg] (#852)" (#853) This reverts commit 4cfad4c.

nickboldt mentioned this issue Nov 18, 2022

no-op change to cause a build to trigger so... redhat-developer/devspaces#856

Merged

9 tasks

nickboldt added a commit to redhat-developer/devspaces that referenced this issue Nov 18, 2022

no-op change to cause a build to trigger so we can see what happens in …

abef6ea

…eclipse-che/che#21795 [plugin reg] This reverts commit c6007f8.

nickboldt mentioned this issue Nov 18, 2022

no-op change to cause a build to trigger so... redhat-developer/devspaces#857

Merged

9 tasks

nickboldt added a commit to redhat-developer/devspaces that referenced this issue Nov 18, 2022

no-op change to cause a build to trigger so we can see what happens in …

9f3c97f

…eclipse-che/che#21795 [plugin reg] (#857) This reverts commit c6007f8.

nickboldt added a commit to redhat-developer/devspaces that referenced this issue Nov 18, 2022

no-op change to cause a build to trigger so we can see what happens in …

93b6c9d

…eclipse-che/che#21795 [plugin reg] This reverts commit 9f3c97f.

nickboldt mentioned this issue Nov 18, 2022

no-op change to cause a build to trigger so... redhat-developer/devspaces#858

Merged

9 tasks

nickboldt added a commit to redhat-developer/devspaces that referenced this issue Nov 18, 2022

no-op change to cause a build to trigger so we can see what happens in …

ed3a9fc

…eclipse-che/che#21795 [plugin reg] (#858) This reverts commit 9f3c97f.

nickboldt added a commit to redhat-developer/devspaces that referenced this issue Nov 18, 2022

no-op change to cause a build to trigger so we can see what happens in …

3102bb5

…eclipse-che/che#21795 [plugin reg] (#857) This reverts commit c6007f8.

nickboldt mentioned this issue Nov 18, 2022

no-op change to cause a build to trigger so... redhat-developer/devspaces#859

Merged

9 tasks

nickboldt added a commit to redhat-developer/devspaces that referenced this issue Nov 18, 2022

no-op change to cause a build to trigger so we can see what happens in …

786cadc

…eclipse-che/che#21795 [plugin reg] (#857) (#859) This reverts commit c6007f8.

This was referenced Nov 22, 2022

Why your bot comes inside our pull request? redhat-developer/devspaces-images#357

Closed

Sprint 228 #21823

Closed

nickboldt closed this as completed Dec 7, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Set up automation to comment on PRs w/ links to downstream jobs #21795

Set up automation to comment on PRs w/ links to downstream jobs #21795

nickboldt commented Oct 31, 2022 •

edited

Loading

nickboldt commented Nov 16, 2022 •

edited

Loading

nickboldt commented Nov 17, 2022 •

edited

Loading

nickboldt commented Nov 18, 2022

nickboldt commented Nov 18, 2022

nickboldt commented Nov 18, 2022 •

edited

Loading

nickboldt commented Nov 18, 2022

nickboldt commented Nov 18, 2022 •

edited

Loading

nickboldt commented Nov 21, 2022

nickboldt commented Nov 21, 2022 •

edited

Loading

nickboldt commented Nov 22, 2022 •

edited

Loading

nickboldt commented Nov 22, 2022

nickboldt commented Nov 22, 2022 •

edited

Loading

nickboldt commented Nov 23, 2022 •

edited

Loading

Set up automation to comment on PRs w/ links to downstream jobs #21795

Set up automation to comment on PRs w/ links to downstream jobs #21795

Comments

nickboldt commented Oct 31, 2022 • edited Loading

Is your enhancement related to a problem? Please describe

Describe the solution you'd like

Describe alternatives you've considered

Additional context

nickboldt commented Nov 16, 2022 • edited Loading

nickboldt commented Nov 17, 2022 • edited Loading

nickboldt commented Nov 18, 2022

nickboldt commented Nov 18, 2022

nickboldt commented Nov 18, 2022 • edited Loading

nickboldt commented Nov 18, 2022

nickboldt commented Nov 18, 2022 • edited Loading

nickboldt commented Nov 21, 2022

nickboldt commented Nov 21, 2022 • edited Loading

nickboldt commented Nov 22, 2022 • edited Loading

nickboldt commented Nov 22, 2022

nickboldt commented Nov 22, 2022 • edited Loading

nickboldt commented Nov 23, 2022 • edited Loading

nickboldt commented Oct 31, 2022 •

edited

Loading

nickboldt commented Nov 16, 2022 •

edited

Loading

nickboldt commented Nov 17, 2022 •

edited

Loading

nickboldt commented Nov 18, 2022 •

edited

Loading

nickboldt commented Nov 18, 2022 •

edited

Loading

nickboldt commented Nov 21, 2022 •

edited

Loading

nickboldt commented Nov 22, 2022 •

edited

Loading

nickboldt commented Nov 22, 2022 •

edited

Loading

nickboldt commented Nov 23, 2022 •

edited

Loading