Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Move gradle-check code to its own scripts and upload codecov #3742

Merged
merged 3 commits into from
Jun 30, 2022
Merged

Move gradle-check code to its own scripts and upload codecov #3742

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
6f84aad
Move gradle-check to script, add codecov, add comments
peterzhuamazon Jun 29, 2022
57b7190
Add more improvements on comments handling
peterzhuamazon Jun 30, 2022
f1df2ea
Add more comments
peterzhuamazon Jun 30, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
90 changes: 41 additions & 49 deletions .github/workflows/gradle-check.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,58 +16,50 @@ jobs:
echo "pr_title=$(jq --raw-output .pull_request.title $GITHUB_EVENT_PATH)" >> $GITHUB_ENV
echo "pr_number=$(jq --raw-output .pull_request.number $GITHUB_EVENT_PATH)" >> $GITHUB_ENV

- name: Checkout opensearch-build repo
uses: actions/checkout@v2
with:
repository: opensearch-project/opensearch-build
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of having dependency on the build repo. Can we do something suggested here: #3717 (comment)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope, that workflow have not been maintained for over 3 years and it is in 0.0.1 version.
I will not use that for production purposes.

Thanks.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also that one is for API token not the same as generic webhook, where we can pass a lot of information between github and Jenkins.

I dont see a dependency on opensearch-build is an issue the jenkinsfile is in opensearch-build already.

Copy link
Member

@owaiskazi19 owaiskazi19 Jun 29, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we have the scripts present in OpenSearch repo? Also, a good practice to mark the conversation resolved by the reviewer can be find here

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The point to present it in opensearch-build repo is so that you dont need to backport to every single branch whenever there is a tiny change.

Remember you have more than 10+ branches need gradle check.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for raising this issues tho, but I believe this is the best way to get gradle check start running.
We can always improve in later PRs.

Thanks.

Copy link
Member

@dreamer-89 dreamer-89 Jun 30, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@peterzhuamazon : Thanks for your effort in pulling up this change.

Regarding location of gradle script, I think it logically belongs to OpenSearch repository. Once the change is stabilized (green runs on main), we can backport this to remaining branches. Going forward; there wouldn't be work needed for future branches cut from main (or release branches).

@dblock @reta WDYT ?

Copy link
Collaborator

@reta reta Jun 30, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dreamer-89 that's a good point, I have seen both (central repo for build scripts and replicating build scripts across many branches). I think having build material along each branch is the best option when build scripts are trivial or non-existent (like pure Github actions let say). When build scripts are non-trivial (I think this is our case at least now), keeping them centralized at some place (build repository / library) serves better.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @reta @dreamer-89

Yes that is the reason why I keep the script in build repo for now.
All the existing actions on market are either not meeting our needs, or too old without any maintenance.
We can improve this later by either forking one of the existing actions, or write a new one to replace.

Right now, like you suggested, it is best to leave it in build repo for now.

Thanks.

ref: main

- name: Trigger jenkins workflow to run gradle check
run: |
echo "${{ env.pr_to_clone_url }} PR: ${{ env.pr_number }}"
echo "Raise from ${{ env.pr_from_clone_url }} ${{ env.pr_from_sha }}"
echo "PR Title: ${{ env.pr_title }}"
JENKINS_URL="https://build.ci.opensearch.org"
TIMEPASS=0
TIMEOUT=7200
RESULT="null"

echo "Trigger Jenkins workflows"
JENKINS_REQ=$(curl -s -XPOST \
-H "Authorization: Bearer ${{ secrets.JENKINS_GRADLE_CHECK_GENERIC_WEBHOOK_TOKEN }}" \
-H "Content-Type: application/json" \
--data '{"pr_from_sha": "${{ env.pr_from_sha }}", "pr_from_clone_url": "${{ env.pr_from_clone_url }}", "pr_to_clone_url": "${{ env.pr_to_clone_url }}", "pr_title": "${{ env.pr_title }}", "pr_number": "${{ env.pr_number }}"}' \
'https://build.ci.opensearch.org/generic-webhook-trigger/invoke')

QUEUE_URL=$(echo $JENKINS_REQ | jq --raw-output '.jobs."gradle-check".url')
echo QUEUE_URL $QUEUE_URL
echo "wait for jenkins to start workflow" && sleep 15

echo "Check if queue exist in Jenkins after triggering"
if [ -z "$QUEUE_URL" ] || [ "$QUEUE_URL" != "null" ]; then
WORKFLOW_URL=$(curl -s -XGET ${JENKINS_URL}/${QUEUE_URL}api/json | jq --raw-output .executable.url)
echo WORKFLOW_URL $WORKFLOW_URL
set -e
set -o pipefail
bash scripts/gradle/gradle-check.sh ${{ secrets.JENKINS_GRADLE_CHECK_GENERIC_WEBHOOK_TOKEN }} | tee -a gradle-check.log

echo "Use queue information to find build number in Jenkins if available"
if [ -z "$WORKFLOW_URL" ] || [ "$WORKFLOW_URL" != "null" ]; then

RUNNING="true"

echo "Waiting for Jenkins to complete the run"
while [ "$RUNNING" = "true" ] && [ "$TIMEPASS" -le "$TIMEOUT" ]; do
echo "Still running, wait for another 30 seconds before checking again, max timeout $TIMEOUT"
echo "Jenkins Workflow Url: $WORKFLOW_URL"
TIMEPASS=$(( TIMEPASS + 30 )) && echo time pass: $TIMEPASS
sleep 30
RUNNING=$(curl -s -XGET ${WORKFLOW_URL}api/json | jq --raw-output .building)
done

echo "Complete the run, checking results now......"
RESULT=$(curl -s -XGET ${WORKFLOW_URL}api/json | jq --raw-output .result)
- name: Setup Result Status
if: always()
run: |
WORKFLOW_URL=`cat gradle-check.log | grep 'WORKFLOW_URL' | awk '{print $2}'`
RESULT=`cat gradle-check.log | grep 'Result:' | awk '{print $2}'`
echo "workflow_url=$WORKFLOW_URL" >> $GITHUB_ENV
echo "result=$RESULT" >> $GITHUB_ENV

fi
fi
- name: Upload Coverage Report
if: success()
uses: codecov/codecov-action@v2
with:
files: ./codeCoverage.xml

echo "Please check jenkins url for logs: $WORKFLOW_URL"
- name: Create Comment Success
if: success()
uses: peter-evans/create-or-update-comment@v2
with:
issue-number: ${{ env.pr_number }}
body: |
### Gradle Check (Jenkins) Run Completed with:
* **RESULT:** ${{ env.result }} :white_check_mark:
* **URL:** ${{ env.workflow_url }}
* **CommitID:** ${{ env.pr_from_sha }}

if [ "$RESULT" != "SUCCESS" ]; then
echo "Result: $RESULT"
exit 1
else
echo "Result: $RESULT"
echo 0
fi
- name: Create Comment Failure
if: failure()
uses: peter-evans/create-or-update-comment@v2
with:
issue-number: ${{ env.pr_number }}
body: |
### Gradle Check (Jenkins) Run Completed with:
* **RESULT:** ${{ env.result }} :x:
* **URL:** ${{ env.workflow_url }}
* **CommitID:** ${{ env.pr_from_sha }}