-
Notifications
You must be signed in to change notification settings - Fork 5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comment with dependency diff #120
Merged
Merged
Changes from 42 commits
Commits
Show all changes
44 commits
Select commit
Hold shift + click to select a range
2072102
Raw copy of the script from Day One Android
wzieba 8c3785f
Accept configuration, owner and repo parameters
wzieba f9f6019
Support Kotlin script (`*.gradle.kts`) files as well
wzieba b1d21df
Support Dependencies Catalog (`*.toml`) files
wzieba b9fc9f3
Use fastlane action for commenting on a PR
wzieba 8a79cae
Provide module argument
wzieba 1c0d130
Remove dependencyTreeDiffCommentToGitHub execution if no .gradle file…
wzieba 65a0f10
Fix checking for arguments
wzieba 453286e
Use -E for file pattern matching
wzieba 48a72d9
Fix typo when providing reuse-identifier
wzieba 1d4c242
Fix running fastlane action
wzieba 8a7c97d
Add support for buildEnvironment diff
wzieba 05b0c1e
Use template to generate comment body
wzieba 0eb94ec
Actually create $DIFF_BUILD_ENV_FILE. Check if diff files exist befor…
wzieba 8fecda9
Apply fixes to how comment body is created
wzieba 9b54d2a
Don't call `diff` tool
wzieba 8ecae63
Improve formatting of the comment messages
wzieba 1b4f2d8
Log status and diff before checkout
wzieba 470dc33
Sometimes, a diff might be generated (e.g. Gemfile.lock after gem ins…
wzieba 06908dc
Use file to keep comment body
wzieba 87bbed0
Another approach to fix Argument list too long
wzieba 48964e6
Verify if the problem is size of the comment
wzieba f7a7dfc
Use comment_on_pr script instead of fastlane action
wzieba f9ed024
Pass comment file to comment_on_pr
wzieba e6036b3
Possible fix for Argument list too long in comment_on_pr
wzieba 5ce8db0
Pass json_payload_file as a file
wzieba d3b312d
Pass raw file to jq
wzieba 6b9b0d8
Cover against GitHub comment max size
wzieba 3fabdf1
Check if adding the new content will exceed the maximum allowed size
wzieba 1f47440
Improve markdown formatting
wzieba c13194f
Remove quotation marks from configuration name
wzieba a907fe1
Don't require providing owner and repo names
wzieba fbefec7
Add a changelog entry
wzieba a5c13b6
Remove forgotten debug code
wzieba 32fc243
Improve clarity with json_payload_file creation
wzieba 0e36309
Add user manual for the comment_with_dependency_diff
wzieba c00a998
Protect against lack of BUILDKITE_PULL_REQUEST
wzieba 20681ce
Use $BUILDKITE_PULL_REQUEST_BASE_BRANCH instead of fetching base bran…
wzieba e12d73b
Use uppercase for baseBranch
wzieba 823cf96
Move dependency tree checksum
wzieba 613fcab
Account the warnings message + header size when calculating possible …
wzieba 6696f47
Add a link to "content exceeds" message
wzieba a60c013
Include header in "content exceeds" case
wzieba 242bf1e
Merge branch 'trunk' into comment_with_dependency_diff
wzieba File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,181 @@ | ||
#!/bin/bash | ||
|
||
# This script is used in a Buildkite CI pipeline to analyze and comment on dependency changes in a | ||
# GitHub pull request (PR). It compares the Gradle project dependencies and build environment between the | ||
# base branch and the head branch to identify changes caused by updates to Gradle files. | ||
# | ||
# Usage: | ||
# ./comment_with_dependency_diff <MODULE> <CONFIGURATION> | ||
# | ||
# Parameters: | ||
# <MODULE> : The Gradle module whose dependencies will be checked. | ||
# <CONFIGURATION> : The Gradle configuration to inspect for dependencies (e.g., "runtimeClasspath"). | ||
# | ||
# Steps: | ||
# 1. The script fetches the base branch of the PR and merges it into the head branch. | ||
# 2. It generates the dependency and build environment reports for both the base and head branches. | ||
# 3. The script compares these reports to detect any changes in the project's dependencies. | ||
# 4. If changes are found, the script creates a comment on the GitHub PR with the results. | ||
# 5. If no changes are detected, the script exits without posting any comment. | ||
# | ||
# Notes: | ||
# - The GitHub token must be set via the environment variable `GITHUB_TOKEN` to authenticate API requests. | ||
# - The comment on the PR will include a summary of changes and a detailed list in expandable sections. | ||
# - A jar file `dependency-tree-diff.jar` is downloaded and executed to perform the dependency diff. | ||
# | ||
# Example: | ||
# ./comment_with_dependency_diff app release | ||
# | ||
# This will analyze the `app` module with the `release` configuration and post the result to the PR. | ||
|
||
set -euo pipefail | ||
|
||
MODULE="${1:-}" | ||
CONFIGURATION="${2:-}" | ||
|
||
if [ -z "$MODULE" ] || [ -z "$CONFIGURATION" ]; then | ||
echo "Not enough arguments provided. Usage: ./comment_with_dependency_diff <MODULE> <CONFIGURATION>" | ||
exit 1 | ||
fi | ||
|
||
if [ "${BUILDKITE_PULL_REQUEST:-false}" == "false" ]; then | ||
echo "This script should only be called on pull requests. Be sure to set \`if: build.pull_request.id != null\` on the step invoking it in your Buildkite pipeline" | ||
exit 2 | ||
fi | ||
|
||
DIFF_DEPENDENCIES_FOLDER="./build/reports/diff" | ||
BASE_BRANCH_DEPENDENCIES_FILE="$DIFF_DEPENDENCIES_FOLDER/base_branch_dependencies.txt" | ||
HEAD_BRANCH_DEPENDENCIES_FILE="$DIFF_DEPENDENCIES_FOLDER/head_branch_dependencies.txt" | ||
BASE_BRANCH_BUILD_ENV_FILE="$DIFF_DEPENDENCIES_FOLDER/base_branch_build_env.txt" | ||
HEAD_BRANCH_BUILD_ENV_FILE="$DIFF_DEPENDENCIES_FOLDER/head_branch_build_env.txt" | ||
DIFF_DEPENDENCIES_FILE="$DIFF_DEPENDENCIES_FOLDER/diff_dependencies.txt" | ||
DIFF_BUILD_ENV_FILE="$DIFF_DEPENDENCIES_FOLDER/diff_build_env.txt" | ||
|
||
DEPENDENCY_TREE_VERSION="1.2.1" | ||
DEPENDENCY_TREE_CHECKSUM="39c50f22cd4211a3e52afec43120b7fdf90f9890" | ||
|
||
git config --global user.email "[email protected]" | ||
git config --global user.name "Dependency Tree Diff Tool" | ||
|
||
echo "--> Starting the check" | ||
BASE_BRANCH=$BUILDKITE_PULL_REQUEST_BASE_BRANCH | ||
echo "--> Base branch is $BASE_BRANCH" | ||
|
||
git merge "origin/$BASE_BRANCH" --no-edit | ||
|
||
if (git diff --name-status "origin/$BASE_BRANCH" | grep -E "\.gradle(\.kts)?|\.toml" --quiet); then | ||
echo "Gradle files have been changed. Looking for caused dependency changes" | ||
else | ||
echo "Gradle files haven't been changed. There is no need to run the diff" | ||
exit 0 | ||
fi | ||
|
||
mkdir -p "$DIFF_DEPENDENCIES_FOLDER" | ||
|
||
echo "--> Generating dependencies to the file $HEAD_BRANCH_DEPENDENCIES_FILE" | ||
./gradlew :"$MODULE":dependencies --configuration "$CONFIGURATION" >$HEAD_BRANCH_DEPENDENCIES_FILE | ||
|
||
echo "--> Generating build environment dependencies for the head branch" | ||
./gradlew buildEnvironment >$HEAD_BRANCH_BUILD_ENV_FILE | ||
|
||
echo "--> Generating dependencies to the file $BASE_BRANCH_DEPENDENCIES_FILE" | ||
|
||
if ! git diff --quiet; then | ||
echo "Changes detected. Stashing and continuing with the script" | ||
git diff | ||
git stash | ||
fi | ||
|
||
git checkout "$BASE_BRANCH" | ||
./gradlew :"$MODULE":dependencies --configuration "$CONFIGURATION" >$BASE_BRANCH_DEPENDENCIES_FILE | ||
|
||
echo "--> Generating build environment dependencies for the base branch" | ||
./gradlew buildEnvironment >$BASE_BRANCH_BUILD_ENV_FILE | ||
|
||
git checkout "-" | ||
|
||
echo "--> Downloading dependency-tree-diff.jar" | ||
curl -v -L "https://github.com/JakeWharton/dependency-tree-diff/releases/download/$DEPENDENCY_TREE_VERSION/dependency-tree-diff.jar" -o dependency-tree-diff.jar | ||
sha=$(sha1sum dependency-tree-diff.jar | cut -d' ' -f1) | ||
if [[ $sha != "$DEPENDENCY_TREE_CHECKSUM" ]]; then | ||
echo "dependency-tree-diff.jar file has unexpected sha1" | ||
exit 1 | ||
fi | ||
chmod +x dependency-tree-diff.jar | ||
|
||
echo "--> Running dependency-tree-diff.jar" | ||
./dependency-tree-diff.jar $BASE_BRANCH_DEPENDENCIES_FILE $HEAD_BRANCH_DEPENDENCIES_FILE >$DIFF_DEPENDENCIES_FILE | ||
|
||
./dependency-tree-diff.jar $BASE_BRANCH_BUILD_ENV_FILE $HEAD_BRANCH_BUILD_ENV_FILE >$DIFF_BUILD_ENV_FILE | ||
|
||
generate_project_dependencies_comment_body() { | ||
cat <<EOF | ||
|
||
The following changes in project dependencies were detected (configuration \`$CONFIGURATION\`): | ||
|
||
<details><summary>expand</summary> | ||
|
||
\`\`\`diff | ||
$(cat "$DIFF_DEPENDENCIES_FILE") | ||
\`\`\` | ||
|
||
</details> | ||
|
||
|
||
EOF | ||
} | ||
|
||
generate_build_dependencies_comment_body() { | ||
cat <<EOF | ||
|
||
The following changes in the build classpath were detected: | ||
|
||
<details><summary>expand</summary> | ||
|
||
\`\`\`diff | ||
$(cat "$DIFF_BUILD_ENV_FILE") | ||
\`\`\` | ||
|
||
</details> | ||
|
||
EOF | ||
} | ||
|
||
if [ -n "$DIFF_DEPENDENCIES_FILE" ] || [ -n "$DIFF_BUILD_ENV_FILE" ]; then | ||
echo "--> Commenting result to GitHub" | ||
|
||
# Create or clear the comment body file | ||
COMMENT_FILE="$DIFF_DEPENDENCIES_FOLDER/comment_body.txt" | ||
true > $COMMENT_FILE # 'true' used to clear the file as a no-op | ||
|
||
append_content() { | ||
local header="$1" | ||
local content="$2" | ||
local github_comment_max_size=65536 | ||
local warning_message="Content exceeds $github_comment_max_size characters. [Navigate to Buildkite build artifacts](${BUILDKITE_BUILD_URL}#${BUILDKITE_JOB_ID}) to see details." | ||
|
||
local current_size | ||
current_size=$(wc -c < "$COMMENT_FILE") | ||
|
||
# Calculate if there's enough space for the content and header | ||
if (( current_size + ${#header} + ${#content} > github_comment_max_size - ${#warning_message} )); then | ||
printf "%s\n" "$warning_message" > $COMMENT_FILE | ||
return | ||
fi | ||
|
||
printf "\n%s\n" "$header" >> $COMMENT_FILE | ||
printf "%s\n" "$content" >> $COMMENT_FILE | ||
} | ||
|
||
if [ -s "$DIFF_DEPENDENCIES_FILE" ]; then | ||
append_content "## Project dependencies changes" "$(generate_project_dependencies_comment_body)" | ||
fi | ||
|
||
if [ -s "$DIFF_BUILD_ENV_FILE" ]; then | ||
append_content "## Build environment changes" "$(generate_build_dependencies_comment_body)" | ||
fi | ||
|
||
comment_on_pr --id "dependency-diff" --if-exist update "$COMMENT_FILE" | ||
else | ||
comment_on_pr --id "dependency-diff" --if-exist delete | ||
fi |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
📚 The more we add utility scripts like those to this plugin, the more I realize we should have made a better job as providing help / documentation at what each of those do. Especially so that people not as familiar with our CI (e.g. devs that are interested in using some of those helper scripts in their pipelines themselves but have no idea what each utility does) can better understand the purpose of each.
For example, the comment at the start of
comment_on_pr
is a good example of having some code-comment documentation on what the utility does and how it's expected to be usedI wish we took that habit sooner and applied it to all of our other
bin/*
utilities in this repo, making them all start with a code-comment explaining their role / context in which those helpers would be used and could help, documenting what the script does and its expected parameters and inputs (e.g. "this script should only run after./gradlew …
has been called and generated the build artifacts", etc).We should probably make a pass at all of them at some point.
In the meantime, wdyt about us already starting moving towards this good habit and add some intro doc comments in any new helpers we introduce, starting from this one?