Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backport PR #2773 to release/v1.7 for Refactor github actions #2779

Conversation

vdaas-ci
Copy link
Collaborator

@vdaas-ci vdaas-ci commented Dec 13, 2024

Description

SSIA

Related Issue

Versions

  • Vald Version: v1.7.14
  • Go Version: v1.23.3
  • Rust Version: v1.82.0
  • Docker Version: v27.3.1
  • Kubernetes Version: v1.31.2
  • Helm Version: v3.16.2
  • NGT Version: v2.3.4
  • Faiss Version: v1.9.0

Checklist

Special notes for your reviewer

Summary by CodeRabbit

  • New Features

    • Introduced new steps to check for installations of Go, Helm, k3d, and yq, enhancing the setup process.
    • Added a new input parameter for the Vald read replica deployment action.
  • Bug Fixes

    • Corrected grammatical errors in action descriptions across multiple workflows.
  • Chores

    • Removed unnecessary setup steps for Go, Helm, and yq from various workflows.
    • Updated Makefiles to reflect changes in yq installation processes.
    • Added a new target for installing yq in the tools Makefile.

Signed-off-by: kpango <[email protected]>
Co-authored-by: Kiichiro YUKAWA <[email protected]>
Copy link

cloudflare-workers-and-pages bot commented Dec 13, 2024

Deploying vald with  Cloudflare Pages  Cloudflare Pages

Latest commit: ce1af2b
Status: ✅  Deploy successful!
Preview URL: https://00f19c08.vald.pages.dev
Branch Preview URL: https://backport-release-v1-7-refact-5usm.vald.pages.dev

View logs

Copy link
Contributor

coderabbitai bot commented Dec 13, 2024

📝 Walkthrough

Walkthrough

This pull request primarily focuses on updating the descriptions of various GitHub Actions to correct grammatical errors. The changes span multiple action configuration files, with each update changing "A action" to "An action." Additionally, some actions had minor modifications in their input parameters, particularly regarding default values. The overall functionality and structure of the actions remain unchanged, with no alterations to their logic or execution steps.

Changes

File Path Change Summary
.github/actions/deploy-chaos-mesh/action.yaml Description updated from "A action to deploy Chaos Mesh" to "An action to deploy Chaos Mesh."
.github/actions/detect-docker-image-tags/action.yaml Description updated from "A action to detect Docker image tags" to "An action to detect Docker image tags."
.github/actions/determine-docker-image-tag/action.yaml Description updated from "A action to determine Docker image tag" to "An action to determine Docker image tag."
.github/actions/docker-build/action.yaml Description updated from "A action to build Docker images and publish them" to "An action to build Docker images and publish them."
.github/actions/dump-context/action.yaml Description updated from "A action to dump context to log" to "An action to dump context to log."
.github/actions/e2e-deploy-vald-helm-operator/action.yaml Description updated from "A action to deploy vald using vald-helm-operator for E2E test" to "An action to deploy vald using vald-helm-operator for E2E test." Default value for valdrelease changed from "true" to no default value.
.github/actions/e2e-deploy-vald-readreplica/action.yaml Description updated from "A action to deploy vald read replica for E2E test" to "An action to deploy vald read replica for E2E test."
.github/actions/e2e-deploy-vald/action.yaml Description updated from "A action to deploy vald for E2E test" to "An action to deploy vald for E2E test."
.github/actions/notify-slack/action.yaml Description updated from "A action to notify Slack" to "An action to notify Slack."
.github/actions/scan-docker-image/action.yaml Description updated from "A action to scan the Docker image" to "An action to scan the Docker image."
.github/actions/setup-e2e/action.yaml Description updated from "A action to set up the environment for executing E2E test" to "An action to set up the environment for executing E2E test."
.github/actions/setup-go/action.yaml Description updated from "A action to set up Go environment" to "An action to set up Go environment." New step added to check if Go is installed. Step renamed from "Check Go version" to "Verify Go version."
.github/actions/setup-helm/action.yaml Description updated from "A action to set up Helm environment" to "An action to set up Helm environment." New step added to check if Helm is installed. Step renamed from "Check Helm version" to "Verify Helm version."
.github/actions/setup-k3d/action.yaml Description updated from "A action to set up k3d (k3s in Docker)" to "An action to set up k3d (k3s in Docker)." New step added to check if k3d is installed.
.github/actions/setup-yq/action.yaml Description updated from "A action to set up yq" to "An action to set up yq."
.github/actions/wait-for-docker-image/action.yaml Description updated from "A action to wait for Docker images to be published" to "An action to wait for Docker images to be published."
.github/workflows/build-binaries.yaml Step removed that set up the Go environment using setup-go.
.github/workflows/chatops.yaml Added container specification for several jobs and updated job execution conditions. Removed yq setup from jobs.
.github/workflows/e2e-code-bench-agent.yaml Removed setup-go action from grpc-sequential and grpc-stream jobs.
.github/workflows/format.yaml Removed "Setup Go environment" step from format and check-format-diff jobs.
.github/workflows/helm-lint.yaml Removed setup steps for Go, Helm, and YQ from multiple jobs.
.github/workflows/helm.yaml Removed "Setup Helm environment" step from update-helm-chart job.
Makefile.d/helm.mk Removed installation target for yq.
Makefile.d/tools.mk New target added for installing yq.

Possibly related PRs

Suggested labels

priority/low, type/refactoring, size/XXXL, type/ci, area/makefile, actions/backport/release/v1.7

Suggested reviewers

  • hlts2
  • kmrmt
  • vankichi

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR. (Experiment)
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Outside diff range and nitpick comments (5)
Makefile.d/tools.mk (1)

264-274: LGTM! Consider adding error handling for the curl command.

The yq installation target follows the established pattern and correctly handles architecture differences. However, the curl command could benefit from error handling.

Consider adding error checking:

 $(BINDIR)/yq:
 	mkdir -p $(BINDIR)
 	$(eval DARCH := $(subst aarch64,arm64,$(ARCH)))
 	cd $(TEMP_DIR) \
-	    && curl -fsSL https://github.com/mikefarah/yq/releases/download/$(YQ_VERSION)/yq_$(OS)_$(subst x86_64,amd64,$(shell echo $(DARCH) | tr '[:upper:]' '[:lower:]')) -o $(BINDIR)/yq \
+	    && curl -fsSL https://github.com/mikefarah/yq/releases/download/$(YQ_VERSION)/yq_$(OS)_$(subst x86_64,amd64,$(shell echo $(DARCH) | tr '[:upper:]' '[:lower:]')) -o $(BINDIR)/yq || (echo "Failed to download yq" && exit 1) \
 	    && chmod a+x $(BINDIR)/yq
.github/actions/setup-go/action.yaml (1)

34-43: Consider improving version extraction robustness

While the Go installation check works, the version extraction could be more robust:

  1. The current awk '{print $3}' assumes a specific format of go version output
  2. The version comparison in the subsequent step might not handle semantic versioning correctly

Consider this more robust version extraction:

- echo "installed_version=$(go version | awk '{print $3}')" >> $GITHUB_OUTPUT
+ echo "installed_version=$(go version | sed -n 's/.*go\([0-9.]*\).*/\1/p')" >> $GITHUB_OUTPUT
.github/actions/setup-k3d/action.yaml (2)

70-84: Improve robustness of k3d installation check

The new installation check and conditional installation logic is a good improvement that prevents unnecessary reinstallation. However, consider these enhancements:

  1. Add explicit error handling for the make command
  2. Implement proper semantic version comparison

Consider this enhancement:

 if command -v k3d &> /dev/null; then
   echo "k3d_installed=true" >> $GITHUB_OUTPUT
-  echo "installed_version=$(k3d version --short | sed 's/^v//')" >> $GITHUB_OUTPUT
+  INSTALLED_VERSION=$(k3d version --short | sed 's/^v//')
+  echo "installed_version=${INSTALLED_VERSION}" >> $GITHUB_OUTPUT
 else
   echo "k3d_installed=false" >> $GITHUB_OUTPUT
 fi

 if steps.check_k3d.outputs.k3d_installed == 'false' || steps.check_k3d.outputs.installed_version != steps.k3d_version.outputs.tag
   shell: bash
   run: |
-    make k3d/install
+    if ! make k3d/install; then
+      echo "Failed to install k3d"
+      exit 1
+    fi

Security: Confirmed sensitive information exposure across multiple actions

The verification reveals a pattern of exposing sensitive cluster information across multiple GitHub Actions:

  • .github/actions/setup-k3d/action.yaml: Prints full KUBECONFIG contents
  • .github/actions/setup-e2e/action.yaml: Executes full cluster-info dump
  • .github/actions/deploy-chaos-mesh/action.yaml: Executes full cluster-info dump

Recommendations:

  1. Replace cat $KUBECONFIG with a filtered output showing only non-sensitive fields
  2. Replace kubectl cluster-info dump with kubectl cluster-info which provides basic connectivity information
  3. If detailed debugging is needed, implement proper log redaction or limit the scope of information exposed
🔗 Analysis chain

Line range hint 124-125: Security: Consider limiting sensitive information exposure

The action currently prints sensitive information to logs:

  1. Full KUBECONFIG contents
  2. Complete cluster-info dump

Consider:

  1. Masking sensitive values in KUBECONFIG output
  2. Limiting cluster-info dump to essential information only

Let's check if this information exposure is consistent with other k8s-related actions:

Also applies to: 131-132


Let me check the content of these files to understand the context better.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Search for similar patterns in other action files
rg -l 'cluster-info dump' .github/actions/
rg -l 'cat.*KUBECONFIG' .github/actions/

Length of output: 247


Script:

#!/bin/bash
# Check the content of related action files
cat .github/actions/setup-e2e/action.yaml
cat .github/actions/deploy-chaos-mesh/action.yaml

Length of output: 5655

.github/actions/setup-helm/action.yaml (1)

34-43: LGTM! Robust Helm installation check implementation.

The implementation correctly:

  • Uses POSIX-compliant command checking
  • Handles both installed and not installed cases
  • Extracts version information when available

Consider adding error handling for the version extraction command in case it fails:

 if command -v helm &> /dev/null; then
   echo "helm_installed=true" >> $GITHUB_OUTPUT
-  echo "installed_version=$(helm version --short --client | awk '{print $2}' | sed 's/^v//')" >> $GITHUB_OUTPUT
+  if ! version=$(helm version --short --client | awk '{print $2}' | sed 's/^v//'); then
+    echo "::warning::Failed to extract Helm version"
+    echo "installed_version=" >> $GITHUB_OUTPUT
+  else
+    echo "installed_version=$version" >> $GITHUB_OUTPUT
+  fi
 else
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 7c25316 and ce1af2b.

📒 Files selected for processing (24)
  • .github/actions/deploy-chaos-mesh/action.yaml (1 hunks)
  • .github/actions/detect-docker-image-tags/action.yaml (1 hunks)
  • .github/actions/determine-docker-image-tag/action.yaml (1 hunks)
  • .github/actions/docker-build/action.yaml (1 hunks)
  • .github/actions/dump-context/action.yaml (1 hunks)
  • .github/actions/e2e-deploy-vald-helm-operator/action.yaml (1 hunks)
  • .github/actions/e2e-deploy-vald-readreplica/action.yaml (1 hunks)
  • .github/actions/e2e-deploy-vald/action.yaml (1 hunks)
  • .github/actions/notify-slack/action.yaml (1 hunks)
  • .github/actions/scan-docker-image/action.yaml (1 hunks)
  • .github/actions/setup-e2e/action.yaml (1 hunks)
  • .github/actions/setup-go/action.yaml (2 hunks)
  • .github/actions/setup-helm/action.yaml (2 hunks)
  • .github/actions/setup-k3d/action.yaml (2 hunks)
  • .github/actions/setup-yq/action.yaml (1 hunks)
  • .github/actions/wait-for-docker-image/action.yaml (1 hunks)
  • .github/workflows/build-binaries.yaml (0 hunks)
  • .github/workflows/chatops.yaml (5 hunks)
  • .github/workflows/e2e-code-bench-agent.yaml (0 hunks)
  • .github/workflows/format.yaml (0 hunks)
  • .github/workflows/helm-lint.yaml (0 hunks)
  • .github/workflows/helm.yaml (0 hunks)
  • Makefile.d/helm.mk (0 hunks)
  • Makefile.d/tools.mk (1 hunks)
💤 Files with no reviewable changes (6)
  • .github/workflows/helm.yaml
  • .github/workflows/build-binaries.yaml
  • .github/workflows/e2e-code-bench-agent.yaml
  • Makefile.d/helm.mk
  • .github/workflows/helm-lint.yaml
  • .github/workflows/format.yaml
✅ Files skipped from review due to trivial changes (13)
  • .github/actions/setup-e2e/action.yaml
  • .github/actions/docker-build/action.yaml
  • .github/actions/deploy-chaos-mesh/action.yaml
  • .github/actions/dump-context/action.yaml
  • .github/actions/wait-for-docker-image/action.yaml
  • .github/actions/notify-slack/action.yaml
  • .github/actions/e2e-deploy-vald/action.yaml
  • .github/actions/e2e-deploy-vald-readreplica/action.yaml
  • .github/actions/detect-docker-image-tags/action.yaml
  • .github/actions/e2e-deploy-vald-helm-operator/action.yaml
  • .github/actions/scan-docker-image/action.yaml
  • .github/actions/setup-yq/action.yaml
  • .github/actions/determine-docker-image-tag/action.yaml
🔇 Additional comments (8)
.github/workflows/chatops.yaml (2)

27-27: LGTM! Good security practice to prevent potential infinite loops.

The condition if: github.event.comment.user.login != 'vdaas-ci' prevents the bot from responding to its own comments, which could otherwise lead to infinite loops.

Also applies to: 77-77, 152-152, 264-264, 396-396


30-31: LGTM! Good practice using standardized container environment.

Using a pre-configured container image (ghcr.io/vdaas/vald/vald-ci-container:nightly) ensures consistent environment across all jobs and eliminates the need for redundant setup steps.

Let's verify the container image exists and is from a trusted source:

Also applies to: 80-81, 157-158, 270-271, 402-403

.github/actions/setup-go/action.yaml (3)

17-17: LGTM! Grammar fix looks good.

The description now correctly uses "An" instead of "A" before the word "action".


49-49: LGTM! Step name change is appropriate.

The rename from "Check" to "Verify" better reflects the step's purpose of confirming the final Go installation state.


44-46: Verify version comparison logic

The condition steps.check_go.outputs.installed_version != steps.go_version.outputs.version might not work as expected due to format differences:

  • installed_version might include "go" prefix
  • Version formats might differ (e.g., "1.20" vs "1.20.0")

Let's verify the version comparison logic:

✅ Verification successful

Let me check the actual implementation of the version comparison logic in the action.


Looking at the implementation, I can now generate the final response:

Version comparison is correctly implemented

The version comparison logic is actually correct because:

  • The installed_version is extracted using go version | awk '{print $3}' which gets just the version part (e.g., "go1.20.1" -> "go1.20.1")
  • The go_version step uses make version/go which would output the same format as it's the source of truth
  • The final verification step ensures the correct version is installed
🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Check how go version is formatted in different contexts
echo "Testing go version output format:"
go version

echo -e "\nChecking make version/go output format:"
make version/go 2>/dev/null || echo "Note: make command not available in sandbox"

Length of output: 392


Script:

#!/bin/bash
# Check the implementation of check_go step
echo "Checking check_go step implementation:"
cat .github/actions/setup-go/action.yaml

# Search for any test files that might show version comparison
echo -e "\nSearching for test files:"
fd -e test.yaml -e test.yml -e test.js -e test.sh . .github/actions/setup-go/

Length of output: 2122

.github/actions/setup-k3d/action.yaml (1)

17-17: LGTM: Grammar fix in description

The grammatical correction from "A action" to "An action" is appropriate.

.github/actions/setup-helm/action.yaml (2)

17-17: LGTM! Grammar correction applied.

The description now correctly uses "An" instead of "A" before the word "action".


49-52: LGTM! Clean version verification implementation.

The command uses appropriate flags to produce clean, parseable output.

.github/actions/setup-helm/action.yaml Show resolved Hide resolved
@vdaas-ci
Copy link
Collaborator Author

[CHATOPS:HELP] ChatOps commands.

  • 🙆‍♀️ /approve - approve
  • 🍱 /format - format codes and add licenses
  • /gen-test - generate test codes
  • 🏷️ /label - add labels
  • 🔚 2️⃣ 🔚 /label actions/e2e-deploy - run E2E deploy & integration test

@vdaas-ci
Copy link
Collaborator Author

[FORMAT] Failed to format.

@vankichi vankichi merged commit 8537f27 into release/v1.7 Dec 16, 2024
180 of 181 checks passed
@vankichi vankichi deleted the backport/release/v1.7/refactor/github-actions/remove-setup-go branch December 16, 2024 08:16
This was referenced Dec 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants