Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update ubuntu version for devcontainer #2736

Merged

Conversation

vankichi
Copy link
Contributor

@vankichi vankichi commented Nov 8, 2024

Description

SSIA

Related Issue

Versions

  • Vald Version: v1.7.13
  • Go Version: v1.23.2
  • Rust Version: v1.81.0
  • Docker Version: v27.3.1
  • Kubernetes Version: v1.31.1
  • Helm Version: v3.16.2
  • NGT Version: v2.2.4
  • Faiss Version: v1.9.0

Checklist

Special notes for your reviewer

Summary by CodeRabbit

  • New Features

    • Updated base image for development environment to Ubuntu 24.04.
    • Enhanced installation processes for Helm and Helm Docs to support multiple architectures.
  • Bug Fixes

    • Streamlined package installation process for development tools and dependencies.
  • Chores

    • Removed unnecessary components from the installation sequence, improving efficiency.

Copy link
Contributor

coderabbitai bot commented Nov 8, 2024

📝 Walkthrough

Walkthrough

The pull request includes updates to two main files: the Dockerfile and the Helm Makefile. The Dockerfile changes the base image from ubuntu22.04 to ubuntu24.04 and modifies the package installation process by removing certain packages and adding others. The Helm Makefile enhances the installation process for Helm and Helm Docs by introducing architecture-specific variables, allowing for a more flexible and consistent installation across different systems.

Changes

File Change Summary
dockers/dev/Dockerfile Updated base image to ubuntu24.04; modified package installation by removing software-properties-common and adding tzdata, locales, and development tools.
Makefile.d/helm.mk Improved Helm and Helm Docs installation processes by introducing DARCH variable for architecture-specific installations.

Possibly related PRs

Suggested labels

actions/backport/release/v1.7

Suggested reviewers

  • kpango
  • kmrmt

📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 03f5177 and 51540ff.

⛔ Files ignored due to path filters (1)
  • hack/docker/gen/main.go is excluded by !**/gen/**
📒 Files selected for processing (1)
  • dockers/dev/Dockerfile (2 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • dockers/dev/Dockerfile

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between a583f09 and 6b59437.

⛔ Files ignored due to path filters (1)
  • hack/docker/gen/main.go is excluded by !**/gen/**
📒 Files selected for processing (2)
  • dockers/dev/Dockerfile (1 hunks)
  • dockers/index/job/deletion/Dockerfile (1 hunks)
🔇 Additional comments (4)
dockers/index/job/deletion/Dockerfile (2)

88-88: LGTM: Entrypoint configuration.

The entrypoint configuration is correct and maintains the principle of least privilege by running as nonroot.


85-85: Verify the use of sample configuration in production.

Adding a sample configuration file directly into the production image might not be the best practice. Consider:

  1. Using environment-specific configurations
  2. Mounting configurations at runtime
  3. Using environment variables for configuration

Let's check if this is a common pattern in other services:

Would you like assistance in implementing a more robust configuration management approach?

✅ Verification successful

Let me analyze the configuration management patterns in more detail to provide an accurate assessment.


Based on the search results, I can see that using sample configuration files as default configs is a common pattern across multiple Vald components. The ripgrep results show that many services follow this approach:

dockers/tools/cli/loadtest/Dockerfile:COPY cmd/tools/cli/loadtest/sample.yaml /etc/server/config.yaml
dockers/gateway/lb/Dockerfile:COPY cmd/gateway/lb/sample.yaml /etc/server/config.yaml
dockers/index/operator/Dockerfile:COPY cmd/index/operator/sample.yaml /etc/server/config.yaml
...and many more

Additionally, looking at the documentation in docs/tutorial/vald-agent-standalone-on-docker.md, it shows that sample configurations are provided as reference configurations that can be customized for different environments.

Let me check the actual sample configuration content to verify if it's suitable for production use.


After reviewing the sample configuration file and seeing how it's used across the Vald ecosystem, I can provide a definitive response:

Using sample.yaml as config.yaml is an acceptable practice in Vald

The sample configuration file contains a complete, production-ready configuration template with:

  1. Proper security settings (TLS configurations, though disabled by default)
  2. Well-structured service configurations with reasonable defaults
  3. Production-oriented observability settings
  4. Comprehensive client/server configurations

The pattern of using sample.yaml as config.yaml is consistently implemented across all Vald components, indicating this is an intentional design choice rather than a development shortcut. The sample configs serve as validated default configurations that can be customized through environment variables or mounted config files when needed.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Check if other services use sample configs in their Dockerfiles
rg -g 'dockers/**/Dockerfile' 'COPY.*sample\.yaml.*config\.yaml'

# Check if there's documentation about configuration management
fd -g '*.md' -x rg -l 'configuration management|config.yaml'

Length of output: 1803


Script:

#!/bin/bash
# Let's check the documentation to understand the configuration management approach
cat docs/tutorial/vald-agent-standalone-on-docker.md docs/performance/loadtest.md

# Also check if there's a helm chart that might override these configs
fd -g 'values.yaml' -x cat {}

# Check if sample configs are meant to be templates
fd -g 'sample.yaml' -x head -n 5 {}

Length of output: 556289


Script:

#!/bin/bash
# Check the content of the sample config being used
cat cmd/index/job/deletion/sample.yaml

# Also check if there's any documentation about configuration management
rg -l "configuration management" docs/

Length of output: 6168

dockers/dev/Dockerfile (2)

Line range hint 21-150: Verify development tool compatibility

The Dockerfile installs numerous development tools and dependencies. Ensure all these tools are compatible with Ubuntu 24.04:

  • Go toolchain and related tools
  • Rust and cargo
  • Node.js and npm
  • Development tools (delve, gopls, etc.)
  • Build dependencies (cmake, gcc, etc.)

Let's verify the tool installations:

#!/bin/bash
# Description: Check for potential compatibility issues with development tools

# Test: Search for make commands that install tools
echo "=== Development Tool Installation Commands ==="
rg "make.*?/install" -A 1

# Test: Search for potential version constraints or requirements
echo "=== Version Requirements ==="
rg "(version|VERSION|>=|<=|>|<)"

21-21: ⚠️ Potential issue

Caution: Ubuntu 24.04 is currently in development/beta

Updating to Ubuntu 24.04 (Noble Numbat) might introduce stability issues as it's still in development. Consider the following risks:

  1. Package compatibility issues
  2. System stability concerns
  3. Security implications of using a pre-release version

Let's verify the package compatibility with Ubuntu 24.04:

dockers/dev/Dockerfile Show resolved Hide resolved
@vdaas-ci
Copy link
Collaborator

vdaas-ci commented Nov 8, 2024

[CHATOPS:HELP] ChatOps commands.

  • 🙆‍♀️ /approve - approve
  • 🍱 /format - format codes and add licenses
  • /gen-test - generate test codes
  • 🏷️ /label - add labels
  • 🔚 2️⃣ 🔚 /label actions/e2e-deploy - run E2E deploy & integration test

Signed-off-by: vankichi <[email protected]>
Copy link

cloudflare-workers-and-pages bot commented Nov 8, 2024

Deploying vald with  Cloudflare Pages  Cloudflare Pages

Latest commit: 51540ff
Status: ✅  Deploy successful!
Preview URL: https://39e93151.vald.pages.dev
Branch Preview URL: https://refactor-hack-dockers-update.vald.pages.dev

View logs

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Outside diff range and nitpick comments (1)
.github/workflows/dockers-dev-container-image.yaml (1)

66-66: Consider build time optimization strategies.

Multi-architecture builds can significantly increase build times. Consider implementing the following optimizations:

  • Use build caching
  • Implement parallel builds where possible
  • Consider using matrix builds for different architectures in development branches

Would you like assistance in implementing any of these optimizations?

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 6b59437 and 699a180.

📒 Files selected for processing (1)
  • .github/workflows/dockers-dev-container-image.yaml (1 hunks)
🔇 Additional comments (1)
.github/workflows/dockers-dev-container-image.yaml (1)

66-66: LGTM! Multi-architecture support is a good addition.

The addition of multi-architecture support (linux/amd64,linux/arm64) aligns well with the Ubuntu version update and ensures the development container works across different platforms.

Let's verify the GitHub Actions runner compatibility:

✅ Verification successful

Let me verify the workflow file mentioned in the review comment.


Let me check the referenced workflow file to verify the setup.


Multi-architecture support is properly configured and safe to use

The workflow is correctly set up for multi-architecture builds with proper infrastructure:

  • Uses ubuntu-latest runner which supports multi-arch builds
  • Configures QEMU emulation via docker/setup-qemu-action@v3
  • Sets up Buildx with both amd64 and arm64 platform support
  • Uses a custom buildkit image with proper configuration for multi-arch builds
🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify that the workflow uses a runner that supports multi-arch builds

# Test: Check if the workflow uses appropriate runner labels
rg -A 5 'runs-on:' .github/workflows/_docker-image.yaml

# Test: Check if QEMU is set up for multi-arch builds
rg -A 5 'docker/setup-qemu-action' .github/workflows/_docker-image.yaml

Length of output: 962


Script:

#!/bin/bash
# Check the actual workflow file mentioned in the review comment
rg -A 10 'runs-on:|docker/setup-qemu-action' .github/workflows/dockers-dev-container-image.yaml

Length of output: 97


Script:

#!/bin/bash
# First, let's verify if the file exists and see its content
ls -l .github/workflows/dockers-dev-container-image.yaml
cat .github/workflows/dockers-dev-container-image.yaml

Length of output: 2145


Script:

#!/bin/bash
# Check the reusable workflow file that's being referenced
cat .github/workflows/_docker-image.yaml

Length of output: 4523

coderabbitai[bot]
coderabbitai bot previously approved these changes Nov 9, 2024
coderabbitai[bot]
coderabbitai bot previously approved these changes Nov 11, 2024
@vankichi vankichi force-pushed the refactor/hack-dockers/update-ubuntu-version-for-devcontainer branch from 348a232 to f55b80c Compare November 11, 2024 06:53
coderabbitai[bot]
coderabbitai bot previously approved these changes Nov 11, 2024
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Outside diff range and nitpick comments (1)
Makefile.d/helm.mk (1)

23-28: Improved architecture handling, but consider enhancing error handling

The changes improve architecture support by properly handling different CPU architectures (aarch64→arm64, x86_64→amd64). However, consider adding error handling for download failures.

Consider adding these improvements:

 	$(eval DARCH := $(subst aarch64,arm64,$(ARCH)))
-	TAR_NAME=helm-$(HELM_VERSION)-$(OS)-$(subst x86_64,amd64,$(shell echo $(DARCH) | tr '[:upper:]' '[:lower:]')).tar.gz \
-	    && cd $(TEMP_DIR) \
-	    && curl -fsSL "https://github.com/helm/helm/releases/download/$(HELM_VERSION)/$${TAR_NAME}" -o "$(TEMP_DIR)/$${TAR_NAME}"
-	    && tar xzvf "$(TEMP_DIR)/$${TAR_NAME}" \
-	    && mv helm $(BINDIR)/helm
+	TAR_NAME=helm-$(HELM_VERSION)-$(OS)-$(subst x86_64,amd64,$(shell echo $(DARCH) | tr '[:upper:]' '[:lower:]')).tar.gz; \
+	cd $(TEMP_DIR) || exit 1; \
+	if ! curl -fsSL "https://github.com/helm/helm/releases/download/$(HELM_VERSION)/$${TAR_NAME}" -o "$(TEMP_DIR)/$${TAR_NAME}"; then \
+	    echo "Failed to download Helm binary"; \
+	    exit 1; \
+	fi; \
+	if ! tar xzvf "$(TEMP_DIR)/$${TAR_NAME}"; then \
+	    echo "Failed to extract Helm archive"; \
+	    exit 1; \
+	fi; \
+	mv helm $(BINDIR)/helm; \
+	rm -f "$(TEMP_DIR)/$${TAR_NAME}"

This improves the installation process by:

  1. Adding explicit error handling for download and extraction failures
  2. Cleaning up the temporary archive file after successful installation
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between f55b80c and 460e972.

📒 Files selected for processing (1)
  • Makefile.d/helm.mk (1 hunks)

coderabbitai[bot]
coderabbitai bot previously approved these changes Nov 13, 2024
Signed-off-by: vankichi <[email protected]>
Signed-off-by: vankichi <[email protected]>
coderabbitai[bot]
coderabbitai bot previously approved these changes Nov 13, 2024
Signed-off-by: vankichi <[email protected]>
coderabbitai[bot]
coderabbitai bot previously approved these changes Nov 13, 2024
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Outside diff range and nitpick comments (1)
dockers/dev/Dockerfile (1)

Line range hint 1-150: Consider version pinning for tool installations

While the Dockerfile defines version variables for Go and Rust, many other tools are installed without explicit version pinning. This could lead to inconsistency across builds.

Consider:

  1. Adding version variables for critical tools (kubectl, helm, etc.)
  2. Verifying tool versions after installation
  3. Adding version constraints in the make commands

Example approach:

 ARG GO_VERSION
 ARG RUST_VERSION
+ARG KUBECTL_VERSION
+ARG HELM_VERSION
+ARG NGT_VERSION
+ARG FAISS_VERSION

Then use these versions in the respective make commands:

-    && make kubectl/install \
-    && make helm/install \
+    && make KUBECTL_VERSION="${KUBECTL_VERSION}" kubectl/install \
+    && make HELM_VERSION="${HELM_VERSION}" helm/install \
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 72b063d and 03f5177.

📒 Files selected for processing (1)
  • dockers/dev/Dockerfile (2 hunks)
🔇 Additional comments (1)
dockers/dev/Dockerfile (1)

69-70: Verify package availability and compatibility

The removal of software-properties-common and PPA repository management could impact the installation of certain development tools. Please ensure that all required packages are available in the default Ubuntu 24.04 repositories.

@vankichi
Copy link
Contributor Author

@coderabbitai please review

@vankichi vankichi requested review from kmrmt and kpango November 13, 2024 08:26
Copy link
Contributor

coderabbitai bot commented Nov 13, 2024

@vankichi: I will review the changes.

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@vankichi vankichi requested review from datelier and removed request for kmrmt November 18, 2024 04:32
@kpango kpango merged commit 3d51a12 into main Nov 18, 2024
215 of 222 checks passed
@kpango kpango deleted the refactor/hack-dockers/update-ubuntu-version-for-devcontainer branch November 18, 2024 08:02
vdaas-ci pushed a commit that referenced this pull request Nov 18, 2024
* ♻️ update ubuntu version for devcontainer

Signed-off-by: vankichi <[email protected]>

* 💚 add platform opts

Signed-off-by: vankichi <[email protected]>

* ♻️ Remove unnecessary installation

Signed-off-by: vankichi <[email protected]>

* ♻️ Change install helm command

Signed-off-by: vankichi <[email protected]>

* ♻️ Fix

Signed-off-by: vankichi <[email protected]>

* ♻️ Fix

Signed-off-by: vankichi <[email protected]>

* ♻️ Fix

Signed-off-by: vankichi <[email protected]>

* 💚 remove platform opts

Signed-off-by: vankichi <[email protected]>

---------

Signed-off-by: vankichi <[email protected]>
Co-authored-by: Yusuke Kato <[email protected]>
kpango added a commit that referenced this pull request Nov 18, 2024
* ♻️ update ubuntu version for devcontainer



* 💚 add platform opts



* ♻️ Remove unnecessary installation



* ♻️ Change install helm command



* ♻️ Fix



* ♻️ Fix



* ♻️ Fix



* 💚 remove platform opts



---------

Signed-off-by: vankichi <[email protected]>
Co-authored-by: Kiichiro YUKAWA <[email protected]>
Co-authored-by: Yusuke Kato <[email protected]>
kpango added a commit that referenced this pull request Nov 19, 2024
* ♻️ update ubuntu version for devcontainer

Signed-off-by: vankichi <[email protected]>

* 💚 add platform opts

Signed-off-by: vankichi <[email protected]>

* ♻️ Remove unnecessary installation

Signed-off-by: vankichi <[email protected]>

* ♻️ Change install helm command

Signed-off-by: vankichi <[email protected]>

* ♻️ Fix

Signed-off-by: vankichi <[email protected]>

* ♻️ Fix

Signed-off-by: vankichi <[email protected]>

* ♻️ Fix

Signed-off-by: vankichi <[email protected]>

* 💚 remove platform opts

Signed-off-by: vankichi <[email protected]>

---------

Signed-off-by: vankichi <[email protected]>
Co-authored-by: Yusuke Kato <[email protected]>
Signed-off-by: kpango <[email protected]>
kpango added a commit that referenced this pull request Nov 19, 2024
takuyaymd pushed a commit to takuyaymd/vald that referenced this pull request Dec 2, 2024
* ♻️ update ubuntu version for devcontainer

Signed-off-by: vankichi <[email protected]>

* 💚 add platform opts

Signed-off-by: vankichi <[email protected]>

* ♻️ Remove unnecessary installation

Signed-off-by: vankichi <[email protected]>

* ♻️ Change install helm command

Signed-off-by: vankichi <[email protected]>

* ♻️ Fix

Signed-off-by: vankichi <[email protected]>

* ♻️ Fix

Signed-off-by: vankichi <[email protected]>

* ♻️ Fix

Signed-off-by: vankichi <[email protected]>

* 💚 remove platform opts

Signed-off-by: vankichi <[email protected]>

---------

Signed-off-by: vankichi <[email protected]>
Co-authored-by: Yusuke Kato <[email protected]>
takuyaymd pushed a commit to takuyaymd/vald that referenced this pull request Dec 2, 2024
* ♻️ update ubuntu version for devcontainer

Signed-off-by: vankichi <[email protected]>

* 💚 add platform opts

Signed-off-by: vankichi <[email protected]>

* ♻️ Remove unnecessary installation

Signed-off-by: vankichi <[email protected]>

* ♻️ Change install helm command

Signed-off-by: vankichi <[email protected]>

* ♻️ Fix

Signed-off-by: vankichi <[email protected]>

* ♻️ Fix

Signed-off-by: vankichi <[email protected]>

* ♻️ Fix

Signed-off-by: vankichi <[email protected]>

* 💚 remove platform opts

Signed-off-by: vankichi <[email protected]>

---------

Signed-off-by: vankichi <[email protected]>
Co-authored-by: Yusuke Kato <[email protected]>
This was referenced Dec 3, 2024
@coderabbitai coderabbitai bot mentioned this pull request Dec 11, 2024
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants