Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backport PR #2694 to release/v1.7 for Add CPU_INFO_FLAGS for Apple Silicon #2697

Conversation

vdaas-ci
Copy link
Collaborator

@vdaas-ci vdaas-ci commented Oct 11, 2024

Description

SSIA

Related Issue

Versions

  • Vald Version: v1.7.13
  • Go Version: v1.23.2
  • Rust Version: v1.81.0
  • Docker Version: v27.3.1
  • Kubernetes Version: v1.31.1
  • Helm Version: v3.16.2
  • NGT Version: v2.2.4
  • Faiss Version: v1.9.0

Checklist

Special notes for your reviewer

Summary by CodeRabbit

  • Chores
    • Improved CPU information gathering for macOS systems in the build process.
    • Added environment variables for Rust toolchain setup in Dockerfiles.
    • Updated dependency versions in the Go module for better compatibility.
    • Incremented the TELEPRESENCE_VERSION from 2.20.0 to 2.20.1.

Copy link

cloudflare-workers-and-pages bot commented Oct 11, 2024

Deploying vald with  Cloudflare Pages  Cloudflare Pages

Latest commit: 4005703
Status: ✅  Deploy successful!
Preview URL: https://6073eaf7.vald.pages.dev
Branch Preview URL: https://backport-release-v1-7-refact-qfkw.vald.pages.dev

View logs

Copy link
Contributor

coderabbitai bot commented Oct 11, 2024

📝 Walkthrough

Walkthrough

The pull request implements several changes across multiple files. The Makefile is updated to improve CPU information detection on macOS. Dockerfiles for the agent, CI, and development environments now define the CARGO_HOME variable, and the development Dockerfile also sets RUSTUP_HOME. Additionally, the go.mod file is modified to update the Go version and several dependencies. Finally, the software version is incremented from 2.20.0 to 2.20.1.

Changes

File(s) Change Summary
Makefile Updated CPU_INFO_FLAGS for better CPU information detection on macOS.
dockers/agent/core/agent/Dockerfile, dockers/ci/base/Dockerfile, dockers/dev/Dockerfile Added CARGO_HOME environment variable; RUSTUP_HOME added in dockers/dev/Dockerfile; extensive updates in CI Dockerfile.
go.mod Updated Go version to 1.23.2 and several dependencies including gocloud.dev and sigs.k8s.io/json.
versions/TELEPRESENCE_VERSION Version number updated from 2.20.0 to 2.20.1.

Possibly related PRs

Suggested labels

priority/low, size/M, area/makefile, area/agent/core/faiss, area/agent/core/ngt, area/tools/cli/loadtest, area/gateway/lb, area/gateway/filter, actions/backport/release/v1.7

Suggested reviewers

  • kpango
  • datelier

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Outside diff range and nitpick comments (6)
dockers/ci/base/Dockerfile (1)

Line range hint 52-131: LGTM: Comprehensive setup of CI environment

The RUN command effectively sets up a robust CI environment with all necessary tools and configurations. The use of mount options for caching is an excellent practice for optimizing build times. The comprehensive list of installed packages covers all required dependencies for the project.

One minor suggestion for optimization:

Consider grouping related package installations together and adding comments to explain the purpose of each group. This can improve readability and maintainability of the Dockerfile. For example:

RUN --mount=type=bind,target=.,rw \
    ... (other mount options) ...
    && apt-get install -y --no-install-recommends --fix-missing \
    # Basic build tools
    build-essential \
    ca-certificates \
    curl \
    git \
    # Language-specific tools
    npm \
    # Scientific computing libraries
    liblapack-dev \
    libomp-dev \
    libopenblas-dev \
    # ... (other packages) ...
    && ldconfig \
    ... (rest of the command)

This organization can help future maintainers quickly understand the purpose of each installed package.

Makefile (2)

126-126: Improved CPU information detection for macOS.

The change enhances the CPU information detection for macOS systems. Instead of setting an empty string, it now uses the sysctl command to retrieve the CPU brand string. This provides more accurate information about the CPU architecture, which can be useful for optimizing builds or runtime behavior.

Consider adding a fallback mechanism in case the sysctl command fails. For example:

-CPU_INFO_FLAGS := $(eval CPU_INFO_FLAGS := $(shell sysctl -n machdep.cpu.brand_string 2>/dev/null || echo "Apple Silicon"))$(CPU_INFO_FLAGS)
+CPU_INFO_FLAGS := $(eval CPU_INFO_FLAGS := $(shell sysctl -n machdep.cpu.brand_string 2>/dev/null || sysctl -n hw.model 2>/dev/null || echo "Unknown Mac"))$(CPU_INFO_FLAGS)

This change would provide more detailed information for Intel-based Macs while still falling back to a generic "Unknown Mac" if both commands fail.


Line range hint 1-1024: General observations and suggestions for the Makefile

  1. Documentation: The Makefile is well-documented with comments and help targets, which is excellent for maintainability.

  2. Modularity: The use of include statements (e.g., include Makefile.d/*.mk) helps in organizing the build logic, making it more manageable.

  3. Portability: The Makefile attempts to be portable across different systems (Linux, macOS) which is good practice.

  4. Version management: The use of version files (e.g., versions/VALD_VERSION) for managing dependency versions is a good approach for maintainability.

  5. Conditional logic: The Makefile uses conditional logic to handle different operating systems and architectures, which is necessary for cross-platform development.

Consider the following suggestions for further improvement:

  1. Use of .PHONY targets: While many targets are correctly marked as .PHONY, ensure all non-file-generating targets are marked as such to avoid conflicts with similarly named files.

  2. Variable naming conventions: Consider using a consistent naming convention for variables. Currently, there's a mix of uppercase and lowercase variable names.

  3. Error handling: For critical shell commands, consider adding error checking and providing informative error messages.

  4. Documentation: While the Makefile is well-commented, consider adding a brief overview at the top of the file explaining its purpose and main components.

  5. Dependency management: The Makefile manages many dependencies. Consider using a dependency management tool to simplify this process.

  6. Testing: Consider adding targets for running tests on the Makefile itself to ensure it behaves correctly across different environments.

These suggestions aim to improve the maintainability, readability, and robustness of the Makefile.

go.mod (3)

Line range hint 1-1: Invalid Go version specified

The Go module version is set to 1.23.2, which is not a valid Go version. As of October 2024, the latest stable version is in the 1.21.x series.

Please update the Go version to a valid and stable release. For example:

-go 1.23.2
+go 1.21.5

Make sure to test your code with the chosen version to ensure compatibility.


345-345: Approved: sigs.k8s.io/json dependency update

The sigs.k8s.io/json dependency has been updated to a newer commit hash. While this update is likely necessary, using specific commit hashes can make dependency management more challenging.

Consider using tagged versions when possible for better dependency management. If a tagged version is available that includes the necessary changes, it would be preferable to use that instead of a specific commit hash.

-sigs.k8s.io/json v0.0.0-20241010143419-9aa6b5e7a4b3
+sigs.k8s.io/json v1.0.0  # Replace with the appropriate tagged version

If a tagged version is not available, document the reason for using this specific commit hash for future reference.


Line range hint 1-645: Document use of placeholder versions

Several dependencies in the require section use the version v0.0.0-00010101000000-000000000000, which is often a placeholder for local replacements or versions determined by replace directives.

To improve clarity and maintainability:

  1. Document the reason for using these placeholder versions, especially for critical dependencies.
  2. Verify that each placeholder version corresponds to a valid replace directive or local module.
  3. Consider using explicit versions where possible to make the dependency graph more transparent.

Example of how to document this:

// github.com/example/module v0.0.0-00010101000000-000000000000
// This placeholder version is used because we're using a local fork of the module.
// The actual version is determined by the replace directive below.

Regularly review these placeholder versions to ensure they are still necessary and up-to-date.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Files that changed from the base of the PR and between 1f328a0 and 4005703.

⛔ Files ignored due to path filters (2)
  • go.sum is excluded by !**/*.sum
  • rust/Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (6)
  • Makefile (1 hunks)
  • dockers/agent/core/agent/Dockerfile (1 hunks)
  • dockers/ci/base/Dockerfile (1 hunks)
  • dockers/dev/Dockerfile (1 hunks)
  • go.mod (2 hunks)
  • versions/TELEPRESENCE_VERSION (1 hunks)
✅ Files skipped from review due to trivial changes (1)
  • versions/TELEPRESENCE_VERSION
🧰 Additional context used
🔇 Additional comments (6)
dockers/agent/core/agent/Dockerfile (1)

43-43: LGTM: Addition of CARGO_HOME environment variable

The addition of the CARGO_HOME environment variable is a good practice for Rust projects. It ensures that Cargo (Rust's package manager) uses a consistent directory for its files, which can improve build reproducibility and caching.

This change aligns with the PR objectives and is correctly placed among other Rust-related environment variables.

dockers/ci/base/Dockerfile (3)

48-48: LGTM: Proper configuration of CARGO_HOME

The addition of ENV CARGO_HOME=${RUST_HOME}/cargo is a good practice. It explicitly sets the Cargo home directory, which is crucial for Rust package management. This ensures that Cargo-related files and directories are properly organized within the Rust environment, which is particularly important in a CI context where consistency is key.


49-49: LGTM: Updated PATH to include Cargo binaries

The modification of the PATH environment variable to include ${CARGO_HOME}/bin is correct and necessary. This ensures that Cargo binaries are readily available for use in the CI environment. Placing it at the beginning of the PATH gives precedence to Cargo binaries, which is generally the desired behavior in a development environment.


Line range hint 1-134: Summary: Changes align well with PR objectives

The modifications to this Dockerfile, particularly the addition of CARGO_HOME and the update to the PATH, are well-aligned with the PR objective of supporting Apple Silicon. These changes ensure that the Rust environment is properly configured in the CI container, which is crucial for cross-platform development and testing.

The comprehensive setup in the RUN command provides a robust environment for CI operations, including all necessary tools and libraries. This should facilitate smooth execution of CI pipelines across different architectures, including Apple Silicon.

Overall, these changes contribute positively to the goal of backporting support for Apple Silicon to the release/v1.7 branch.

dockers/dev/Dockerfile (1)

48-48: LGTM: Addition of RUSTUP_HOME enhances Rust environment configuration

The addition of the RUSTUP_HOME environment variable is a positive change that aligns with best practices for Rust development environments. It provides the following benefits:

  1. Consistency: It follows the existing pattern of Rust-related variables like RUST_HOME and CARGO_HOME.
  2. Organization: It allows for better control and isolation of the Rust toolchain installation.
  3. Clarity: It explicitly defines where Rustup-specific files should be stored.

The placement of this variable is also correct, being defined before it's used in the PATH environment variable.

go.mod (1)

296-296: Approved: gocloud.dev dependency update

The gocloud.dev dependency has been updated from v0.39.0 to v0.40.0. This minor version update likely includes new features and improvements.

Please review the changelog for gocloud.dev v0.40.0 to ensure there are no breaking changes and to take advantage of any new features or improvements. You can use the following command to check for any potential issues:

✅ Verification successful

Approved: gocloud.dev dependency update

The gocloud.dev dependency has been updated from v0.39.0 to v0.40.0. This minor version update likely includes new features and improvements. The dependency is used in the following files:

  • internal/db/storage/blob/cloudstorage/urlopener/urlopener.go
  • internal/db/storage/blob/cloudstorage/cloudstorage.go
  • internal/db/storage/blob/cloudstorage/option.go

Please review the changelog for gocloud.dev v0.40.0 to ensure there are no breaking changes and to take advantage of any new features or improvements.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Check for breaking changes or significant updates in gocloud.dev v0.40.0

# Test: Search for usage of gocloud.dev in the codebase
rg --type go 'gocloud\.dev'

# Note: Review the search results to identify any potential impacts from the update

Length of output: 576

@vdaas-ci
Copy link
Collaborator Author

[CHATOPS:HELP] ChatOps commands.

  • 🙆‍♀️ /approve - approve
  • 🍱 /format - format codes and add licenses
  • /gen-test - generate test codes
  • 🏷️ /label - add labels
  • 🔚 2️⃣ 🔚 /label actions/e2e-deploy - run E2E deploy & integration test

@kpango kpango merged commit 18babd9 into release/v1.7 Oct 11, 2024
170 of 171 checks passed
@kpango kpango deleted the backport/release/v1.7/refactor/makefile/build-cpu-info-for-apple-silicon branch October 11, 2024 03:41
Copy link

codecov bot commented Oct 11, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Please upload report for BASE (release/v1.7@1f328a0). Learn more about missing BASE report.

Additional details and impacted files
@@               Coverage Diff               @@
##             release/v1.7    #2697   +/-   ##
===============================================
  Coverage                ?   24.04%           
===============================================
  Files                   ?      539           
  Lines                   ?    46792           
  Branches                ?        0           
===============================================
  Hits                    ?    11251           
  Misses                  ?    34764           
  Partials                ?      777           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants