Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: add e2e consensus test #3126

Merged
merged 4 commits into from
Nov 7, 2024
Merged

test: add e2e consensus test #3126

merged 4 commits into from
Nov 7, 2024

Conversation

gartnera
Copy link
Member

@gartnera gartnera commented Nov 7, 2024

Description

Add start-e2e-consensus-test which starts up localnet with the develop version of zetacored pushed in #3121.

Opt in on your PR with the CONSENSUS_TESTS label. This will generally require your PR to be up to date with develop.

To acknowledge a breaking change, you must add the CONSENSUS_BREAKING_ACK label to your PR. If your PR is consensus breaking, you should also have an explanation point in your PR title (feat!, fix!, etc).

If CONSENSUS_BREAKING_ACK is not applied to your PR, the consensus tests will be run on the mergequeue automatically. The mergequeue is the best place to run this test as the head branch is guaranteed to be one commit above develop (at least with our current config).

e2e will still point at zetacore0 so any rpc only changes should not result in failure.

Also add block production monitor to zetae2e local which will halt the e2e tests quickly if blocks are not being produced.

Demo in #3129

Closes #3120

How Has This Been Tested?

  • make start-e2e-consensus-test on arm64 macos

Summary by CodeRabbit

  • New Features

    • Introduced a new end-to-end consensus test that can be initiated via a new Makefile target.
    • Enhanced the GitHub Actions workflow to conditionally execute tests based on pull request labels and merge events.
  • Improvements

    • Updated the zetacore1 service in the Docker Compose configuration for improved configurability with dynamic image and platform settings.
  • Bug Fixes

    • Minor formatting adjustments made in the Makefile for clarity.

@gartnera gartnera added the no-changelog Skip changelog CI check label Nov 7, 2024
Copy link
Contributor

coderabbitai bot commented Nov 7, 2024

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

📝 Walkthrough
📝 Walkthrough
📝 Walkthrough
📝 Walkthrough
📝 Walkthrough

Walkthrough

The pull request introduces enhancements to the GitHub Actions workflow for end-to-end testing, specifically in the .github/workflows/e2e.yml file. A new output variable, CONSENSUS_TESTS, is added to the matrix-conditionals job, allowing for dynamic test execution based on pull request labels. The Makefile is updated with a new target for initiating consensus tests, while the docker-compose.yml file is modified to use environment variables for the zetacore1 service image and platform, improving configurability.

Changes

File Change Summary
.github/workflows/e2e.yml Added output variable CONSENSUS_TESTS and modified logic for pull_request and merge_group events. Introduced helper function getPrLabels. Updated e2e job with a new matrix entry for start-e2e-consensus-test.
Makefile Introduced new target start-e2e-consensus-test and made minor formatting adjustments to existing targets.
contrib/localnet/docker-compose.yml Changed zetacore1 service image to use an environment variable and added a platform field.

Assessment against linked issues

Objective Addressed Explanation
Add consensus breaking e2e test (#3120)
Build and push zetacored docker images for develop No changes related to building or pushing images.
Add a docker overlay for develop version Unclear if overlay changes are included in the PR.

Possibly related PRs

Suggested labels

UPGRADE_TESTS

Suggested reviewers

  • skosito
  • lumtis
  • kingpinXD
  • fbac
  • swift1337

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@github-actions github-actions bot added the ci Changes to CI pipeline or github actions label Nov 7, 2024
@gartnera gartnera added the CONSENSUS_TESTS Run make start-e2e-consensus-tests label Nov 7, 2024
@gartnera gartnera marked this pull request as ready for review November 7, 2024 17:25
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Outside diff range and nitpick comments (3)
contrib/localnet/docker-compose.yml (1)

71-71: Consider documenting platform configuration

The platform configuration enables cross-architecture testing, which is valuable for the reported arm64 macOS testing scenario. However, consider documenting the supported platforms in the file's header comments to assist other developers.

Add documentation like:

 # - ZetaChain with 2 nodes (zetacore0, zetacore1). When profile set to stress, 4 nodes will be created.
+# Note: zetacore1 supports custom platform configuration via ZETACORE1_PLATFORM for cross-architecture testing
.github/workflows/e2e.yml (1)

132-141: Fix indentation in getPrLabels function

The function implementation is solid, but the indentation is inconsistent with the rest of the file.

Apply this diff to fix the indentation:

 const getPrLabels = async (pull_number) => {
   const { data: pr } = await github.rest.pulls.get({
-    owner: context.repo.owner,
-    repo: context.repo.repo,
-    pull_number: pull_number,
-  });
-  const labels = pr.labels.map(label => label.name);
-  console.log(`labels for ${pull_number}:`, labels);
-  return labels;
+  owner: context.repo.owner,
+  repo: context.repo.repo,
+  pull_number: pull_number,
+});
+const labels = pr.labels.map(label => label.name);
+console.log(`labels for ${pull_number}:`, labels);
+return labels;
 }
Makefile (1)

289-294: Consider adding documentation for the consensus test target.

The implementation looks correct, but it would be beneficial to add a comment explaining:

  • The purpose of using the development version of zetacore
  • The significance of the platform specification
  • Any prerequisites or requirements for running the test

Apply this diff to add documentation:

+# start-e2e-consensus-test initiates a local network using the development version of zetacored.
+# This target is designed for testing consensus-breaking changes and requires specific PR labels:
+# - CONSENSUS_TESTS: Required to opt-in for consensus tests
+# - CONSENSUS_BREAKING_ACK: Required if there are breaking changes
 start-e2e-consensus-test: e2e-images
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 7c475be and f30e24e.

📒 Files selected for processing (3)
  • .github/workflows/e2e.yml (4 hunks)
  • Makefile (2 hunks)
  • contrib/localnet/docker-compose.yml (1 hunks)
🔇 Additional comments (6)
contrib/localnet/docker-compose.yml (2)

70-70: LGTM: Dynamic image configuration enables consensus testing

The environment variable ${ZETACORE1_IMAGE-zetanode:latest} with fallback aligns with the PR's objective to support consensus testing with development versions of zetacored.


70-71: Verify configuration consistency with orchestrator

The changes enable dynamic configuration of zetacore1, but we should verify that the orchestrator service properly handles these configurations during consensus tests.

.github/workflows/e2e.yml (2)

114-114: LGTM: Output variable properly declared

The CONSENSUS_TESTS output variable follows the established pattern and is correctly integrated with other test flags.


228-230: LGTM: Matrix configuration properly structured

The consensus test matrix entry is well-configured, following the established pattern and using appropriate runner specifications.

Makefile (2)

271-271: LGTM!

The formatting change improves consistency with other similar commands in the Makefile.


291-292: Verify the image and platform compatibility.

The target uses a hardcoded platform and image. Let's verify that these values are consistently used across the codebase.

✅ Verification successful

Platform and image specifications are consistent with the development environment

The verification reveals that:

  • The ZETACORE1_IMAGE and ZETACORE1_PLATFORM variables are used correctly in the docker-compose setup with appropriate fallback values
  • The platform specification linux/amd64 is consistently used across various services that require explicit platform definition
  • The image versioning follows a consistent pattern using either latest, develop, or old tags depending on the context

The hardcoded values in the Makefile align with the broader development environment configuration and are appropriate for their intended use.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Check for other occurrences of the development image and platform specifications

# Test 1: Search for other references to the development image
echo "Checking for development image references:"
rg "ghcr.io/zeta-chain/zetanode:develop"

# Test 2: Search for platform specifications
echo "Checking for platform specifications:"
rg "ZETACORE.*_PLATFORM"

Length of output: 447


Script:

#!/bin/bash
# Let's check the docker-compose file to understand the context better
echo "Checking docker-compose.yml content:"
cat contrib/localnet/docker-compose.yml

# Let's also check for any other docker platform specifications
echo -e "\nChecking for other platform specifications:"
rg "platform.*linux"

# Check for other image references in docker compose
echo -e "\nChecking for other image references in docker compose:"
rg "image:.*zetanode"

Length of output: 13349

.github/workflows/e2e.yml Outdated Show resolved Hide resolved
Copy link
Member

@lumtis lumtis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would make sense to add the description in the PR in docs/development/RELEASES.md

Makefile Show resolved Hide resolved
@gartnera
Copy link
Member Author

gartnera commented Nov 7, 2024

Would make sense to add the description in the PR in docs/development/RELEASES.md

This check only works on develop PRs atm. I'd need to push release docker images and turn on mergequeue for release PRs for it to work there.

Copy link
Member

@CharlieMc0 CharlieMc0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@gartnera gartnera added this pull request to the merge queue Nov 7, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Nov 7, 2024
@gartnera gartnera added this pull request to the merge queue Nov 7, 2024
Merged via the queue into develop with commit 0e86494 Nov 7, 2024
43 checks passed
@gartnera gartnera deleted the e2e-consensus-test branch November 7, 2024 23:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking:cli ci Changes to CI pipeline or github actions CONSENSUS_TESTS Run make start-e2e-consensus-tests no-changelog Skip changelog CI check
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ci: add consensus breaking e2e test
4 participants