Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI] Cypress test run spec input #5142

Merged

Conversation

kavilla
Copy link
Member

@kavilla kavilla commented Sep 28, 2023

Description

Originally:
#5134

I believed it to be a good idea to ensure that triggering the CI should always run a batch of tests. But I realized that some tests can be added and just be bad tests until updated. Or if we want to verify a flaky test in the CI.

So instead of only making it additional tests, the spec input will complete replace the default tests (unless you append to the default).

Empty will run the default spec still. This will append on to the PR comment with the spec that was run. So at least maintainers can see it passed for a set of tests but it did not run the ones we usually run.

Also adding some formatting for empty values for the PR comment.

Issues Related

#4019

Check List

  • All tests pass
    • yarn test:jest
    • yarn test:jest_integration
    • yarn test:ftr
  • New functionality includes testing.
  • New functionality has been documented.
  • Update CHANGELOG.md
  • Commits are signed per the DCO using --signoff

Originally:
opensearch-project#5134

I believed it to be a good idea to ensure that triggering the CI
should always run a batch of tests. But I realized that some tests
can be added and just be bad tests until updated. Or if we want
to verify a flaky test in the CI.

So instead of only making it additional tests, the spec input will
complete replace the default tests (unless you append to the default).

Empty will run the default spec still. This will append on to the
PR comment with the spec that was run. So at least maintainers can
see it passed for a set of tests but it did not run the ones
we usually run.

Also adding some formatting for empty values for the PR comment.

Issue related:
opensearch-project#4019

Signed-off-by: Kawika Avilla <[email protected]>
@kavilla
Copy link
Member Author

kavilla commented Sep 28, 2023

Example: #5134 (comment)

I thought it was having issues but turns out there was just a lot of tests. Which might need to consider CI groups for this workflow.

@codecov
Copy link

codecov bot commented Sep 28, 2023

Codecov Report

Merging #5142 (f9dd558) into main (34a8594) will increase coverage by 0.00%.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main    #5142   +/-   ##
=======================================
  Coverage   66.50%   66.50%           
=======================================
  Files        3403     3403           
  Lines       65026    65026           
  Branches    10401    10401           
=======================================
+ Hits        43243    43245    +2     
- Misses      19214    19219    +5     
+ Partials     2569     2562    -7     
Flag Coverage Δ
Linux_1 34.82% <ø> (ø)
Linux_2 55.31% <ø> (ø)
Linux_3 44.61% <ø> (+<0.01%) ⬆️
Linux_4 34.89% <ø> (ø)
Windows_1 34.83% <ø> (ø)
Windows_2 55.27% <ø> (ø)
Windows_3 44.61% <ø> (ø)
Windows_4 34.89% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

see 6 files with indirect coverage changes

required: false
type: string

env:
SOURCE_REPO: ${{ github.repository }}
SOURCE_BRANCH: ${{ github.base_ref }}
SOURCE_BRANCH: "${{ github.base_ref }}"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit - it looks like prefer single quotes elsewhere in this yml file?

@kavilla kavilla merged commit 6154e2d into opensearch-project:main Sep 29, 2023
65 of 82 checks passed
@opensearch-trigger-bot
Copy link
Contributor

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch-Dashboards/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch-Dashboards/backport-2.x
# Create a new branch
git switch --create backport/backport-5142-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 6154e2d3c1cd96e64bfcae7ec14cffcf54c6805b
# Push it to GitHub
git push --set-upstream origin backport/backport-5142-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch-Dashboards/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-5142-to-2.x.

opensearch-trigger-bot bot pushed a commit that referenced this pull request Oct 2, 2023
Originally:
#5134

I believed it to be a good idea to ensure that triggering the CI
should always run a batch of tests. But I realized that some tests
can be added and just be bad tests until updated. Or if we want
to verify a flaky test in the CI.

So instead of only making it additional tests, the spec input will
complete replace the default tests (unless you append to the default).

Empty will run the default spec still. This will append on to the
PR comment with the spec that was run. So at least maintainers can
see it passed for a set of tests but it did not run the ones
we usually run.

Also adding some formatting for empty values for the PR comment.

Issue related:
#4019

Signed-off-by: Kawika Avilla <[email protected]>
(cherry picked from commit 6154e2d)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
kavilla pushed a commit that referenced this pull request Oct 2, 2023
Originally:
#5134

I believed it to be a good idea to ensure that triggering the CI
should always run a batch of tests. But I realized that some tests
can be added and just be bad tests until updated. Or if we want
to verify a flaky test in the CI.

So instead of only making it additional tests, the spec input will
complete replace the default tests (unless you append to the default).

Empty will run the default spec still. This will append on to the
PR comment with the spec that was run. So at least maintainers can
see it passed for a set of tests but it did not run the ones
we usually run.

Also adding some formatting for empty values for the PR comment.

Issue related:
#4019


(cherry picked from commit 6154e2d)

Signed-off-by: Kawika Avilla <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x ci Skip-Changelog PRs that are too trivial to warrant a changelog or release notes entry
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants