Skip to content

Commit

Permalink
Renamed to flaky test. (#4912)
Browse files Browse the repository at this point in the history
Signed-off-by: Daniel (dB.) Doubrovkine <[email protected]>

Signed-off-by: Daniel (dB.) Doubrovkine <[email protected]>
  • Loading branch information
dblock authored Oct 26, 2022
1 parent ae7a4e9 commit d96da50
Show file tree
Hide file tree
Showing 3 changed files with 10 additions and 10 deletions.
5 changes: 2 additions & 3 deletions .github/workflows/gradle-check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,5 @@ jobs:
* **RESULT:** ${{ env.result }} :x:
* **URL:** ${{ env.workflow_url }}
* **CommitID:** ${{ env.pr_from_sha }}
Please examine the workflow log, locate, and copy-paste the failure below.
[Flakey test?](https://github.com/opensearch-project/OpenSearch/blob/main/DEVELOPER_GUIDE.md#flakey-tests)
Iterate to green.
Please examine the workflow log, locate, and copy-paste the failure below, then iterate to green.
Is the failure [a flaky test](https://github.com/opensearch-project/OpenSearch/blob/main/DEVELOPER_GUIDE.md#flaky-tests) unrelated to your change?
5 changes: 3 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,15 +36,16 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
- Add icon for IntelliJidea toolbox ([#4882](https://github.com/opensearch-project/OpenSearch/pull/4882))
- Add groupId value propagation tests for ZIP publication task ([#4772](https://github.com/opensearch-project/OpenSearch/pull/4772))
- Add support for GeoJson Point type in GeoPoint field ([#4597](https://github.com/opensearch-project/OpenSearch/pull/4597))
- Add dev guide for dealing with flakey tests ([4868](https://github.com/opensearch-project/OpenSearch/pull/4868))
- Add dev guide for dealing with flaky tests ([4868](https://github.com/opensearch-project/OpenSearch/pull/4868))
- Update pull request template ([#4851](https://github.com/opensearch-project/OpenSearch/pull/4851))
- Added missing no-jdk distributions ([#4722](https://github.com/opensearch-project/OpenSearch/pull/4722))
- Add dev help in gradle check CI failures ([4872](https://github.com/opensearch-project/OpenSearch/pull/4872))
- Copy `build.sh` over from opensearch-build ([#4887](https://github.com/opensearch-project/OpenSearch/pull/4887))
- Add project health badges to the README.md ([#4843](https://github.com/opensearch-project/OpenSearch/pull/4843))
- Added changes for graceful node decommission ([#4586](https://github.com/opensearch-project/OpenSearch/pull/4586))
- Build no-jdk distributions as part of release build ([#4902](https://github.com/opensearch-project/OpenSearch/pull/4902))

- Renamed flaky tests ([#4912](https://github.com/opensearch-project/OpenSearch/pull/4912))

### Dependencies
- Bumps `log4j-core` from 2.18.0 to 2.19.0
- Bumps `reactor-netty-http` from 1.0.18 to 1.0.23
Expand Down
10 changes: 5 additions & 5 deletions DEVELOPER_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@
- [Backports](#backports)
- [LineLint](#linelint)
- [Lucene Snapshots](#lucene-snapshots)
- [Flakey Tests](#flakey-tests)
- [Flaky Tests](#flaky-tests)

# Developer Guide

Expand Down Expand Up @@ -543,15 +543,15 @@ Pass a list of files or directories to limit your search.
The Github workflow in [lucene-snapshots.yml](.github/workflows/lucene-snapshots.yml) is a Github worfklow executable by maintainers to build a top-down snapshot build of lucene.
These snapshots are available to test compatibility with upcoming changes to Lucene by updating the version at [version.properties](buildsrc/version.properties) with the `version-snapshot-sha` version. Example: `lucene = 10.0.0-snapshot-2e941fc`.

### Flakey Tests
### Flaky Tests

OpenSearch has a very large test suite with long running, often failing (flakey), integration tests. Such individual tests are labelled as [Flakey Random Test Failure](https://github.com/opensearch-project/OpenSearch/issues?q=is%3Aopen+is%3Aissue+label%3A%22Flakey+Random+Test+Failure%22). Your help is wanted fixing these!
OpenSearch has a very large test suite with long running, often failing (flaky), integration tests. Such individual tests are labelled as [Flaky Random Test Failure](https://github.com/opensearch-project/OpenSearch/issues?q=is%3Aopen+is%3Aissue+label%3A%22flaky-test%22). Your help is wanted fixing these!

If you encounter a build/test failure in CI that is unrelated to the change in your pull request, it may be a known flakey test, or a new test failure.
If you encounter a build/test failure in CI that is unrelated to the change in your pull request, it may be a known flaky test, or a new test failure.

1. Follow failed CI links, and locate the failing test(s).
2. Copy-paste the failure into a comment of your PR.
3. Search through [issues](https://github.com/opensearch-project/OpenSearch/issues?q=is%3Aopen+is%3Aissue+label%3A%22Flakey+Random+Test+Failure%22) using the name of the failed test for whether this is a known flakey test.
3. Search through [issues](https://github.com/opensearch-project/OpenSearch/issues?q=is%3Aopen+is%3Aissue+label%3A%22flaky-test%22) using the name of the failed test for whether this is a known flaky test.
5. If an existing issue is found, paste a link to the known issue in a comment to your PR.
6. If no existing issue is found, open one.
7. Retry CI via the GitHub UX or by pushing an update to your PR.

0 comments on commit d96da50

Please sign in to comment.