[Cases] Fix flaky functional tests #181230
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cnasikas
added
release_note:skip
This was
linked to
issues
Apr 19, 2024
cnasikas
commented
Apr 19, 2024
| @@ -179,10 +179,6 @@ export function CasesTableServiceProvider( | |||
| await testSubjects.click(`options-filter-popover-item-${status}`); | |||
| // to close the popup | |||
| await testSubjects.click('options-filter-popover-button-status'); | |||
|
|
|||
| await testSubjects.missingOrFail(`options-filter-popover-item-${status}`, { | |||
There was a problem hiding this comment.
Potentially flaky code. Either way, it is not needed.
|
Pinging @elastic/response-ops (Team:ResponseOps) |
|
Pinging @elastic/response-ops-cases (Feature:Cases) |
|
@elasticmachine merge upstream |
💚 Build Succeeded
Metrics [docs]Async chunks
To update your PR or re-run it, just comment with: cc @cnasikas |
adcoelho
approved these changes
Apr 22, 2024
| } | ||
|
|
||
| await retry.waitFor(`filterByOwner popover opened`, async () => { |
There was a problem hiding this comment.
super nit:
Suggested change
| await retry.waitFor(`filterByOwner popover opened`, async () => { | |
| await retry.waitFor('filterByOwner popover opened', async () => { |
| @@ -753,8 +752,7 @@ export default ({ getPageObject, getService }: FtrProviderContext) => { | |||
| expect(await testSubjects.exists('all-cases-clear-filters-link-icon')).to.be(false); | |||
| }); | |||
|
|
|||
| // FLAKY: https://github.com/elastic/kibana/issues/176716 | |||
| describe.skip('assignees filtering', () => { | |||
| describe('assignees filtering', () => { | |||
There was a problem hiding this comment.
What about this one? None of the changes in the CasesTableServiceProvider seem to be used here.
There was a problem hiding this comment.
Yes, I am not sure why it was flaky. I ran the flaky test runner a lot and I did not see any flakyness. I suspect that it was a one time thing issue with the CI.
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Apr 22, 2024
## Summary I ran the flaky test runner multiple times. I did not find any good reason why some of the tests were flaky in the first place. Flaky test runner: https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5720, https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5721, https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5727 Fixes: elastic#179016, elastic#178991, elastic#176716, elastic#178690 ### Checklist Delete any items that are not applicable to this PR. - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed ### For maintainers - [x] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) --------- Co-authored-by: Kibana Machine <[email protected]> (cherry picked from commit 8a99299)
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
kibanamachine
added a commit
that referenced
this pull request
Apr 22, 2024
# Backport This will backport the following commits from `main` to `8.14`: - [[Cases] Fix flaky functional tests (#181230)](#181230) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Christos Nasikas","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-04-22T10:44:19Z","message":"[Cases] Fix flaky functional tests (#181230)\n\n## Summary\r\n\r\nI ran the flaky test runner multiple times. I did not find any good\r\nreason why some of the tests were flaky in the first place.\r\n\r\nFlaky test runner:\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5720,\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5721,\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5727\r\n\r\nFixes: https://github.com/elastic/kibana/issues/179016,\r\nhttps://github.com/elastic/kibana/issues/178991,\r\nhttps://github.com/elastic/kibana/issues/176716,\r\nhttps://github.com/elastic/kibana/issues/178690\r\n\r\n### Checklist\r\n\r\nDelete any items that are not applicable to this PR.\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n- [x] [Flaky Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\r\nused on any tests changed\r\n\r\n### For maintainers\r\n\r\n- [x] This was checked for breaking API changes and was [labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\r\n\r\n---------\r\n\r\nCo-authored-by: Kibana Machine <[email protected]>","sha":"8a992997e9f942b39182a087df44604dc69957cf","branchLabelMapping":{"^v8.15.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:ResponseOps","Feature:Cases","v8.14.0","v8.15.0"],"title":"[Cases] Fix flaky functional tests","number":181230,"url":"https://github.com/elastic/kibana/pull/181230","mergeCommit":{"message":"[Cases] Fix flaky functional tests (#181230)\n\n## Summary\r\n\r\nI ran the flaky test runner multiple times. I did not find any good\r\nreason why some of the tests were flaky in the first place.\r\n\r\nFlaky test runner:\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5720,\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5721,\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5727\r\n\r\nFixes: https://github.com/elastic/kibana/issues/179016,\r\nhttps://github.com/elastic/kibana/issues/178991,\r\nhttps://github.com/elastic/kibana/issues/176716,\r\nhttps://github.com/elastic/kibana/issues/178690\r\n\r\n### Checklist\r\n\r\nDelete any items that are not applicable to this PR.\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n- [x] [Flaky Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\r\nused on any tests changed\r\n\r\n### For maintainers\r\n\r\n- [x] This was checked for breaking API changes and was [labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\r\n\r\n---------\r\n\r\nCo-authored-by: Kibana Machine <[email protected]>","sha":"8a992997e9f942b39182a087df44604dc69957cf"}},"sourceBranch":"main","suggestedTargetBranches":["8.14"],"targetPullRequestStates":[{"branch":"8.14","label":"v8.14.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.15.0","branchLabelMappingKey":"^v8.15.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/181230","number":181230,"mergeCommit":{"message":"[Cases] Fix flaky functional tests (#181230)\n\n## Summary\r\n\r\nI ran the flaky test runner multiple times. I did not find any good\r\nreason why some of the tests were flaky in the first place.\r\n\r\nFlaky test runner:\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5720,\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5721,\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5727\r\n\r\nFixes: https://github.com/elastic/kibana/issues/179016,\r\nhttps://github.com/elastic/kibana/issues/178991,\r\nhttps://github.com/elastic/kibana/issues/176716,\r\nhttps://github.com/elastic/kibana/issues/178690\r\n\r\n### Checklist\r\n\r\nDelete any items that are not applicable to this PR.\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n- [x] [Flaky Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\r\nused on any tests changed\r\n\r\n### For maintainers\r\n\r\n- [x] This was checked for breaking API changes and was [labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\r\n\r\n---------\r\n\r\nCo-authored-by: Kibana Machine <[email protected]>","sha":"8a992997e9f942b39182a087df44604dc69957cf"}}]}] BACKPORT--> Co-authored-by: Christos Nasikas <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
I ran the flaky test runner multiple times. I did not find any good reason why some of the tests were flaky in the first place.
Flaky test runner: https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5720, https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5721, https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5727
Fixes: #179016, #178991, #176716, #178690
Checklist
Delete any items that are not applicable to this PR.
For maintainers