[Security Solution] Fix flaky test for multiLineStringDiffAlgorithm #205038
+3
−3
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
banderror
added
test
release_note:skip
|
Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management) |
|
Pinging @elastic/security-solution (Team: SecuritySolution) |
|
Pinging @elastic/security-detections-response (Team:Detections and Resp) |
banderror
requested review from
nikitaindik and
dplumlee
and removed request for
jpdjere
banderror
force-pushed
the
fix-test-timeout-for-multiLineStringDiffAlgorithm
branch
from
218c892 to
e17a280
Compare
|
/ci |
dplumlee
approved these changes
Dec 21, 2024
banderror
force-pushed
the
fix-test-timeout-for-multiLineStringDiffAlgorithm
branch
from
e17a280 to
8e3375a
Compare
|
Starting backport for target branches: 8.16, 8.17, 8.x https://github.com/elastic/kibana/actions/runs/12466179256 |
💚 Build Succeeded
Metrics [docs]
History
cc @banderror |
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Dec 23, 2024
…lastic#205038) **Fixes: elastic#205014 ## Summary This test on CI runs at least 10x slower than locally, and apparently even 1000ms timeout is not enough. Bumping it to 2000ms and hopefully that will be it. Not sure if it makes sense to bump it even higher, because we need this threshold to be reasonably low, and more than 2 seconds doesn't sound low for local test runs. ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed (cherry picked from commit 8e0561a)
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Dec 23, 2024
…lastic#205038) **Fixes: elastic#205014 ## Summary This test on CI runs at least 10x slower than locally, and apparently even 1000ms timeout is not enough. Bumping it to 2000ms and hopefully that will be it. Not sure if it makes sense to bump it even higher, because we need this threshold to be reasonably low, and more than 2 seconds doesn't sound low for local test runs. ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed (cherry picked from commit 8e0561a)
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Dec 23, 2024
…lastic#205038) **Fixes: elastic#205014 ## Summary This test on CI runs at least 10x slower than locally, and apparently even 1000ms timeout is not enough. Bumping it to 2000ms and hopefully that will be it. Not sure if it makes sense to bump it even higher, because we need this threshold to be reasonably low, and more than 2 seconds doesn't sound low for local test runs. ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed (cherry picked from commit 8e0561a)
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
kibanamachine
added a commit
that referenced
this pull request
Dec 23, 2024
…rithm (#205038) (#205081) # Backport This will backport the following commits from `main` to `8.16`: - [[Security Solution] Fix flaky test for multiLineStringDiffAlgorithm (#205038)](#205038) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Georgii Gorbachev","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-12-23T11:36:28Z","message":"[Security Solution] Fix flaky test for multiLineStringDiffAlgorithm (#205038)\n\n**Fixes: https://github.com/elastic/kibana/issues/205014**\r\n\r\n## Summary\r\n\r\nThis test on CI runs at least 10x slower than locally, and apparently\r\neven 1000ms timeout is not enough.\r\n\r\nBumping it to 2000ms and hopefully that will be it. Not sure if it makes\r\nsense to bump it even higher, because we need this threshold to be\r\nreasonably low, and more than 2 seconds doesn't sound low for local test\r\nruns.\r\n\r\n\r\n### Checklist\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n- [ ] [Flaky Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\r\nused on any tests changed","sha":"8e0561a27b99a99d1b279e6f152b17622a685f1c","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["test","release_note:skip","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v8.18.0","v8.16.3","v8.17.1"],"title":"[Security Solution] Fix flaky test for multiLineStringDiffAlgorithm","number":205038,"url":"https://github.com/elastic/kibana/pull/205038","mergeCommit":{"message":"[Security Solution] Fix flaky test for multiLineStringDiffAlgorithm (#205038)\n\n**Fixes: https://github.com/elastic/kibana/issues/205014**\r\n\r\n## Summary\r\n\r\nThis test on CI runs at least 10x slower than locally, and apparently\r\neven 1000ms timeout is not enough.\r\n\r\nBumping it to 2000ms and hopefully that will be it. Not sure if it makes\r\nsense to bump it even higher, because we need this threshold to be\r\nreasonably low, and more than 2 seconds doesn't sound low for local test\r\nruns.\r\n\r\n\r\n### Checklist\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n- [ ] [Flaky Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\r\nused on any tests changed","sha":"8e0561a27b99a99d1b279e6f152b17622a685f1c"}},"sourceBranch":"main","suggestedTargetBranches":["8.x","8.16","8.17"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/205038","number":205038,"mergeCommit":{"message":"[Security Solution] Fix flaky test for multiLineStringDiffAlgorithm (#205038)\n\n**Fixes: https://github.com/elastic/kibana/issues/205014**\r\n\r\n## Summary\r\n\r\nThis test on CI runs at least 10x slower than locally, and apparently\r\neven 1000ms timeout is not enough.\r\n\r\nBumping it to 2000ms and hopefully that will be it. Not sure if it makes\r\nsense to bump it even higher, because we need this threshold to be\r\nreasonably low, and more than 2 seconds doesn't sound low for local test\r\nruns.\r\n\r\n\r\n### Checklist\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n- [ ] [Flaky Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\r\nused on any tests changed","sha":"8e0561a27b99a99d1b279e6f152b17622a685f1c"}},{"branch":"8.x","label":"v8.18.0","branchLabelMappingKey":"^v8.18.0$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.16","label":"v8.16.3","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.17","label":"v8.17.1","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Georgii Gorbachev <[email protected]>
kibanamachine
added a commit
that referenced
this pull request
Dec 23, 2024
…rithm (#205038) (#205082) # Backport This will backport the following commits from `main` to `8.17`: - [[Security Solution] Fix flaky test for multiLineStringDiffAlgorithm (#205038)](#205038) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Georgii Gorbachev","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-12-23T11:36:28Z","message":"[Security Solution] Fix flaky test for multiLineStringDiffAlgorithm (#205038)\n\n**Fixes: https://github.com/elastic/kibana/issues/205014**\r\n\r\n## Summary\r\n\r\nThis test on CI runs at least 10x slower than locally, and apparently\r\neven 1000ms timeout is not enough.\r\n\r\nBumping it to 2000ms and hopefully that will be it. Not sure if it makes\r\nsense to bump it even higher, because we need this threshold to be\r\nreasonably low, and more than 2 seconds doesn't sound low for local test\r\nruns.\r\n\r\n\r\n### Checklist\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n- [ ] [Flaky Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\r\nused on any tests changed","sha":"8e0561a27b99a99d1b279e6f152b17622a685f1c","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["test","release_note:skip","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v8.18.0","v8.16.3","v8.17.1"],"title":"[Security Solution] Fix flaky test for multiLineStringDiffAlgorithm","number":205038,"url":"https://github.com/elastic/kibana/pull/205038","mergeCommit":{"message":"[Security Solution] Fix flaky test for multiLineStringDiffAlgorithm (#205038)\n\n**Fixes: https://github.com/elastic/kibana/issues/205014**\r\n\r\n## Summary\r\n\r\nThis test on CI runs at least 10x slower than locally, and apparently\r\neven 1000ms timeout is not enough.\r\n\r\nBumping it to 2000ms and hopefully that will be it. Not sure if it makes\r\nsense to bump it even higher, because we need this threshold to be\r\nreasonably low, and more than 2 seconds doesn't sound low for local test\r\nruns.\r\n\r\n\r\n### Checklist\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n- [ ] [Flaky Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\r\nused on any tests changed","sha":"8e0561a27b99a99d1b279e6f152b17622a685f1c"}},"sourceBranch":"main","suggestedTargetBranches":["8.x","8.16","8.17"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/205038","number":205038,"mergeCommit":{"message":"[Security Solution] Fix flaky test for multiLineStringDiffAlgorithm (#205038)\n\n**Fixes: https://github.com/elastic/kibana/issues/205014**\r\n\r\n## Summary\r\n\r\nThis test on CI runs at least 10x slower than locally, and apparently\r\neven 1000ms timeout is not enough.\r\n\r\nBumping it to 2000ms and hopefully that will be it. Not sure if it makes\r\nsense to bump it even higher, because we need this threshold to be\r\nreasonably low, and more than 2 seconds doesn't sound low for local test\r\nruns.\r\n\r\n\r\n### Checklist\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n- [ ] [Flaky Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\r\nused on any tests changed","sha":"8e0561a27b99a99d1b279e6f152b17622a685f1c"}},{"branch":"8.x","label":"v8.18.0","branchLabelMappingKey":"^v8.18.0$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.16","label":"v8.16.3","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.17","label":"v8.17.1","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Georgii Gorbachev <[email protected]>
kibanamachine
added a commit
that referenced
this pull request
Dec 23, 2024
…ithm (#205038) (#205083) # Backport This will backport the following commits from `main` to `8.x`: - [[Security Solution] Fix flaky test for multiLineStringDiffAlgorithm (#205038)](#205038) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Georgii Gorbachev","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-12-23T11:36:28Z","message":"[Security Solution] Fix flaky test for multiLineStringDiffAlgorithm (#205038)\n\n**Fixes: https://github.com/elastic/kibana/issues/205014**\r\n\r\n## Summary\r\n\r\nThis test on CI runs at least 10x slower than locally, and apparently\r\neven 1000ms timeout is not enough.\r\n\r\nBumping it to 2000ms and hopefully that will be it. Not sure if it makes\r\nsense to bump it even higher, because we need this threshold to be\r\nreasonably low, and more than 2 seconds doesn't sound low for local test\r\nruns.\r\n\r\n\r\n### Checklist\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n- [ ] [Flaky Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\r\nused on any tests changed","sha":"8e0561a27b99a99d1b279e6f152b17622a685f1c","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["test","release_note:skip","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v8.18.0","v8.16.3","v8.17.1"],"title":"[Security Solution] Fix flaky test for multiLineStringDiffAlgorithm","number":205038,"url":"https://github.com/elastic/kibana/pull/205038","mergeCommit":{"message":"[Security Solution] Fix flaky test for multiLineStringDiffAlgorithm (#205038)\n\n**Fixes: https://github.com/elastic/kibana/issues/205014**\r\n\r\n## Summary\r\n\r\nThis test on CI runs at least 10x slower than locally, and apparently\r\neven 1000ms timeout is not enough.\r\n\r\nBumping it to 2000ms and hopefully that will be it. Not sure if it makes\r\nsense to bump it even higher, because we need this threshold to be\r\nreasonably low, and more than 2 seconds doesn't sound low for local test\r\nruns.\r\n\r\n\r\n### Checklist\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n- [ ] [Flaky Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\r\nused on any tests changed","sha":"8e0561a27b99a99d1b279e6f152b17622a685f1c"}},"sourceBranch":"main","suggestedTargetBranches":["8.x","8.16","8.17"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/205038","number":205038,"mergeCommit":{"message":"[Security Solution] Fix flaky test for multiLineStringDiffAlgorithm (#205038)\n\n**Fixes: https://github.com/elastic/kibana/issues/205014**\r\n\r\n## Summary\r\n\r\nThis test on CI runs at least 10x slower than locally, and apparently\r\neven 1000ms timeout is not enough.\r\n\r\nBumping it to 2000ms and hopefully that will be it. Not sure if it makes\r\nsense to bump it even higher, because we need this threshold to be\r\nreasonably low, and more than 2 seconds doesn't sound low for local test\r\nruns.\r\n\r\n\r\n### Checklist\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n- [ ] [Flaky Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\r\nused on any tests changed","sha":"8e0561a27b99a99d1b279e6f152b17622a685f1c"}},{"branch":"8.x","label":"v8.18.0","branchLabelMappingKey":"^v8.18.0$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.16","label":"v8.16.3","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.17","label":"v8.17.1","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Georgii Gorbachev <[email protected]>
banderror
deleted the
fix-test-timeout-for-multiLineStringDiffAlgorithm
branch
stratoula
pushed a commit
to stratoula/kibana
that referenced
this pull request
Jan 2, 2025
…lastic#205038) **Fixes: elastic#205014 ## Summary This test on CI runs at least 10x slower than locally, and apparently even 1000ms timeout is not enough. Bumping it to 2000ms and hopefully that will be it. Not sure if it makes sense to bump it even higher, because we need this threshold to be reasonably low, and more than 2 seconds doesn't sound low for local test runs. ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed
benakansara
pushed a commit
to benakansara/kibana
that referenced
this pull request
Jan 2, 2025
…lastic#205038) **Fixes: elastic#205014 ## Summary This test on CI runs at least 10x slower than locally, and apparently even 1000ms timeout is not enough. Bumping it to 2000ms and hopefully that will be it. Not sure if it makes sense to bump it even higher, because we need this threshold to be reasonably low, and more than 2 seconds doesn't sound low for local test runs. ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed
CAWilson94
pushed a commit
to CAWilson94/kibana
that referenced
this pull request
Jan 13, 2025
…lastic#205038) **Fixes: elastic#205014 ## Summary This test on CI runs at least 10x slower than locally, and apparently even 1000ms timeout is not enough. Bumping it to 2000ms and hopefully that will be it. Not sure if it makes sense to bump it even higher, because we need this threshold to be reasonably low, and more than 2 seconds doesn't sound low for local test runs. ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes: #205014
Summary
This test on CI runs at least 10x slower than locally, and apparently even 1000ms timeout is not enough.
Bumping it to 2000ms and hopefully that will be it. Not sure if it makes sense to bump it even higher, because we need this threshold to be reasonably low, and more than 2 seconds doesn't sound low for local test runs.
Checklist