Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[8.12] [Security Solution] Unskip bulk actions Cypress tests (#174365) (
#174599) # Backport This will backport the following commits from `main` to `8.12`: - [[Security Solution] Unskip bulk actions Cypress tests (#174365)](#174365) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Nikita Indik","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-01-10T13:23:31Z","message":"[Security Solution] Unskip bulk actions Cypress tests (#174365)\n\n## Summary\r\n\r\n**Resolves: https://github.com/elastic/kibana/issues/171101**\r\n\r\n200 runs of bulk_edit_rules_actions.cy.ts in ESS env: [*Buildkite\r\n4776*](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4776)\r\n200 runs of bulk_edit_rules_actions.cy.ts in Serverless env: [*Buildkite\r\n4777*](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4777)\r\n\r\nAll tests were green except a couple tests that stopped abruptly mid-way\r\nbecause of CI runner timeouts. So there weren't failed tests.\r\n\r\nTwo issues were causing fails:\r\n1. `\"hunter_no_actions\"` role that was used in \"User with no privileges\r\ncan't add rule actions\" test doesn't exist in Serverless env. Changed it\r\nto `\"t1_analyst\"` – it exists in both ESS and Serverless and doesn't\r\ngive permission to edit rules.\r\n2. Race condition caused by disabled auto-refresh\r\n- In the `beforeAll` hook the auto-refresh is disabled for the Rule\r\nManagement page.\r\n- Then `excessivelyInstallAllPrebuiltRules` is called, which installs\r\nall 1000+ prebuilt rules (only in Serverless, because ESS has 0 prebuilt\r\nrules available to install).\r\n- While the installation is in progress the Rule Management page loads\r\nand displays 9 rules.\r\n- Then the test selects all rules (9) and executes a bulk update on\r\nthem.\r\n- Once the bulk action succeeds, the user sees a toast with \"1000+\r\nactions have been updated\" while the test expects \"9 actions have been\r\nupdated\" because of disabled auto-refresh.\r\n- I decided to skip installing all the Elastic prebuilt rules because\r\nthe operation is very heavy and we check that bulk actions work for\r\nprebuilt rules anyways since two test prebuilt rules are created and\r\ninstalled in beforeAll.","sha":"53ffb143ed1e105f448632c27c1d7b858e13386e","branchLabelMapping":{"^v8.13.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","test-coverage","Team:Detections and Resp","Team: SecuritySolution","Feature:Rule Management","Team:Detection Rule Management","v8.12.1","v8.13.0"],"title":"[Security Solution] Unskip bulk actions Cypress tests","number":174365,"url":"https://github.com/elastic/kibana/pull/174365","mergeCommit":{"message":"[Security Solution] Unskip bulk actions Cypress tests (#174365)\n\n## Summary\r\n\r\n**Resolves: https://github.com/elastic/kibana/issues/171101**\r\n\r\n200 runs of bulk_edit_rules_actions.cy.ts in ESS env: [*Buildkite\r\n4776*](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4776)\r\n200 runs of bulk_edit_rules_actions.cy.ts in Serverless env: [*Buildkite\r\n4777*](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4777)\r\n\r\nAll tests were green except a couple tests that stopped abruptly mid-way\r\nbecause of CI runner timeouts. So there weren't failed tests.\r\n\r\nTwo issues were causing fails:\r\n1. `\"hunter_no_actions\"` role that was used in \"User with no privileges\r\ncan't add rule actions\" test doesn't exist in Serverless env. Changed it\r\nto `\"t1_analyst\"` – it exists in both ESS and Serverless and doesn't\r\ngive permission to edit rules.\r\n2. Race condition caused by disabled auto-refresh\r\n- In the `beforeAll` hook the auto-refresh is disabled for the Rule\r\nManagement page.\r\n- Then `excessivelyInstallAllPrebuiltRules` is called, which installs\r\nall 1000+ prebuilt rules (only in Serverless, because ESS has 0 prebuilt\r\nrules available to install).\r\n- While the installation is in progress the Rule Management page loads\r\nand displays 9 rules.\r\n- Then the test selects all rules (9) and executes a bulk update on\r\nthem.\r\n- Once the bulk action succeeds, the user sees a toast with \"1000+\r\nactions have been updated\" while the test expects \"9 actions have been\r\nupdated\" because of disabled auto-refresh.\r\n- I decided to skip installing all the Elastic prebuilt rules because\r\nthe operation is very heavy and we check that bulk actions work for\r\nprebuilt rules anyways since two test prebuilt rules are created and\r\ninstalled in beforeAll.","sha":"53ffb143ed1e105f448632c27c1d7b858e13386e"}},"sourceBranch":"main","suggestedTargetBranches":["8.12"],"targetPullRequestStates":[{"branch":"8.12","label":"v8.12.1","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.13.0","branchLabelMappingKey":"^v8.13.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/174365","number":174365,"mergeCommit":{"message":"[Security Solution] Unskip bulk actions Cypress tests (#174365)\n\n## Summary\r\n\r\n**Resolves: https://github.com/elastic/kibana/issues/171101**\r\n\r\n200 runs of bulk_edit_rules_actions.cy.ts in ESS env: [*Buildkite\r\n4776*](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4776)\r\n200 runs of bulk_edit_rules_actions.cy.ts in Serverless env: [*Buildkite\r\n4777*](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4777)\r\n\r\nAll tests were green except a couple tests that stopped abruptly mid-way\r\nbecause of CI runner timeouts. So there weren't failed tests.\r\n\r\nTwo issues were causing fails:\r\n1. `\"hunter_no_actions\"` role that was used in \"User with no privileges\r\ncan't add rule actions\" test doesn't exist in Serverless env. Changed it\r\nto `\"t1_analyst\"` – it exists in both ESS and Serverless and doesn't\r\ngive permission to edit rules.\r\n2. Race condition caused by disabled auto-refresh\r\n- In the `beforeAll` hook the auto-refresh is disabled for the Rule\r\nManagement page.\r\n- Then `excessivelyInstallAllPrebuiltRules` is called, which installs\r\nall 1000+ prebuilt rules (only in Serverless, because ESS has 0 prebuilt\r\nrules available to install).\r\n- While the installation is in progress the Rule Management page loads\r\nand displays 9 rules.\r\n- Then the test selects all rules (9) and executes a bulk update on\r\nthem.\r\n- Once the bulk action succeeds, the user sees a toast with \"1000+\r\nactions have been updated\" while the test expects \"9 actions have been\r\nupdated\" because of disabled auto-refresh.\r\n- I decided to skip installing all the Elastic prebuilt rules because\r\nthe operation is very heavy and we check that bulk actions work for\r\nprebuilt rules anyways since two test prebuilt rules are created and\r\ninstalled in beforeAll.","sha":"53ffb143ed1e105f448632c27c1d7b858e13386e"}}]}] BACKPORT--> Co-authored-by: Nikita Indik <[email protected]>
- Loading branch information