Skip to content

Commit

Permalink
[8.12] [Security Solution] Unskip bulk actions Cypress tests (#174365) (
Browse files Browse the repository at this point in the history
#174599)

# Backport

This will backport the following commits from `main` to `8.12`:
- [[Security Solution] Unskip bulk actions Cypress tests
(#174365)](#174365)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Nikita
Indik","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-01-10T13:23:31Z","message":"[Security
Solution] Unskip bulk actions Cypress tests (#174365)\n\n##
Summary\r\n\r\n**Resolves:
https://github.com/elastic/kibana/issues/171101**\r\n\r\n200 runs of
bulk_edit_rules_actions.cy.ts in ESS env:
[*Buildkite\r\n4776*](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4776)\r\n200
runs of bulk_edit_rules_actions.cy.ts in Serverless env:
[*Buildkite\r\n4777*](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4777)\r\n\r\nAll
tests were green except a couple tests that stopped abruptly
mid-way\r\nbecause of CI runner timeouts. So there weren't failed
tests.\r\n\r\nTwo issues were causing fails:\r\n1.
`\"hunter_no_actions\"` role that was used in \"User with no
privileges\r\ncan't add rule actions\" test doesn't exist in Serverless
env. Changed it\r\nto `\"t1_analyst\"` – it exists in both ESS and
Serverless and doesn't\r\ngive permission to edit rules.\r\n2. Race
condition caused by disabled auto-refresh\r\n- In the `beforeAll` hook
the auto-refresh is disabled for the Rule\r\nManagement page.\r\n- Then
`excessivelyInstallAllPrebuiltRules` is called, which installs\r\nall
1000+ prebuilt rules (only in Serverless, because ESS has 0
prebuilt\r\nrules available to install).\r\n- While the installation is
in progress the Rule Management page loads\r\nand displays 9 rules.\r\n-
Then the test selects all rules (9) and executes a bulk update
on\r\nthem.\r\n- Once the bulk action succeeds, the user sees a toast
with \"1000+\r\nactions have been updated\" while the test expects \"9
actions have been\r\nupdated\" because of disabled auto-refresh.\r\n- I
decided to skip installing all the Elastic prebuilt rules because\r\nthe
operation is very heavy and we check that bulk actions work
for\r\nprebuilt rules anyways since two test prebuilt rules are created
and\r\ninstalled in
beforeAll.","sha":"53ffb143ed1e105f448632c27c1d7b858e13386e","branchLabelMapping":{"^v8.13.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","test-coverage","Team:Detections
and Resp","Team: SecuritySolution","Feature:Rule
Management","Team:Detection Rule
Management","v8.12.1","v8.13.0"],"title":"[Security Solution] Unskip
bulk actions Cypress
tests","number":174365,"url":"https://github.com/elastic/kibana/pull/174365","mergeCommit":{"message":"[Security
Solution] Unskip bulk actions Cypress tests (#174365)\n\n##
Summary\r\n\r\n**Resolves:
https://github.com/elastic/kibana/issues/171101**\r\n\r\n200 runs of
bulk_edit_rules_actions.cy.ts in ESS env:
[*Buildkite\r\n4776*](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4776)\r\n200
runs of bulk_edit_rules_actions.cy.ts in Serverless env:
[*Buildkite\r\n4777*](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4777)\r\n\r\nAll
tests were green except a couple tests that stopped abruptly
mid-way\r\nbecause of CI runner timeouts. So there weren't failed
tests.\r\n\r\nTwo issues were causing fails:\r\n1.
`\"hunter_no_actions\"` role that was used in \"User with no
privileges\r\ncan't add rule actions\" test doesn't exist in Serverless
env. Changed it\r\nto `\"t1_analyst\"` – it exists in both ESS and
Serverless and doesn't\r\ngive permission to edit rules.\r\n2. Race
condition caused by disabled auto-refresh\r\n- In the `beforeAll` hook
the auto-refresh is disabled for the Rule\r\nManagement page.\r\n- Then
`excessivelyInstallAllPrebuiltRules` is called, which installs\r\nall
1000+ prebuilt rules (only in Serverless, because ESS has 0
prebuilt\r\nrules available to install).\r\n- While the installation is
in progress the Rule Management page loads\r\nand displays 9 rules.\r\n-
Then the test selects all rules (9) and executes a bulk update
on\r\nthem.\r\n- Once the bulk action succeeds, the user sees a toast
with \"1000+\r\nactions have been updated\" while the test expects \"9
actions have been\r\nupdated\" because of disabled auto-refresh.\r\n- I
decided to skip installing all the Elastic prebuilt rules because\r\nthe
operation is very heavy and we check that bulk actions work
for\r\nprebuilt rules anyways since two test prebuilt rules are created
and\r\ninstalled in
beforeAll.","sha":"53ffb143ed1e105f448632c27c1d7b858e13386e"}},"sourceBranch":"main","suggestedTargetBranches":["8.12"],"targetPullRequestStates":[{"branch":"8.12","label":"v8.12.1","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.13.0","branchLabelMappingKey":"^v8.13.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/174365","number":174365,"mergeCommit":{"message":"[Security
Solution] Unskip bulk actions Cypress tests (#174365)\n\n##
Summary\r\n\r\n**Resolves:
https://github.com/elastic/kibana/issues/171101**\r\n\r\n200 runs of
bulk_edit_rules_actions.cy.ts in ESS env:
[*Buildkite\r\n4776*](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4776)\r\n200
runs of bulk_edit_rules_actions.cy.ts in Serverless env:
[*Buildkite\r\n4777*](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4777)\r\n\r\nAll
tests were green except a couple tests that stopped abruptly
mid-way\r\nbecause of CI runner timeouts. So there weren't failed
tests.\r\n\r\nTwo issues were causing fails:\r\n1.
`\"hunter_no_actions\"` role that was used in \"User with no
privileges\r\ncan't add rule actions\" test doesn't exist in Serverless
env. Changed it\r\nto `\"t1_analyst\"` – it exists in both ESS and
Serverless and doesn't\r\ngive permission to edit rules.\r\n2. Race
condition caused by disabled auto-refresh\r\n- In the `beforeAll` hook
the auto-refresh is disabled for the Rule\r\nManagement page.\r\n- Then
`excessivelyInstallAllPrebuiltRules` is called, which installs\r\nall
1000+ prebuilt rules (only in Serverless, because ESS has 0
prebuilt\r\nrules available to install).\r\n- While the installation is
in progress the Rule Management page loads\r\nand displays 9 rules.\r\n-
Then the test selects all rules (9) and executes a bulk update
on\r\nthem.\r\n- Once the bulk action succeeds, the user sees a toast
with \"1000+\r\nactions have been updated\" while the test expects \"9
actions have been\r\nupdated\" because of disabled auto-refresh.\r\n- I
decided to skip installing all the Elastic prebuilt rules because\r\nthe
operation is very heavy and we check that bulk actions work
for\r\nprebuilt rules anyways since two test prebuilt rules are created
and\r\ninstalled in
beforeAll.","sha":"53ffb143ed1e105f448632c27c1d7b858e13386e"}}]}]
BACKPORT-->

Co-authored-by: Nikita Indik <[email protected]>
  • Loading branch information
kibanamachine and nikitaindik authored Jan 10, 2024
1 parent 16b2c7b commit e40d00c
Showing 1 changed file with 4 additions and 15 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ import { createRuleAssetSavedObject } from '../../../../../helpers/rules';
import {
RULES_BULK_EDIT_ACTIONS_INFO,
RULES_BULK_EDIT_ACTIONS_WARNING,
ADD_RULE_ACTIONS_MENU_ITEM,
BULK_ACTIONS_BTN,
} from '../../../../../screens/rules_bulk_actions';
import { actionFormSelector } from '../../../../../screens/common/rule_actions';

Expand Down Expand Up @@ -47,7 +47,6 @@ import {
submitBulkEditForm,
checkOverwriteRuleActionsCheckbox,
openBulkEditRuleActionsForm,
openBulkActionsMenu,
} from '../../../../../tasks/rules_bulk_actions';
import { login } from '../../../../../tasks/login';
import { visitRulesManagementTable } from '../../../../../tasks/rules_management';
Expand All @@ -65,7 +64,6 @@ import {
} from '../../../../../objects/rule';
import {
createAndInstallMockedPrebuiltRules,
excessivelyInstallAllPrebuiltRules,
preventPrebuiltRulesPackageInstallation,
} from '../../../../../tasks/api_calls/prebuilt_rules';

Expand All @@ -74,10 +72,9 @@ const ruleNameToAssert = 'Custom rule name with actions';
const expectedExistingSlackMessage = 'Existing slack action';
const expectedSlackMessage = 'Slack action test message';

// TODO: Fix and unskip in Serverless https://github.com/elastic/kibana/issues/171101
describe(
'Detection rules, bulk edit of rule actions',
{ tags: ['@ess', '@serverless', '@brokenInServerless', '@brokenInServerlessQA'] },
{ tags: ['@ess', '@serverless', '@brokenInServerlessQA'] },
() => {
beforeEach(() => {
login();
Expand Down Expand Up @@ -148,7 +145,7 @@ describe(

context('Restricted action privileges', () => {
it("User with no privileges can't add rule actions", () => {
login(ROLES.hunter_no_actions);
login(ROLES.t1_analyst);
visitRulesManagementTable();

expectManagementTableRules([
Expand All @@ -164,11 +161,7 @@ describe(
]);
waitForCallOutToBeShown(MISSING_PRIVILEGES_CALLOUT, 'primary');

selectAllRules();

openBulkActionsMenu();

cy.get(ADD_RULE_ACTIONS_MENU_ITEM).should('be.disabled');
cy.get(BULK_ACTIONS_BTN).should('not.exist');
});
});

Expand Down Expand Up @@ -197,8 +190,6 @@ describe(
throttleUnit: 'd',
};

excessivelyInstallAllPrebuiltRules();

getRulesManagementTableRows().then((rows) => {
// select both custom and prebuilt rules
selectAllRules();
Expand Down Expand Up @@ -227,8 +218,6 @@ describe(
});

it('Overwrite rule actions in rules', () => {
excessivelyInstallAllPrebuiltRules();

getRulesManagementTableRows().then((rows) => {
// select both custom and prebuilt rules
selectAllRules();
Expand Down

0 comments on commit e40d00c

Please sign in to comment.