Skip to content

Commit

Permalink
[8.9] [Security Solution] Unskip rules table auto-refresh Cypress tes…
Browse files Browse the repository at this point in the history
…ts (#163451) (#164062)

# Backport

This will backport the following commits from `main` to `8.9`:
- [[Security Solution] Unskip rules table auto-refresh Cypress tests
(#163451)](#163451)

<!--- Backport version: 8.9.8 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Maxim
Palenov","email":"[email protected]"},"sourceCommit":{"committedDate":"2023-08-11T12:07:35Z","message":"[Security
Solution] Unskip rules table auto-refresh Cypress tests
(#163451)\n\n**Addresses:**
https://github.com/elastic/kibana/issues/154694\r\n\r\n##
Summary\r\n\r\nThis PR unskips rules table auto-refresh tests.\r\n\r\n##
Details\r\n\r\nRules table auto-refresh Cypress tests were outdated so
it was hard to determine the original flakiness reason. To make it
working the following has been done\r\n\r\n- `mockGlobalClock()` moved
above `visit()` to mock the time before visiting the page. It didn't
look like working so the test had to wait for 60 seconds (our refresh
interval)\r\n- removed unnecessary waiting
`waitForRulesTableToBeLoaded()` as the test should work without it\r\n-
removed `setRowsPerPageTo(5)` as the test doesn't look related to the
page size\r\n- `AutoRefreshButtonComponent ` was refactored in attempt
to fix popover closing related flakiness but it doesn't seem to help but
the refactoring looks improving readability so leaving as a part of this
PR\r\n- auto-refresh popover's behavior has been changed. The trigger
button gets disabled if there is a rule selected. Clicking the disabled
button won't make the auto-refresh popover to appear so this part was
changed.\r\n\r\n### Encountered issues\r\n\r\n- `EuiPopover` should
close by clicking the trigger button, clicking outside or pressing `Esc`
button. The latter two cause flakiness for some reason. I spent quite
some time to investigate the reason but without success. It looks like
there is a race condition related to
[EuiFocusTrap](https://github.com/elastic/eui/blob/main/src/components/focus_trap/focus_trap.tsx)
but it doesn't look to be an issue outside Cypress.\r\n-
`waitForRulesTableToBeLoaded()` basically does nothing after changes in
the rules table. Looking at tis contents\r\n\r\n```ts\r\nexport const
waitForRulesTableToBeLoaded = () => {\r\n // Wait up to 5 minutes for
the rules to load as in CI containers this can be very slow\r\n
cy.get(RULES_TABLE_INITIAL_LOADING_INDICATOR, { timeout: 300000
}).should('not.exist');\r\n};\r\n```\r\n\r\nreveals that
`RULES_TABLE_INITIAL_LOADING_INDICATOR` defined as
`[data-test-subj=\"initialLoadingPanelAllRulesTable\"]` doesn't exist
anymore in our codebase so this function instantly returns. The selector
will be fixed in a separate PR.\r\n\r\n- invoking `mockGlobalClock()`
during the test leads to appearing a lot of `Executing a cancelled
action` error messages.\r\n\r\n\r\n### Flaky test runner\r\n\r\n[50 runs
succeeded](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2842)","sha":"589ef228fb9c2525a49d6abc5db64eb98b2c381e","branchLabelMapping":{"^v8.10.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["test","skipped-test","release_note:skip","impact:high","Team:Detections
and Resp","Team: SecuritySolution","Feature:Rule
Management","Team:Detection Rule
Management","backport:prev-minor","v8.10.0"],"number":163451,"url":"https://github.com/elastic/kibana/pull/163451","mergeCommit":{"message":"[Security
Solution] Unskip rules table auto-refresh Cypress tests
(#163451)\n\n**Addresses:**
https://github.com/elastic/kibana/issues/154694\r\n\r\n##
Summary\r\n\r\nThis PR unskips rules table auto-refresh tests.\r\n\r\n##
Details\r\n\r\nRules table auto-refresh Cypress tests were outdated so
it was hard to determine the original flakiness reason. To make it
working the following has been done\r\n\r\n- `mockGlobalClock()` moved
above `visit()` to mock the time before visiting the page. It didn't
look like working so the test had to wait for 60 seconds (our refresh
interval)\r\n- removed unnecessary waiting
`waitForRulesTableToBeLoaded()` as the test should work without it\r\n-
removed `setRowsPerPageTo(5)` as the test doesn't look related to the
page size\r\n- `AutoRefreshButtonComponent ` was refactored in attempt
to fix popover closing related flakiness but it doesn't seem to help but
the refactoring looks improving readability so leaving as a part of this
PR\r\n- auto-refresh popover's behavior has been changed. The trigger
button gets disabled if there is a rule selected. Clicking the disabled
button won't make the auto-refresh popover to appear so this part was
changed.\r\n\r\n### Encountered issues\r\n\r\n- `EuiPopover` should
close by clicking the trigger button, clicking outside or pressing `Esc`
button. The latter two cause flakiness for some reason. I spent quite
some time to investigate the reason but without success. It looks like
there is a race condition related to
[EuiFocusTrap](https://github.com/elastic/eui/blob/main/src/components/focus_trap/focus_trap.tsx)
but it doesn't look to be an issue outside Cypress.\r\n-
`waitForRulesTableToBeLoaded()` basically does nothing after changes in
the rules table. Looking at tis contents\r\n\r\n```ts\r\nexport const
waitForRulesTableToBeLoaded = () => {\r\n // Wait up to 5 minutes for
the rules to load as in CI containers this can be very slow\r\n
cy.get(RULES_TABLE_INITIAL_LOADING_INDICATOR, { timeout: 300000
}).should('not.exist');\r\n};\r\n```\r\n\r\nreveals that
`RULES_TABLE_INITIAL_LOADING_INDICATOR` defined as
`[data-test-subj=\"initialLoadingPanelAllRulesTable\"]` doesn't exist
anymore in our codebase so this function instantly returns. The selector
will be fixed in a separate PR.\r\n\r\n- invoking `mockGlobalClock()`
during the test leads to appearing a lot of `Executing a cancelled
action` error messages.\r\n\r\n\r\n### Flaky test runner\r\n\r\n[50 runs
succeeded](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2842)","sha":"589ef228fb9c2525a49d6abc5db64eb98b2c381e"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v8.10.0","labelRegex":"^v8.10.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/163451","number":163451,"mergeCommit":{"message":"[Security
Solution] Unskip rules table auto-refresh Cypress tests
(#163451)\n\n**Addresses:**
https://github.com/elastic/kibana/issues/154694\r\n\r\n##
Summary\r\n\r\nThis PR unskips rules table auto-refresh tests.\r\n\r\n##
Details\r\n\r\nRules table auto-refresh Cypress tests were outdated so
it was hard to determine the original flakiness reason. To make it
working the following has been done\r\n\r\n- `mockGlobalClock()` moved
above `visit()` to mock the time before visiting the page. It didn't
look like working so the test had to wait for 60 seconds (our refresh
interval)\r\n- removed unnecessary waiting
`waitForRulesTableToBeLoaded()` as the test should work without it\r\n-
removed `setRowsPerPageTo(5)` as the test doesn't look related to the
page size\r\n- `AutoRefreshButtonComponent ` was refactored in attempt
to fix popover closing related flakiness but it doesn't seem to help but
the refactoring looks improving readability so leaving as a part of this
PR\r\n- auto-refresh popover's behavior has been changed. The trigger
button gets disabled if there is a rule selected. Clicking the disabled
button won't make the auto-refresh popover to appear so this part was
changed.\r\n\r\n### Encountered issues\r\n\r\n- `EuiPopover` should
close by clicking the trigger button, clicking outside or pressing `Esc`
button. The latter two cause flakiness for some reason. I spent quite
some time to investigate the reason but without success. It looks like
there is a race condition related to
[EuiFocusTrap](https://github.com/elastic/eui/blob/main/src/components/focus_trap/focus_trap.tsx)
but it doesn't look to be an issue outside Cypress.\r\n-
`waitForRulesTableToBeLoaded()` basically does nothing after changes in
the rules table. Looking at tis contents\r\n\r\n```ts\r\nexport const
waitForRulesTableToBeLoaded = () => {\r\n // Wait up to 5 minutes for
the rules to load as in CI containers this can be very slow\r\n
cy.get(RULES_TABLE_INITIAL_LOADING_INDICATOR, { timeout: 300000
}).should('not.exist');\r\n};\r\n```\r\n\r\nreveals that
`RULES_TABLE_INITIAL_LOADING_INDICATOR` defined as
`[data-test-subj=\"initialLoadingPanelAllRulesTable\"]` doesn't exist
anymore in our codebase so this function instantly returns. The selector
will be fixed in a separate PR.\r\n\r\n- invoking `mockGlobalClock()`
during the test leads to appearing a lot of `Executing a cancelled
action` error messages.\r\n\r\n\r\n### Flaky test runner\r\n\r\n[50 runs
succeeded](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2842)","sha":"589ef228fb9c2525a49d6abc5db64eb98b2c381e"}}]}]
BACKPORT-->

Co-authored-by: Kibana Machine <[email protected]>
  • Loading branch information
maximpn and kibanamachine authored Aug 21, 2023
1 parent ad228fe commit 6a21870
Show file tree
Hide file tree
Showing 4 changed files with 88 additions and 81 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -8,37 +8,36 @@
import {
RULE_CHECKBOX,
REFRESH_RULES_STATUS,
REFRESH_SETTINGS_SWITCH,
REFRESH_SETTINGS_SELECTION_NOTE,
RULES_TABLE_AUTOREFRESH_INDICATOR,
RULES_MANAGEMENT_TABLE,
} from '../../screens/alerts_detection_rules';
import {
checkAutoRefresh,
waitForRulesTableToBeLoaded,
selectAllRules,
openRefreshSettingsPopover,
clearAllRuleSelection,
selectNumberOfRules,
mockGlobalClock,
disableAutoRefresh,
checkAutoRefreshIsDisabled,
checkAutoRefreshIsEnabled,
expectAutoRefreshIsDisabled,
expectAutoRefreshIsEnabled,
expectAutoRefreshIsDeactivated,
expectNumberOfRules,
} from '../../tasks/alerts_detection_rules';
import { login, visit, visitWithoutDateRange } from '../../tasks/login';

import { DETECTIONS_RULE_MANAGEMENT_URL } from '../../urls/navigation';
import { createRule } from '../../tasks/api_calls/rules';
import { cleanKibana } from '../../tasks/common';
import { getNewRule } from '../../objects/rule';
import { setRowsPerPageTo } from '../../tasks/table_pagination';

const DEFAULT_RULE_REFRESH_INTERVAL_VALUE = 60000;
const NUM_OF_TEST_RULES = 6;

// TODO: See https://github.com/elastic/kibana/issues/154694
describe.skip('Alerts detection rules table auto-refresh', () => {
describe('Rules table: auto-refresh', () => {
before(() => {
cleanKibana();
login();
for (let i = 1; i < 7; i += 1) {

for (let i = 1; i <= NUM_OF_TEST_RULES; ++i) {
createRule(getNewRule({ name: `Test rule ${i}`, rule_id: `${i}` }));
}
});
Expand All @@ -48,31 +47,31 @@ describe.skip('Alerts detection rules table auto-refresh', () => {
});

it('Auto refreshes rules', () => {
mockGlobalClock();
visitWithoutDateRange(DETECTIONS_RULE_MANAGEMENT_URL);

waitForRulesTableToBeLoaded();

// ensure rules have rendered. As there is no user interaction in this test,
// rules were not rendered before test completes
cy.get(RULE_CHECKBOX).should('have.length', 6);
expectNumberOfRules(RULES_MANAGEMENT_TABLE, NUM_OF_TEST_RULES);

// // mock 1 minute passing to make sure refresh is conducted
mockGlobalClock();
checkAutoRefresh(DEFAULT_RULE_REFRESH_INTERVAL_VALUE, 'be.visible');
cy.get(RULES_TABLE_AUTOREFRESH_INDICATOR).should('not.exist');
cy.tick(DEFAULT_RULE_REFRESH_INTERVAL_VALUE);
cy.get(RULES_TABLE_AUTOREFRESH_INDICATOR).should('be.visible');

cy.contains(REFRESH_RULES_STATUS, 'Updated now');
});

it('should prevent table from rules refetch if any rule selected', () => {
mockGlobalClock();
visitWithoutDateRange(DETECTIONS_RULE_MANAGEMENT_URL);

waitForRulesTableToBeLoaded();
expectNumberOfRules(RULES_MANAGEMENT_TABLE, NUM_OF_TEST_RULES);

selectNumberOfRules(1);

// mock 1 minute passing to make sure refresh is not conducted
mockGlobalClock();
checkAutoRefresh(DEFAULT_RULE_REFRESH_INTERVAL_VALUE, 'not.exist');
cy.get(RULES_TABLE_AUTOREFRESH_INDICATOR).should('not.exist');
cy.tick(DEFAULT_RULE_REFRESH_INTERVAL_VALUE);
cy.get(RULES_TABLE_AUTOREFRESH_INDICATOR).should('not.exist');

// ensure rule is still selected
cy.get(RULE_CHECKBOX).first().should('be.checked');
Expand All @@ -82,50 +81,37 @@ describe.skip('Alerts detection rules table auto-refresh', () => {

it('should disable auto refresh when any rule selected and enable it after rules unselected', () => {
visit(DETECTIONS_RULE_MANAGEMENT_URL);
waitForRulesTableToBeLoaded();
setRowsPerPageTo(5);

expectNumberOfRules(RULES_MANAGEMENT_TABLE, NUM_OF_TEST_RULES);

// check refresh settings if it's enabled before selecting
openRefreshSettingsPopover();
checkAutoRefreshIsEnabled();
expectAutoRefreshIsEnabled();

selectAllRules();

// auto refresh should be disabled after rules selected
openRefreshSettingsPopover();
checkAutoRefreshIsDisabled();

// if any rule selected, refresh switch should be disabled and help note to users should displayed
cy.get(REFRESH_SETTINGS_SWITCH).should('be.disabled');
cy.contains(
REFRESH_SETTINGS_SELECTION_NOTE,
'Note: Refresh is disabled while there is an active selection.'
);
// auto refresh should be deactivated (which means disabled without an ability to enable it) after rules selected
expectAutoRefreshIsDeactivated();

clearAllRuleSelection();

// after all rules unselected, auto refresh should renew
openRefreshSettingsPopover();
checkAutoRefreshIsEnabled();
// after all rules unselected, auto refresh should be reset to its previous state
expectAutoRefreshIsEnabled();
});

it('should not enable auto refresh after rules were unselected if auto refresh was disabled', () => {
visit(DETECTIONS_RULE_MANAGEMENT_URL);
waitForRulesTableToBeLoaded();
setRowsPerPageTo(5);

openRefreshSettingsPopover();
expectNumberOfRules(RULES_MANAGEMENT_TABLE, NUM_OF_TEST_RULES);

disableAutoRefresh();

selectAllRules();

openRefreshSettingsPopover();
checkAutoRefreshIsDisabled();
expectAutoRefreshIsDeactivated();

clearAllRuleSelection();

// after all rules unselected, auto refresh should still be disabled
openRefreshSettingsPopover();
checkAutoRefreshIsDisabled();
expectAutoRefreshIsDisabled();
});
});
Original file line number Diff line number Diff line change
Expand Up @@ -169,7 +169,7 @@ export const RULES_SELECTED_TAG = '.euiSelectableListItem[aria-checked="true"]';

export const SELECTED_RULES_NUMBER_LABEL = '[data-test-subj="selectedRules"]';

export const REFRESH_SETTINGS_POPOVER = '[data-test-subj="refreshSettings-popover"]';
export const AUTO_REFRESH_POPOVER_TRIGGER_BUTTON = '[data-test-subj="autoRefreshButton"]';

export const REFRESH_RULES_TABLE_BUTTON = '[data-test-subj="refreshRulesAction-linkIcon"]';

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,6 @@ import {
RULES_TAGS_POPOVER_WRAPPER,
INTEGRATIONS_POPOVER,
SELECTED_RULES_NUMBER_LABEL,
REFRESH_SETTINGS_POPOVER,
REFRESH_SETTINGS_SWITCH,
ELASTIC_RULES_BTN,
BULK_EXPORT_ACTION_BTN,
Expand All @@ -65,6 +64,7 @@ import {
DUPLICATE_WITH_EXCEPTIONS_WITHOUT_EXPIRED_OPTION,
TOASTER_CLOSE_ICON,
ADD_ELASTIC_RULES_EMPTY_PROMPT_BTN,
AUTO_REFRESH_POPOVER_TRIGGER_BUTTON,
} from '../screens/alerts_detection_rules';
import type { RULES_MONITORING_TABLE } from '../screens/alerts_detection_rules';
import { EUI_CHECKBOX } from '../screens/common/controls';
Expand Down Expand Up @@ -506,22 +506,45 @@ export const testMultipleSelectedRulesLabel = (rulesCount: number) => {
cy.get(SELECTED_RULES_NUMBER_LABEL).should('have.text', `Selected ${rulesCount} rules`);
};

export const openRefreshSettingsPopover = () => {
cy.get(REFRESH_SETTINGS_POPOVER).click();
const openRefreshSettingsPopover = () => {
cy.get(REFRESH_SETTINGS_SWITCH).should('not.exist');
cy.get(AUTO_REFRESH_POPOVER_TRIGGER_BUTTON).click();
cy.get(REFRESH_SETTINGS_SWITCH).should('be.visible');
};

export const checkAutoRefreshIsDisabled = () => {
const closeRefreshSettingsPopover = () => {
cy.get(REFRESH_SETTINGS_SWITCH).should('be.visible');
cy.get(AUTO_REFRESH_POPOVER_TRIGGER_BUTTON).click();
cy.get(REFRESH_SETTINGS_SWITCH).should('not.exist');
};

export const expectAutoRefreshIsDisabled = () => {
cy.get(AUTO_REFRESH_POPOVER_TRIGGER_BUTTON).should('be.enabled');
cy.get(AUTO_REFRESH_POPOVER_TRIGGER_BUTTON).should('have.text', 'Off');
openRefreshSettingsPopover();
cy.get(REFRESH_SETTINGS_SWITCH).should('have.attr', 'aria-checked', 'false');
closeRefreshSettingsPopover();
};

export const checkAutoRefreshIsEnabled = () => {
export const expectAutoRefreshIsEnabled = () => {
cy.get(AUTO_REFRESH_POPOVER_TRIGGER_BUTTON).should('be.enabled');
cy.get(AUTO_REFRESH_POPOVER_TRIGGER_BUTTON).should('have.text', 'On');
openRefreshSettingsPopover();
cy.get(REFRESH_SETTINGS_SWITCH).should('have.attr', 'aria-checked', 'true');
closeRefreshSettingsPopover();
};

// Expects the auto refresh to be deactivated which means it's disabled without an ability to enable it
// so it's even impossible to open the popover
export const expectAutoRefreshIsDeactivated = () => {
cy.get(AUTO_REFRESH_POPOVER_TRIGGER_BUTTON).should('be.disabled');
cy.get(AUTO_REFRESH_POPOVER_TRIGGER_BUTTON).should('have.text', 'Off');
};

export const disableAutoRefresh = () => {
openRefreshSettingsPopover();
cy.get(REFRESH_SETTINGS_SWITCH).click();
checkAutoRefreshIsDisabled();
expectAutoRefreshIsDisabled();
};

export const mockGlobalClock = () => {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -41,28 +41,50 @@ const AutoRefreshButtonComponent = ({
setIsRefreshOn,
}: AutoRefreshButtonProps) => {
const [isPopoverOpen, setIsPopoverOpen] = useState(false);
const closePopover = useCallback(() => setIsPopoverOpen(false), [setIsPopoverOpen]);
const togglePopover = useCallback(
() => setIsPopoverOpen((prevState) => !prevState),
[setIsPopoverOpen]
);

const handleAutoRefreshSwitch = useCallback(
(closePopover: () => void) => (e: EuiSwitchEvent) => {
(e: EuiSwitchEvent) => {
const refreshOn = e.target.checked;
if (refreshOn) {
reFetchRules();
}
setIsRefreshOn(refreshOn);
closePopover();
},
[reFetchRules, setIsRefreshOn]
[reFetchRules, setIsRefreshOn, closePopover]
);

const handleGetRefreshSettingsPopoverContent = useCallback(
(closePopover: () => void) => (
return (
<EuiPopover
isOpen={isPopoverOpen}
closePopover={closePopover}
button={
<EuiButtonEmpty
data-test-subj="autoRefreshButton"
color={'text'}
iconType={'timeRefresh'}
onClick={togglePopover}
disabled={isDisabled}
css={css`
margin-left: 10px;
`}
>
{isRefreshOn ? 'On' : 'Off'}
</EuiButtonEmpty>
}
>
<EuiContextMenuPanel
items={[
<EuiSwitch
key="allRulesAutoRefreshSwitch"
label={i18n.REFRESH_RULE_POPOVER_DESCRIPTION}
checked={isRefreshOn ?? false}
onChange={handleAutoRefreshSwitch(closePopover)}
onChange={handleAutoRefreshSwitch}
compressed
disabled={isDisabled}
data-test-subj="refreshSettingsSwitch"
Expand All @@ -82,30 +104,6 @@ const AutoRefreshButtonComponent = ({
: []),
]}
/>
),
[isRefreshOn, handleAutoRefreshSwitch, isDisabled]
);

return (
<EuiPopover
isOpen={isPopoverOpen}
closePopover={() => setIsPopoverOpen(false)}
button={
<EuiButtonEmpty
data-test-subj="autoRefreshButton"
color={'text'}
iconType={'timeRefresh'}
onClick={() => setIsPopoverOpen(!isPopoverOpen)}
disabled={isDisabled}
css={css`
margin-left: 10px;
`}
>
{isRefreshOn ? 'On' : 'Off'}
</EuiButtonEmpty>
}
>
{handleGetRefreshSettingsPopoverContent(() => setIsPopoverOpen(false))}
</EuiPopover>
);
};
Expand Down

0 comments on commit 6a21870

Please sign in to comment.