Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flaky alert assignment tests #176930

Merged
merged 9 commits into from
Mar 6, 2024
Merged

Flaky alert assignment tests #176930

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
d1d2f4c
Flaky alert assignment tests
e40pud Feb 14, 2024
432f4de
Merge branch 'main' into security/tests/flaky-alert-assignments
kibanamachine Feb 16, 2024
0f6d112
Merge branch 'main' into security/tests/flaky-alert-assignments
kibanamachine Feb 19, 2024
a9f8b2f
Merge branch 'main' into security/tests/flaky-alert-assignments
kibanamachine Feb 23, 2024
093e8bb
Review feedback
e40pud Feb 23, 2024
c098902
Merge branch 'main' into security/tests/flaky-alert-assignments
kibanamachine Feb 23, 2024
915ea6f
Merge branch 'main' into security/tests/flaky-alert-assignments
kibanamachine Mar 5, 2024
161649f
Remove redundant cypress tests
e40pud Mar 5, 2024
757591d
Merge branch 'main' into security/tests/flaky-alert-assignments
e40pud Mar 6, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
188 changes: 24 additions & 164 deletions ...ss/e2e/detection_response/detection_engine/detection_alerts/assignments/assignments.cy.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
* 2.0.
*/

import { ROLES } from '@kbn/security-solution-plugin/common/test';
import { visitWithTimeRange } from '../../../../../tasks/navigation';
import { getNewRule } from '../../../../../objects/rule';
import {
closeAlertFlyout,
Expand All @@ -17,13 +17,11 @@ import {
} from '../../../../../tasks/alerts';
import { createRule } from '../../../../../tasks/api_calls/rules';
import { deleteAlertsAndRules } from '../../../../../tasks/api_calls/common';
import { login } from '../../../../../tasks/login';
import { getDefaultUserName, login } from '../../../../../tasks/login';
import { ALERTS_URL } from '../../../../../urls/navigation';
import { waitForAlertsToPopulate } from '../../../../../tasks/create_new_rule';
import {
alertDetailsFlyoutShowsAssignees,
alertDetailsFlyoutShowsAssigneesBadge,
alertsTableShowsAssigneesBadgeForFirstAlert,
alertsTableShowsAssigneesForAlert,
updateAssigneesForFirstAlert,
checkEmptyAssigneesStateInAlertDetailsFlyout,
Expand All @@ -38,83 +36,49 @@ import {
updateAssigneesViaAddButtonInFlyout,
updateAssigneesViaTakeActionButtonInFlyout,
removeAllAssigneesViaTakeActionButtonInFlyout,
loadPageAs,
} from '../../../../../tasks/alert_assignments';
import { ALERTS_COUNT } from '../../../../../screens/alerts';

// FLAKY: https://github.com/elastic/kibana/issues/173429
// FLAKY: https://github.com/elastic/kibana/issues/177334
describe.skip('Alert user assignment - ESS & Serverless', { tags: ['@ess', '@serverless'] }, () => {
describe('Alert user assignment - ESS & Serverless', { tags: ['@ess', '@serverless'] }, () => {
before(() => {
cy.task('esArchiverLoad', { archiveName: 'auditbeat_multiple' });

// Login into accounts so that they got activated and visible in user profiles list
login(ROLES.t1_analyst);
login(ROLES.t2_analyst);
login(ROLES.t3_analyst);
login(ROLES.soc_manager);
login(ROLES.detections_admin);
login(ROLES.platform_engineer);
});

after(() => {
cy.task('esArchiverUnload', 'auditbeat_multiple');
});

beforeEach(() => {
loadPageAs(ALERTS_URL);
login();
deleteAlertsAndRules();
createRule(getNewRule({ rule_id: 'new custom rule' }));
visitWithTimeRange(ALERTS_URL);
waitForAlertsToPopulate();
});

context('Basic rendering', () => {
it('alert with no assignees in alerts table', () => {
it('alert with no assignees in alerts table & details flyout', () => {
checkEmptyAssigneesStateInAlertsTable();
});

it(`alert with no assignees in alert's details flyout`, () => {
expandFirstAlert();
checkEmptyAssigneesStateInAlertDetailsFlyout();
});

it('alert with some assignees in alerts table', () => {
const users = [ROLES.detections_admin, ROLES.t1_analyst];
rylnd marked this conversation as resolved.
Show resolved Hide resolved
it('alert with some assignees in alerts table & details flyout', () => {
const users = [getDefaultUserName()];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT: Change all the users constants to user since we only have one.

updateAssigneesForFirstAlert(users);

alertsTableShowsAssigneesForAlert(users);
});

it(`alert with some assignees in alert's details flyout`, () => {
const users = [ROLES.detections_admin, ROLES.t1_analyst];
updateAssigneesForFirstAlert(users);
expandFirstAlert();
alertDetailsFlyoutShowsAssignees(users);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: it would be nice to have some convention (maybe either function naming (assert as a prefix), or folder location (/assertions), or both) to identify these tasks as performing assertions.

I know that for a while @MadameSheema was requesting that we not abstract assertions into helpers at all, but I think if we do so we should make those assertions a bit more discoverable.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MadameSheema any thought/preferences on this one?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey!! the overall preference is to NOT abstract assertions.

});

it('alert with many assignees (collapsed into badge) in alerts table', () => {
const users = [
ROLES.t1_analyst,
ROLES.t2_analyst,
ROLES.t3_analyst,
ROLES.soc_manager,
ROLES.detections_admin,
];
updateAssigneesForFirstAlert(users);
alertsTableShowsAssigneesBadgeForFirstAlert(users);
});

it(`alert with many assignees (collapsed into badge) in alert's details flyout`, () => {
const users = [ROLES.detections_admin, ROLES.t1_analyst, ROLES.t2_analyst];
updateAssigneesForFirstAlert(users);
expandFirstAlert();
alertDetailsFlyoutShowsAssigneesBadge(users);
});
});

context('Updating assignees (single alert)', () => {
it('adding new assignees via `More actions` in alerts table', () => {
// Assign users
const users = [ROLES.detections_admin, ROLES.t1_analyst];
const users = [getDefaultUserName()];
updateAssigneesForFirstAlert(users);

// Assignees should appear in the alerts table
Expand All @@ -129,7 +93,7 @@ describe.skip('Alert user assignment - ESS & Serverless', { tags: ['@ess', '@ser
expandFirstAlert();

// Assign users
const users = [ROLES.detections_admin, ROLES.t1_analyst];
const users = [getDefaultUserName()];
updateAssigneesViaAddButtonInFlyout(users);

// Assignees should appear in the alert's details flyout
Expand All @@ -144,7 +108,7 @@ describe.skip('Alert user assignment - ESS & Serverless', { tags: ['@ess', '@ser
expandFirstAlert();

// Assign users
const users = [ROLES.detections_admin, ROLES.t1_analyst];
const users = [getDefaultUserName()];
updateAssigneesViaTakeActionButtonInFlyout(users);

// Assignees should appear in the alert's details flyout
Expand All @@ -155,73 +119,9 @@ describe.skip('Alert user assignment - ESS & Serverless', { tags: ['@ess', '@ser
alertsTableShowsAssigneesForAlert(users);
});

it('updating assignees via `More actions` in alerts table', () => {
// Initially assigned users
const initialAssignees = [ROLES.detections_admin, ROLES.t1_analyst];
updateAssigneesForFirstAlert(initialAssignees);
alertsTableShowsAssigneesForAlert(initialAssignees);

// Update assignees
const updatedAssignees = [ROLES.t1_analyst, ROLES.t2_analyst];
updateAssigneesForFirstAlert(updatedAssignees);

const expectedAssignees = [ROLES.detections_admin, ROLES.t2_analyst];

// Expected assignees should appear in the alerts table
alertsTableShowsAssigneesForAlert(expectedAssignees);

// Expected assignees should appear in the alert's details flyout
expandFirstAlert();
alertDetailsFlyoutShowsAssignees(expectedAssignees);
});

it('updating assignees via add button in flyout', () => {
expandFirstAlert();

// Initially assigned users
const initialAssignees = [ROLES.detections_admin, ROLES.t1_analyst];
updateAssigneesViaAddButtonInFlyout(initialAssignees);
alertDetailsFlyoutShowsAssignees(initialAssignees);

// Update assignees
const updatedAssignees = [ROLES.t1_analyst, ROLES.t2_analyst];
updateAssigneesViaAddButtonInFlyout(updatedAssignees);

const expectedAssignees = [ROLES.detections_admin, ROLES.t2_analyst];

// Expected assignees should appear in the alert's details flyout
alertDetailsFlyoutShowsAssignees(expectedAssignees);

// Expected assignees should appear in the alerts table
closeAlertFlyout();
alertsTableShowsAssigneesForAlert(expectedAssignees);
});

it('updating assignees via `Take action` button in flyout', () => {
expandFirstAlert();

// Initially assigned users
const initialAssignees = [ROLES.detections_admin, ROLES.t1_analyst];
updateAssigneesViaTakeActionButtonInFlyout(initialAssignees);
alertDetailsFlyoutShowsAssignees(initialAssignees);

// Update assignees
const updatedAssignees = [ROLES.t1_analyst, ROLES.t2_analyst];
updateAssigneesViaTakeActionButtonInFlyout(updatedAssignees);

const expectedAssignees = [ROLES.detections_admin, ROLES.t2_analyst];

// Expected assignees should appear in the alert's details flyout
alertDetailsFlyoutShowsAssignees(expectedAssignees);

// Expected assignees should appear in the alerts table
closeAlertFlyout();
alertsTableShowsAssigneesForAlert(expectedAssignees);
});

it('removing all assignees via `More actions` in alerts table', () => {
// Initially assigned users
const initialAssignees = [ROLES.detections_admin, ROLES.t1_analyst];
const initialAssignees = [getDefaultUserName()];
updateAssigneesForFirstAlert(initialAssignees);
alertsTableShowsAssigneesForAlert(initialAssignees);

Expand All @@ -237,7 +137,7 @@ describe.skip('Alert user assignment - ESS & Serverless', { tags: ['@ess', '@ser
expandFirstAlert();

// Initially assigned users
const initialAssignees = [ROLES.detections_admin, ROLES.t1_analyst];
const initialAssignees = [getDefaultUserName()];
updateAssigneesViaTakeActionButtonInFlyout(initialAssignees);
alertDetailsFlyoutShowsAssignees(initialAssignees);

Expand All @@ -251,41 +151,22 @@ describe.skip('Alert user assignment - ESS & Serverless', { tags: ['@ess', '@ser
});

context('Updating assignees (bulk actions)', () => {
rylnd marked this conversation as resolved.
Show resolved Hide resolved
it('adding new assignees should be reflected in UI (alerts table and details flyout)', () => {
it('adding new assignees should be reflected in UI (alerts table)', () => {
selectFirstPageAlerts();

// Assign users
const users = [ROLES.detections_admin, ROLES.t1_analyst];
const users = [getDefaultUserName()];
bulkUpdateAssignees(users);

// Assignees should appear in the alerts table
alertsTableShowsAssigneesForAllAlerts(users);
});

it('updating assignees should be reflected in UI (alerts table and details flyout)', () => {
it('removing all assignees should be reflected in UI (alerts table)', () => {
selectFirstPageAlerts();

// Initially assigned users
const initialAssignees = [ROLES.detections_admin, ROLES.t1_analyst];
bulkUpdateAssignees(initialAssignees);
alertsTableShowsAssigneesForAllAlerts(initialAssignees);

// Update assignees
selectFirstPageAlerts();
const updatedAssignees = [ROLES.t1_analyst, ROLES.t2_analyst];
bulkUpdateAssignees(updatedAssignees);

const expectedAssignees = [ROLES.detections_admin, ROLES.t2_analyst];

// Expected assignees should appear in the alerts table
alertsTableShowsAssigneesForAllAlerts(expectedAssignees);
});

it('removing all assignees should be reflected in UI (alerts table and details flyout)', () => {
selectFirstPageAlerts();

// Initially assigned users
const initialAssignees = [ROLES.detections_admin, ROLES.t1_analyst];
const initialAssignees = [getDefaultUserName()];
bulkUpdateAssignees(initialAssignees);
alertsTableShowsAssigneesForAllAlerts(initialAssignees);

Expand All @@ -303,7 +184,7 @@ describe.skip('Alert user assignment - ESS & Serverless', { tags: ['@ess', '@ser
const totalNumberOfAlerts = 5;
const numberOfSelectedAlerts = 2;
selectNumberOfAlerts(numberOfSelectedAlerts);
bulkUpdateAssignees([ROLES.t1_analyst]);
bulkUpdateAssignees([getDefaultUserName()]);

filterByAssignees([NO_ASSIGNEES]);

Expand All @@ -314,41 +195,20 @@ describe.skip('Alert user assignment - ESS & Serverless', { tags: ['@ess', '@ser
it('by one assignee', () => {
const numberOfSelectedAlerts = 2;
selectNumberOfAlerts(numberOfSelectedAlerts);
bulkUpdateAssignees([ROLES.t1_analyst]);
bulkUpdateAssignees([getDefaultUserName()]);

filterByAssignees([ROLES.t1_analyst]);
filterByAssignees([getDefaultUserName()]);

cy.get(ALERTS_COUNT).contains(numberOfSelectedAlerts);
});

it('by multiple assignees', () => {
const numberOfSelectedAlerts1 = 1;
selectNumberOfAlerts(numberOfSelectedAlerts1);
bulkUpdateAssignees([ROLES.t1_analyst]);

filterByAssignees([NO_ASSIGNEES]);

const numberOfSelectedAlerts2 = 2;
selectNumberOfAlerts(numberOfSelectedAlerts2);
bulkUpdateAssignees([ROLES.detections_admin]);

clearAssigneesFilter();

cy.get(ALERTS_COUNT).contains(5);

filterByAssignees([ROLES.t1_analyst, ROLES.detections_admin]);

const expectedNumberOfAlerts = numberOfSelectedAlerts1 + numberOfSelectedAlerts2;
cy.get(ALERTS_COUNT).contains(expectedNumberOfAlerts);
});

it('by assignee and alert status', () => {
const totalNumberOfAlerts = 5;
const numberOfAssignedAlerts = 3;
selectNumberOfAlerts(numberOfAssignedAlerts);
bulkUpdateAssignees([ROLES.t1_analyst]);
bulkUpdateAssignees([getDefaultUserName()]);

filterByAssignees([ROLES.t1_analyst]);
filterByAssignees([getDefaultUserName()]);

const numberOfClosedAlerts = 1;
selectNumberOfAlerts(numberOfClosedAlerts);
Expand All @@ -362,7 +222,7 @@ describe.skip('Alert user assignment - ESS & Serverless', { tags: ['@ess', '@ser
const expectedNumberOfAllerts2 = totalNumberOfAlerts - numberOfClosedAlerts;
cy.get(ALERTS_COUNT).contains(expectedNumberOfAllerts2);

filterByAssignees([ROLES.t1_analyst]);
filterByAssignees([getDefaultUserName()]);
selectPageFilterValue(0, 'closed');
cy.get(ALERTS_COUNT).contains(numberOfClosedAlerts);
});
Expand Down
Loading