Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SecuritySolution] Fix flaky timeline template creation tests #173206

Merged
merged 9 commits into from
Dec 14, 2023
Merged

[SecuritySolution] Fix flaky timeline template creation tests #173206

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
c14d94f
you cannot pin events in templates
janmonschke Dec 12, 2023
6515df5
fix flaky timeline template tests
janmonschke Dec 12, 2023
6b314df
flaky test runner config
janmonschke Dec 12, 2023
f27ba63
revert flaky test runner changes
janmonschke Dec 13, 2023
2f23f20
remove unnecessary `createTimeline`
janmonschke Dec 13, 2023
da5e697
Add logs
janmonschke Dec 13, 2023
dc8ec9d
Merge branch 'main' into security/fix-flaky-timeline-template-creatio…
janmonschke Dec 13, 2023
c7c270d
remove comment
janmonschke Dec 13, 2023
1048030
Merge branch 'main' into security/fix-flaky-timeline-template-creatio…
kibanamachine Dec 14, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion x-pack/plugins/security_solution/public/timelines/components/timeline/pin/index.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ export const Pin = React.memo<Props>(
isTemplate,
isPinned: pinned,
});
const pinAriaLabel = ariaLabel != null ? ariaLabel : defaultAriaLabel;
const pinAriaLabel = ariaLabel != null && !isTemplate ? ariaLabel : defaultAriaLabel;

return (
<EuiButtonIcon
Expand Down
71 changes: 34 additions & 37 deletions ...st/security_solution_cypress/cypress/e2e/investigations/timeline_templates/creation.cy.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,98 +8,95 @@
import { getTimeline } from '../../../objects/timeline';

import {
FAVORITE_TIMELINE,
LOCKED_ICON,
NOTES,
NOTES_TAB_BUTTON,
NOTES_TEXT_AREA,
PIN_EVENT,
TIMELINE_DESCRIPTION,
TIMELINE_FLYOUT_WRAPPER,
TIMELINE_QUERY,
TIMELINE_TITLE,
TIMELINE_DATE_PICKER_CONTAINER,
} from '../../../screens/timeline';
import {
TIMELINES_DESCRIPTION,
TIMELINES_PINNED_EVENT_COUNT,
TIMELINES_NOTES_COUNT,
TIMELINES_FAVORITE,
} from '../../../screens/timelines';
import { TIMELINES_DESCRIPTION, TIMELINES_FAVORITE } from '../../../screens/timelines';
import { createTimeline } from '../../../tasks/api_calls/timelines';
import { deleteTimelines } from '../../../tasks/api_calls/common';

import { login } from '../../../tasks/login';
import { visit } from '../../../tasks/navigation';
import { openTimelineUsingToggle } from '../../../tasks/security_main';
import {
addDescriptionToTimeline,
addFilter,
addNameToTimelineAndSave,
addNotesToTimeline,
clickingOnCreateTemplateFromTimelineBtn,
closeTimeline,
createNewTimelineTemplate,
createTimelineTemplateOptionsPopoverBottomBar,
expandEventAction,
markAsFavorite,
openTimelineTemplateFromSettings,
openTimelineTemplate,
populateTimeline,
addNameAndDescriptionToTimeline,
openTimelineTemplatesTab,
} from '../../../tasks/timeline';
import { openTimeline, waitForTimelinesPanelToBeLoaded } from '../../../tasks/timelines';
import {
updateTimelineDates,
showStartEndDate,
setStartDate,
setEndDateNow,
} from '../../../tasks/date_picker';
import { waitForTimelinesPanelToBeLoaded } from '../../../tasks/timelines';

import { TIMELINES_URL } from '../../../urls/navigation';

// FLAKY: https://github.com/elastic/kibana/issues/165661
describe('Timeline Templates', { tags: ['@ess', '@serverless'] }, () => {
beforeEach(() => {
login();
deleteTimelines();
cy.intercept('PATCH', '/api/timeline').as('timeline');
});

it.skip('Creates a timeline template', () => {
it('Creates a timeline template', () => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I get the purpose of this test to kind of test the majority of the timeline template functionality, but do you think it's worth breaking this apart to

describe('Timeline templates', { tags: ['@ess', '@serverless'] }, () => {
 test('should be able to create a template', () => {});
  test('should be able to be save a template', () => { }); 
  test('should be able to update a template', () => {});
});

or something similarly

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would rather test this in one test so that we don't create dependencies between tests or duplicate a lot of the creation code.

To make more sense of the logic and in order to make it easier to debug on the CI, I added a couple of cy.logs. Let me know if that's better. da5e697

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes sense and I understand why. In this scenario I would have suggested putting the creation in a beforeAll and then still breaking the tests apart, but with cypress not running the beforeAll on retries could make those flaky and not make true independent tests. All that being said, this may be the best compromise.

visit(TIMELINES_URL);
waitForTimelinesPanelToBeLoaded();
openTimelineUsingToggle();

createNewTimelineTemplate();
populateTimeline();

cy.log('Add filter');
addFilter(getTimeline().filter);

cy.log('Update date range');
showStartEndDate(TIMELINE_DATE_PICKER_CONTAINER);
setEndDateNow(TIMELINE_DATE_PICKER_CONTAINER);
setStartDate('Jan 18, 2018 @ 00:00:00.000', TIMELINE_DATE_PICKER_CONTAINER);
updateTimelineDates();

cy.log('Try to pin an event');
cy.get(PIN_EVENT).should(
'have.attr',
'aria-label',
'This event may not be pinned while editing a template timeline'
);
cy.get(LOCKED_ICON).should('be.visible');

addNameToTimelineAndSave(getTimeline().title);
cy.log('Update title and description');
addNameAndDescriptionToTimeline(getTimeline());

cy.wait('@timeline').then(({ response }) => {
const timelineId = response?.body.data.persistTimeline.timeline.savedObjectId;

addDescriptionToTimeline(getTimeline().description);
addNotesToTimeline(getTimeline().notes);
markAsFavorite();
createNewTimelineTemplate();
closeTimeline();
openTimelineTemplateFromSettings(timelineId);

cy.log('Open template from templates tab');
openTimelineTemplatesTab();
openTimelineTemplate(timelineId);

cy.log('Check that the template has been created correclty');
cy.contains(getTimeline().title).should('exist');
cy.get(TIMELINE_TITLE).should('have.text', getTimeline().title);
cy.get(TIMELINES_DESCRIPTION).first().should('have.text', getTimeline().description);
cy.get(TIMELINES_PINNED_EVENT_COUNT).first().should('have.text', '1');
cy.get(TIMELINES_NOTES_COUNT).first().should('have.text', '1');
cy.get(TIMELINES_FAVORITE).first().should('exist');

openTimeline(timelineId);

cy.get(FAVORITE_TIMELINE).should('exist');
cy.get(TIMELINE_TITLE).should('have.text', getTimeline().title);
cy.get(TIMELINE_DESCRIPTION).should('have.text', getTimeline().description);
cy.get(TIMELINE_QUERY).should('have.text', getTimeline().query);
// Comments this assertion until we agreed what to do with the filters.
// cy.get(TIMELINE_FILTER(timeline.filter)).should('exist');
// cy.get(NOTES_COUNT).should('have.text', '1');
cy.get(NOTES_TAB_BUTTON).click();
cy.get(NOTES_TEXT_AREA).should('exist');
cy.get(NOTES).should('have.text', getTimeline().notes);
cy.get(TIMELINE_QUERY).should('contain.text', getTimeline().query);
});
});

Expand Down
3 changes: 0 additions & 3 deletions x-pack/test/security_solution_cypress/cypress/screens/timeline.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -108,9 +108,6 @@ export const OPEN_TIMELINE_MODAL = '[data-test-subj="open-timeline-modal"]';

export const CLOSE_OPEN_TIMELINE_MODAL_BTN = `${OPEN_TIMELINE_MODAL} > button`;

export const OPEN_TIMELINE_TEMPLATE_ICON =
'[data-test-subj="open-timeline-modal-body-filter-template"]';

export const PIN_EVENT = '[data-test-subj="pin"]';

export const PINNED_TAB_BUTTON = '[data-test-subj="timelineTabs-pinned"]';
Expand Down
2 changes: 2 additions & 0 deletions x-pack/test/security_solution_cypress/cypress/screens/timelines.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,3 +61,5 @@ export const TIMELINES_OVERVIEW_SEARCH = `${TIMELINES_OVERVIEW} [data-test-subj=
export const TIMELINES_OVERVIEW_TABLE = `${TIMELINES_OVERVIEW} [data-test-subj="timelines-table"]`;

export const ROWS = '.euiTableRow';

export const TIMELINES_TAB_TEMPLATE = '[data-test-subj="timeline-tab-template"]';
15 changes: 8 additions & 7 deletions x-pack/test/security_solution_cypress/cypress/tasks/timeline.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,6 @@ import {
TIMESTAMP_TOGGLE_FIELD,
TOGGLE_TIMELINE_EXPAND_EVENT,
CREATE_NEW_TIMELINE_TEMPLATE,
OPEN_TIMELINE_TEMPLATE_ICON,
TIMELINE_SAVE_MODAL,
TIMELINE_EDIT_MODAL_SAVE_BUTTON,
TIMELINE_EDIT_MODAL_SAVE_AS_NEW_SWITCH,
Expand Down Expand Up @@ -93,7 +92,8 @@ import {
TOGGLE_DATA_PROVIDER_BTN,
SAVE_TIMELINE_ACTION_BTN,
} from '../screens/timeline';
import { REFRESH_BUTTON, TIMELINE } from '../screens/timelines';

import { REFRESH_BUTTON, TIMELINE, TIMELINES_TAB_TEMPLATE } from '../screens/timelines';
import { drag, drop, waitForTabToBeLoaded } from './common';

import { closeFieldsBrowser, filterFieldsBrowser } from './fields_browser';
Expand Down Expand Up @@ -203,7 +203,6 @@ export const addFilter = (filter: TimelineFilter): Cypress.Chainable<JQuery<HTML
cy.get(ADD_FILTER).click();
cy.get(TIMELINE_FILTER_FIELD).type(`${filter.field}{downarrow}{enter}`);
cy.get(TIMELINE_FILTER_OPERATOR).type(`${filter.operator}{downarrow}{enter}`);

if (filter.operator !== 'exists') {
cy.get(TIMELINE_FILTER_VALUE).type(`${filter.value}{enter}`);
}
Expand Down Expand Up @@ -418,10 +417,12 @@ export const openTimelineFromSettings = () => {
cy.get(OPEN_TIMELINE_ICON).click();
};

export const openTimelineTemplateFromSettings = (id: string) => {
openTimelineFromSettings();
cy.get(OPEN_TIMELINE_TEMPLATE_ICON).click({ force: true });
cy.get(TIMELINE_TITLE_BY_ID(id)).click({ force: true });
export const openTimelineTemplate = (id: string) => {
cy.get(TIMELINE_TITLE_BY_ID(id)).click();
};

export const openTimelineTemplatesTab = () => {
cy.get(TIMELINES_TAB_TEMPLATE).click();
};

export const openTimelineById = (timelineId: string): Cypress.Chainable<JQuery<HTMLElement>> => {
Expand Down
11 changes: 10 additions & 1 deletion x-pack/test/security_solution_cypress/cypress/tasks/timelines.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,11 @@ import {
TIMELINE_ITEM_ACTION_BTN,
} from '../screens/timelines';
import { SELECT_ALL_CHECKBOX } from '../screens/shared';
import { CREATE_NEW_TIMELINE_WITH_BORDER } from '../screens/timeline';
import {
CREATE_NEW_TIMELINE_WITH_BORDER,
TIMELINE_COLLAPSED_ITEMS_BTN,
TIMELINE_CREATE_TIMELINE_FROM_TEMPLATE_BTN,
} from '../screens/timeline';

export const expandNotes = () => {
cy.get(EXPAND_NOTES_BTN).click();
Expand Down Expand Up @@ -67,3 +71,8 @@ export const exportSelectedTimelines = () => {

export const createTimeline = () =>
cy.get(CREATE_NEW_TIMELINE_WITH_BORDER).should('be.visible').click();

export const createTimelineFromFirstTemplateInList = () => {
cy.get(TIMELINE_COLLAPSED_ITEMS_BTN).first().click();
cy.get(TIMELINE_CREATE_TIMELINE_FROM_TEMPLATE_BTN).click();
};
Loading