[Security Solution][Threat Hunting] - Unskip threat hunting tests #195662

michaelolo24 · 2024-10-09T18:10:42Z

Added cypress-network-idle to better wait for a full page load as a loading indicator flicker could cause flakiness.
List of addressed open issues:

Resolves #191819
Resolves #178367
Resolves #176757
Resolves #176759
Resolves #176758
Resolves #193840
Resolves #182206
Resolves #181977
Resolves #168768
Resolves #168769
Resolves #168770
Resolves #168771
Resolves #168772
Resolves #183104
Resolves #173854

Successful Flaky Test Runner: https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7267

michaelolo24 · 2024-10-24T01:39:22Z

@patrykkopycinski - Anything we have to do regarding upgrading cypress as I believe you've gone through this process before? Thanks!

x-pack/test/security_solution_cypress/cypress/screens/inspect.ts

michaelolo24 · 2024-10-24T02:50:55Z

...test/security_solution_cypress/cypress/e2e/investigations/alerts/changing_alert_status.cy.ts

@@ -342,7 +338,7 @@ describe.skip('Changing alert status', { tags: ['@ess', '@skipInServerless'] },
        });
    });

-    it('Closes alerts', () => {
+    it.skip('Closes alerts', () => {


t2 Analyst seems to have the ability to close alerts here. @MadameSheema is this behavior accurate?

Solved it in an async way, seems that the t2 analyst has the privilege to close alerts.

Are we going to maintain this test skipped?

Will make it ess only with Roles.reader

michaelolo24 · 2024-10-24T02:52:16Z

package.json

@@ -1687,10 +1687,11 @@
    "cssnano": "^5.1.12",
    "cssnano-preset-default": "^5.2.12",
    "csstype": "^3.0.2",
-    "cypress": "13.6.3",
+    "cypress": "13.15.0",


Updated Cypress due to an issue where cypress tests hang locally when the browser is not focused. Reference: cypress-io/cypress#28392 .

For future reference, Cypress updates should be done in a different PR since there are more solutions a part from us that are using Cypress as well and they might be affected by this change.

Before merging this PR and to make sure that this is not impacting other solutions, please make sure that the ci:all-cypress-suites label and you update your branch with the latest main changes to trigger the execution of all the different Cypress tests of the Kibana repo.

I'll open a separate PR with just these changes! No worries 😄

Done: #198715, will wait on that making it in first 👍🏾

michaelolo24 · 2024-10-24T02:53:09Z

package.json

    "cypress-axe": "^1.5.0",
    "cypress-file-upload": "^5.0.8",
    "cypress-multi-reporters": "^1.6.4",
+    "cypress-network-idle": "^1.14.2",


Instead of relying on the global loading indicator being present for the alerts based tests, we'll instead wait for network requests to complete on the page (this helps avoid flakiness due to any loading indicator flicker)

michaelolo24 · 2024-10-24T18:14:46Z

/ci

michaelolo24 · 2024-10-24T18:15:31Z

/ci

MadameSheema · 2024-10-30T09:45:27Z

.../test/security_solution_cypress/cypress/e2e/investigations/alerts/alert_table_controls.cy.ts

    cy.get(GET_DATA_GRID_HEADER_CELL_ACTION_GROUP(timestampField))
      .should('be.visible')
      .should('contain.text', 'Sort Old-New');

    cy.log('Number column');

    const riskScoreField = DATA_GRID_FIELDS.RISK_SCORE.fieldName;
-    cy.get(GET_DATA_GRID_HEADER(riskScoreField)).trigger('click');
+    cy.get(GET_DATA_GRID_HEADER_ACTION_BUTTON(riskScoreField)).realHover().trigger('click');


Same as: https://github.com/elastic/kibana/pull/195662/files#r1822236029

MadameSheema · 2024-10-30T09:45:47Z

.../test/security_solution_cypress/cypress/e2e/investigations/alerts/alert_table_controls.cy.ts

    cy.get(GET_DATA_GRID_HEADER_CELL_ACTION_GROUP(riskScoreField))
      .should('be.visible')
      .should('contain.text', 'Sort Low-High');

    cy.log('Text Column');

    const ruleField = DATA_GRID_FIELDS.RULE.fieldName;
-    cy.get(GET_DATA_GRID_HEADER(ruleField)).trigger('click');
+    cy.get(GET_DATA_GRID_HEADER_ACTION_BUTTON(ruleField)).realHover().trigger('click');


Same as: https://github.com/elastic/kibana/pull/195662/files#r1822236029

MadameSheema · 2024-10-30T10:11:33Z

...k/test/security_solution_cypress/cypress/e2e/investigations/alerts/alerts_cell_actions.cy.ts

  fillAddFilterForm,
  fillKqlQueryBar,
  openAddFilterPopover,
-  removeKqlFilter,
 } from '../../../tasks/search_bar';
 import { openActiveTimeline } from '../../../tasks/timeline';

 import { ALERTS_URL } from '../../../urls/navigation';

 // Failing: See https://github.com/elastic/kibana/issues/193840


Is there any reason why the comment has not been removed?

just missed this one. Removing it in my next commit 😄

why next PR? This one is outdated so you're going to have to rebase/merge main so the build is going to have to run again anyway... Might as well make this change now?

Miswrote...my next commit* I have it on my local branch now

Fixed. Now you can't prove it said next PR lol 😁

haha, are you sure? 😝

fake news...lol

angorayc

Thanks for the fix @michaelolo24

MadameSheema

Lots of thanks for investing the time on unskipping tests!! :)

Please make sure that #198715 is merged first!

michaelolo24 · 2024-11-04T20:21:14Z

Thanks all! Just waiting on this to update this PR and merge

…-fix'

elasticmachine · 2024-11-07T16:25:36Z

💚 Build Succeeded

Buildkite Build
Commit: d4e4c65

Metrics [docs]

✅ unchanged

History

💛 Build #247848 was flaky d14164b
💔 Build #247822 failed 61d2453
💛 Build #246558 was flaky 98ed313
💔 Build #246442 failed c7f7f9f
💔 Build #246357 failed c157f51bd2ccc1e6164abf54392a6db3d3787e76
💚 Build #246031 succeeded 52b802866f74afe89bfff806a57070ac1c8394e0

1. Added `cypress-network-idle` to better wait for a full page load as a loading indicator flicker could cause flakiness. 2. List of addressed open issues: Resolves elastic#191819 Resolves elastic#178367 Resolves elastic#176757 Resolves elastic#176759 Resolves elastic#176758 Resolves elastic#193840 Resolves elastic#182206 Resolves elastic#181977 Resolves elastic#168768 Resolves elastic#168769 Resolves elastic#168770 Resolves elastic#168771 Resolves elastic#168772 Resolves elastic#183104 Resolves elastic#173854 Successful Flaky Test Runner: https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7267 --------- Co-authored-by: kibanamachine <[email protected]>

michaelolo24 · 2024-11-26T18:42:25Z

💚 All backports created successfully

Status	Branch	Result
✅	8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

1. Added `cypress-network-idle` to better wait for a full page load as a loading indicator flicker could cause flakiness. 2. List of addressed open issues: Resolves elastic#191819 Resolves elastic#178367 Resolves elastic#176757 Resolves elastic#176759 Resolves elastic#176758 Resolves elastic#193840 Resolves elastic#182206 Resolves elastic#181977 Resolves elastic#168768 Resolves elastic#168769 Resolves elastic#168770 Resolves elastic#168771 Resolves elastic#168772 Resolves elastic#183104 Resolves elastic#173854 Successful Flaky Test Runner: https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7267 --------- Co-authored-by: kibanamachine <[email protected]> (cherry picked from commit 3f236b1)

michaelolo24 added release_note:skip Skip the PR/issue when compiling release notes v9.0.0 backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) v8.16.0 labels Oct 9, 2024

michaelolo24 force-pushed the serverless-test-readiness branch 2 times, most recently from b9e33f1 to 13c99ca Compare October 11, 2024 14:32

michaelolo24 force-pushed the serverless-test-readiness branch from 13c99ca to 1235496 Compare October 22, 2024 15:31

michaelolo24 mentioned this pull request Oct 22, 2024

[Security Solution][Serverless] Automation readiness for GA #189468

Open

michaelolo24 force-pushed the serverless-test-readiness branch 5 times, most recently from 4c272d4 to cf970f5 Compare October 23, 2024 23:23

elastic deleted a comment from kibanamachine Oct 23, 2024

elastic deleted a comment from elasticmachine Oct 23, 2024

michaelolo24 requested a review from patrykkopycinski October 24, 2024 01:38

michaelolo24 commented Oct 24, 2024

View reviewed changes

x-pack/test/security_solution_cypress/cypress/screens/inspect.ts Outdated Show resolved Hide resolved

michaelolo24 commented Oct 24, 2024

View reviewed changes

x-pack/test/security_solution_cypress/cypress/screens/inspect.ts Show resolved Hide resolved

michaelolo24 commented Oct 24, 2024

View reviewed changes

x-pack/test/security_solution_cypress/cypress/screens/inspect.ts Outdated Show resolved Hide resolved

michaelolo24 commented Oct 24, 2024

View reviewed changes

michaelolo24 added v8.17.0 backport:version Backport to applied version labels and removed v8.16.0 backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) labels Oct 24, 2024

MadameSheema reviewed Oct 30, 2024

View reviewed changes

michaelolo24 force-pushed the serverless-test-readiness branch 2 times, most recently from 61d2453 to b53a7ed Compare November 1, 2024 21:12

angorayc approved these changes Nov 4, 2024

View reviewed changes

MadameSheema approved these changes Nov 4, 2024

View reviewed changes

michaelolo24 and others added 3 commits November 7, 2024 09:22

unskip threat hunting tests

11e6e5b

[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…

b40fcf4

…-fix'

revert version upgrade

d4e4c65

michaelolo24 force-pushed the serverless-test-readiness branch from d14164b to d4e4c65 Compare November 7, 2024 14:33

michaelolo24 merged commit 3f236b1 into elastic:main Nov 7, 2024
41 checks passed

michaelolo24 changed the title ~~[WIP] - Unskip threat hunting tests~~ [Security Solution][Threat Hunting] - Unskip threat hunting tests Nov 7, 2024

This was referenced Nov 7, 2024

[Security Solution][Data Quality Dashboard][Serverless] add start/end time support for latest_results #199248

Merged

Changes deprecated SO HTTP APIs deprecation field to object #197936

Merged

This was referenced Nov 12, 2024

[IndexAdapter] Extract index-adapter package from data-stream-adapter #199575

Merged

Update ftr (main) #199090

Merged

This was referenced Nov 25, 2024

Update dependency chromedriver to v131 (main) #201053

Merged

[Cloud Security][D4C] Hide K8S #201062

Merged

Update OpenAPI Spec (main) #201609

Merged

michaelolo24 mentioned this pull request Nov 26, 2024

[8.x] [WIP] - Unskip threat hunting tests (#195662) #201853

Merged

kibanamachine mentioned this pull request Nov 26, 2024

feat: [Dashboards] Hide clear control button in floating actions when no filters are selected #200177

Merged

kibanamachine added the v8.18.0 label Nov 27, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Security Solution][Threat Hunting] - Unskip threat hunting tests #195662

[Security Solution][Threat Hunting] - Unskip threat hunting tests #195662

michaelolo24 commented Oct 9, 2024 •

edited

Loading

michaelolo24 commented Oct 24, 2024 •

edited

Loading

michaelolo24 Oct 24, 2024

MadameSheema Oct 30, 2024

MadameSheema Oct 30, 2024

michaelolo24 Nov 1, 2024 •

edited

Loading

michaelolo24 Oct 24, 2024

MadameSheema Oct 30, 2024

michaelolo24 Nov 1, 2024

michaelolo24 Nov 1, 2024

michaelolo24 Oct 24, 2024

michaelolo24 commented Oct 24, 2024

michaelolo24 commented Oct 24, 2024

MadameSheema Oct 30, 2024

MadameSheema Oct 30, 2024

MadameSheema Oct 30, 2024

michaelolo24 Nov 1, 2024 •

edited

Loading

PhilippeOberti Nov 1, 2024

michaelolo24 Nov 1, 2024

michaelolo24 Nov 1, 2024

PhilippeOberti Nov 1, 2024

michaelolo24 Nov 1, 2024

angorayc left a comment

MadameSheema left a comment

michaelolo24 commented Nov 4, 2024

elasticmachine commented Nov 7, 2024

michaelolo24 commented Nov 26, 2024

[Security Solution][Threat Hunting] - Unskip threat hunting tests #195662

[Security Solution][Threat Hunting] - Unskip threat hunting tests #195662

Conversation

michaelolo24 commented Oct 9, 2024 • edited Loading

michaelolo24 commented Oct 24, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

michaelolo24 Nov 1, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

michaelolo24 commented Oct 24, 2024

michaelolo24 commented Oct 24, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

michaelolo24 Nov 1, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

angorayc left a comment

Choose a reason for hiding this comment

MadameSheema left a comment

Choose a reason for hiding this comment

michaelolo24 commented Nov 4, 2024

elasticmachine commented Nov 7, 2024

💚 Build Succeeded

Metrics [docs]

History

michaelolo24 commented Nov 26, 2024

💚 All backports created successfully

Questions ?

michaelolo24 commented Oct 9, 2024 •

edited

Loading

michaelolo24 commented Oct 24, 2024 •

edited

Loading

michaelolo24 Nov 1, 2024 •

edited

Loading

michaelolo24 Nov 1, 2024 •

edited

Loading