Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Security Solution] Flaky tests #153667

Closed
15 tasks done
Tracked by #153633
, #153664
...
MadameSheema opened this issue Mar 24, 2023 · 7 comments
Closed
15 tasks done
Tracked by #153633
, #153664
...

[Security Solution] Flaky tests #153667

MadameSheema opened this issue Mar 24, 2023 · 7 comments
Labels
8.10 candidate Meta Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:Threat Hunting Security Solution Threat Hunting Team

Comments

@MadameSheema
Copy link
Member

MadameSheema commented Mar 24, 2023 •
edited
Loading

As part of the #153664 effort, we need to address all the flakiness we have. This ticket is going to centralize all the flaky tests found thanks to the kibana-unsupported-ftr-alerts slack channel and other that may encounter.

DETECTIONS AND RESPONSE
Preview Give feedback
  1. [Security Solution][Flaky] /x-pack/plugins/security_solution/cypress/e2e/detection_rules/related_integrations.cy.ts #153684
    8.10 candidate Team: SecuritySolution Team:Detection Rule Management Team:Detections and Resp failed-test test test-failure-flaky
    maximpn
  2. [Security Solution] Unskip flaky Cypress tests for Related Integrations #154663
    8.10 candidate Feature:Related Integrations Team: SecuritySolution Team:Detection Rule Management Team:Detections and Resp impact:high skipped-test test test_ui_functional
    maximpn
  3. [Security Solution] Unskip flaky Cypress tests for the Rules table #154694
    8.10 candidate Feature:Rule Management Team: SecuritySolution Team:Detection Rule Management Team:Detections and Resp impact:high skipped-test test test_ui_functional
    maximpn
  4. [Security Solution] Unskip flaky Cypress tests for bulk editing rule actions #154721
    8.10 candidate Feature:Rule Management Team: SecuritySolution Team:Detection Rule Management Team:Detections and Resp impact:high skipped-test test test_ui_functional
    jpdjere
  5. [Security Solution][Flaky] /x-pack/plugins/security_solution/cypress/e2e/detection_rules/bulk_edit_rules_data_view.cy.ts/Bulk editing index patterns of rules with index patterns and rules with a data view -- Add index patterns to custom rules one rule is updated, one rule is skipped #153914
    8.10 candidate Team: SecuritySolution Team:Detection Rule Management Team:Detections and Resp failed-test test test-failure-flaky
    maximpn
  6. [Security Solution] [Flaky] /x-pack/plugins/security_solution/cypress/e2e/rule_details_flow/add_edit_exception_data_view.cy.ts/Add exception using data views from rule details -- Creates an exception item from alert actions overflow menu #153758
    Team: SecuritySolution Team:Detection Engine Team:Detections and Resp failed-test test-failure-flaky
  7. [Security Solution][Flaky] /x-pack/plugins/security_solution/cypress/e2e/exceptions/shared_exception_lists_management/manage_shared_exception_list.cy.ts/Manage shared exception list -- Export exception list #153779
    Team: SecuritySolution Team:Detection Engine Team:Detections and Resp failed-test test-failure-flaky
  8. [Security Solution][Flaky] /x-pack/plugins/security_solution/cypress/e2e/exceptions/shared_exception_lists_management/manage_shared_exception_list.cy.ts/Manage shared exception list -- Deletes exception list with rule reference #153780
    Team: SecuritySolution Team:Detection Engine Team:Detections and Resp failed-test test-failure-flaky
  9. [Security Solution][Flaky] /x-pack/plugins/security_solution/cypress/e2e/exceptions/shared_exception_lists_management/manage_shared_exception_list.cy.ts/Manage shared exception list -- Delete exception list without rule reference #153781
    Team: SecuritySolution Team:Detection Engine Team:Detections and Resp failed-test test-failure-flaky

THREAT HUNTING
Preview Give feedback
  1. [Security Solution][Flaky test] /x-pack/plugins/security_solution/cypress/e2e/detection_alerts/detection_page_filters.cy.ts -- Alert Page Filters Customization -- Add New Controls #153685
    Team: SecuritySolution Team:Threat Hunting Team:Threat Hunting:Investigations fixed
    logeekal
  2. [Security Solution][Flaky test] /x-pack/plugins/security_solution/cypress/e2e/detection_alerts/detection_page_filters.cy.ts/Detections Page Filters -- Alert Page Filters Customization -- should not sync to the URL in edit mode but only in view mode #153686
    Team: SecuritySolution Team:Threat Hunting Team:Threat Hunting:Investigations fixed
    logeekal
  3. [Security Solution][Flaky] /x-pack/plugins/security_solution/cypress/e2e/timelines/search_or_filter.cy.ts/Timeline search and filters -- Update kqlMode for timeline -- should be able to update timeline kqlMode with filter -- before each hook #153761
    Team: SecuritySolution Team:Threat Hunting Team:Threat Hunting:Investigations failed-test test-failure-flaky
  4. [Security Solution][Flaky] /x-pack/plugins/security_solution/cypress/e2e/inspect/inspect_button.cy.ts/Inspect Explore pages -- inspect Network page #153765
    Team: SecuritySolution Team:Threat Hunting failed-test fixed technical debt test-failure-flaky
    angorayc
  5. [Security Solution][Flaky] /x-pack/plugins/security_solution/cypress/e2e/timelines/row_renderers.cy.ts/Row renderers -- Selected renderer can be disabled and enabled #153767
    Team: SecuritySolution Team:Threat Hunting failed-test fixed technical debt test-failure-flaky
    angorayc
  6. [Security Solution][Flaky] /x-pack/plugins/security_solution/cypress/e2e/timelines/flyout_button.cy.ts/timeline flyout button -- the (+) button popover menu owns focus #153771
    Team: SecuritySolution Team:Threat Hunting test-failure-flaky
@MadameSheema MadameSheema added Team:Detections and Resp Security Detection Response Team Team:Threat Hunting Security Solution Threat Hunting Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. labels Mar 24, 2023
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-threat-hunting (Team:Threat Hunting)

@elasticmachine
Copy link
Contributor

Pinging @elastic/security-detections-response (Team:Detections and Resp)

@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

@MadameSheema MadameSheema mentioned this issue Mar 24, 2023
Closed
12 tasks
@banderror
Copy link
Contributor

@MadameSheema In this build the following tests failed:

  • timelines/search_or_filter.cy.ts
  • exceptions/rule_details_flow/add_edit_exception_data_view.cy.ts

I see the second one not for the first time and I believe it's a flaky one

@banderror banderror mentioned this issue Mar 24, 2023
Open
@banderror
Copy link
Contributor

@MadameSheema I added a bunch of new tests to the description

@MadameSheema MadameSheema mentioned this issue May 2, 2023
Closed
@banderror
Copy link
Contributor

@MadameSheema are you actively tracking flaky tests? At least for D&R, the list of tickets looks out of date.

We use the following tickets and GH searches to track them:

@banderror
Copy link
Contributor

All the tickets tracked in this meta ticket have been closed. I'm closing this one, but feel free to reopen and update it if needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
8.10 candidate Meta Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:Threat Hunting Security Solution Threat Hunting Team
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@banderror @elasticmachine @MadameSheema