Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution] [EPIC] Flaky tests of Security solution #156351

Closed
logeekal opened this issue May 2, 2023 · 4 comments
Closed

[Security Solution] [EPIC] Flaky tests of Security solution #156351

logeekal opened this issue May 2, 2023 · 4 comments
Assignees
Labels
bug Fixes for quality problems that affect the customer experience Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. triage_needed

Comments

@logeekal
Copy link
Contributor

logeekal commented May 2, 2023

Summary

Recently, I triggered a flaky_test_runner build of Security solution Cypress tests which can be seen here. Please login in BuildKite to view complete summary of test faliures. Flaky test runner ran Security Solution Tests for around 200 times and below is the count of test faliures.

Build: https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2188

As the partial resolution of this Epic issue, I will be disabling all the tests with failure rate above 10% i.e. 20 failures out of 200 runs and will be creating separate issue to track all the failed tests.

This Epic issue can be used to keep track of our progress.

Test Faliures

Test Count
Detection rules, machine learning Creates and enables a new ml rule 196
Timeline query tab Query tab "before all" hook for "should contain the right query" 85
Detections : Page Filters Changed banner should hide on Reset 52
Detections : Page Filters "after each" hook for "Changed banner should hide on Reset" 52
Open timeline Open timeline modal "before all" hook for "should open a modal" 45
Timeline query tab Query tab should be able to add event note 19
Detection rules, bulk edit Timeline templates Apply timeline template to custom rules 19
timeline data providers presists timeline when a field is added by hover action "Add To Timeline" in data provider 16
Timeline search and filters Update kqlMode for timeline should be able to update timeline kqlMode with filter 15
Inspect Explore pages "before all" hook for "inspect Hosts page" 7
Cases Creates a new case with timeline and opens the timeline 5
timeline flyout button the (+) button popover menu owns focus 3
Timeline search and filters Update kqlMode for timeline should be able to update timeline kqlMode with search 2
Exceptions flyout Validates auto-suggested fields correctly 1
@logeekal logeekal added bug Fixes for quality problems that affect the customer experience triage_needed Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. labels May 2, 2023
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

@logeekal logeekal self-assigned this May 2, 2023
@MadameSheema
Copy link
Member

@logeekal please, track it on the original issue: #153667 thanks

@logeekal
Copy link
Contributor Author

logeekal commented May 2, 2023

Thank you.. Good to know. I will close this issue that case.

@logeekal
Copy link
Contributor Author

logeekal commented May 2, 2023

Closing because it is a duplicate issue.

@logeekal logeekal closed this as completed May 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Fixes for quality problems that affect the customer experience Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. triage_needed
Projects
None yet
Development

No branches or pull requests

3 participants