Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failing test: Security Solution Cypress.x-pack/plugins/security_solution/cypress/e2e/detection_alerts/missing_privileges_callout·cy·ts - Detections > Callouts indicating read-only access to resources On Rule Details page When a user clicks Dismiss on the callouts We hide them and persist the dismissal We hide them and persist the dismissal #163885

Closed
kibanamachine opened this issue Aug 14, 2023 · 4 comments
Assignees
Labels
failed-test A test failure on a tracked branch, potentially flaky-test Team:Detection Engine Security Solution Detection Engine Area Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. triage_needed

Comments

@kibanamachine
Copy link
Contributor

A test failed on a tracked branch

AssertionError: Timed out retrying after 150000ms: Expected <div.euiPanel.euiPanel--primary.euiPanel--paddingMedium.euiCallOut.euiCallOut--primary.css-1lbbq49-euiPanel-none-m-primary-euiCallOut> not to exist in the DOM, but it was continuously found.
    at dismissCallOut (webpack:///./tasks/common/callouts.ts:20:17)
    at Context.eval (webpack:///./e2e/detection_alerts/missing_privileges_callout.cy.ts:92:25)

First failure: CI Build - main

@kibanamachine kibanamachine added the failed-test A test failure on a tracked branch, potentially flaky-test label Aug 14, 2023
@botelastic botelastic bot added the needs-team Issues missing a team label label Aug 14, 2023
@kibanamachine kibanamachine added the Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. label Aug 14, 2023
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

@botelastic botelastic bot removed the needs-team Issues missing a team label label Aug 14, 2023
@MadameSheema MadameSheema added triage_needed Team:Detections and Resp Security Detection Response Team Team:Detection Rule Management Security Detection Rule Management Team labels Aug 14, 2023
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-detections-response (Team:Detections and Resp)

@banderror banderror added Team:Detection Engine Security Solution Detection Engine Area and removed Team:Detection Rule Management Security Detection Rule Management Team labels Aug 21, 2023
@banderror banderror assigned yctercero and unassigned banderror Aug 21, 2023
@maximpn
Copy link
Contributor

maximpn commented Aug 22, 2023

Based on preliminary investigation the reason that sometimes Dismiss button's click doesn't close the callout. It may happen if the click happens before there is a click handler bound to it which is quite strange. Or there is a problem in the business logic like privileges get hashed and used as a part of callout's id so the callout gets shown after changing privileges if privileges don't load properly the hash can be different.

@jpdjere
Copy link
Contributor

jpdjere commented Oct 19, 2023

Flakiness investigated as part of: #169218

Flaky test runner shows no flakiness for this test on 200 runs.

Closing issue but tagging @yctercero and @elastic/security-detection-engine team to keep an eye open on this issue in case it's reopened.

Probably worth investigating what Maxim reported above if flakiness reappears.

@jpdjere jpdjere closed this as completed Oct 19, 2023
maximpn pushed a commit that referenced this issue Oct 24, 2023
…ts and enable Serverless run (#169218)

## Summary

Flaky test runner for Cypress tests in:
- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/detection_alerts` 
- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/value_lists` 
- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation` 

## Changes

- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/custom_query_rule_data_view.cy.ts` **tagged to run on Serverless**
- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/event_correlation_rule.cy.ts` **tagged to run on Serverless**
- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/machine_learning_rule.cy.ts` **tagged to run on Serverless**
- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/new_terms_rule.cy.ts` **tagged to run on Serverless**
- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/override.cy.ts`  **tagged to run on Serverless**
- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/rule_actions/deletion/rule_delete.cy.ts` **added disableAutorefresh() to avoid flakiness**

## Related failing-test issues

### `detection_response/detection_alerts`
1. #169091
**Marked as `legit-flake` and assigned to @elastic/security-detection-engine** 
3. #163885

### `detection_response/value_lists`
4. #165699

### `detection_response/rule_creation`
5. #163691

## Flaky test runner

- https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3583
- https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3615 [SERVERLESS ONLY]
- https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3656 [Serverless without indicator match and saved queries]
jpdjere added a commit to jpdjere/kibana that referenced this issue Oct 24, 2023
…ts and enable Serverless run (elastic#169218)

## Summary

Flaky test runner for Cypress tests in:
- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/detection_alerts`
- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/value_lists`
- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation`

## Changes

- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/custom_query_rule_data_view.cy.ts` **tagged to run on Serverless**
- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/event_correlation_rule.cy.ts` **tagged to run on Serverless**
- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/machine_learning_rule.cy.ts` **tagged to run on Serverless**
- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/new_terms_rule.cy.ts` **tagged to run on Serverless**
- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/override.cy.ts`  **tagged to run on Serverless**
- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/rule_actions/deletion/rule_delete.cy.ts` **added disableAutorefresh() to avoid flakiness**

## Related failing-test issues

### `detection_response/detection_alerts`
1. elastic#169091
**Marked as `legit-flake` and assigned to @elastic/security-detection-engine**
3. elastic#163885

### `detection_response/value_lists`
4. elastic#165699

### `detection_response/rule_creation`
5. elastic#163691

## Flaky test runner

- https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3583
- https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3615 [SERVERLESS ONLY]
- https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3656 [Serverless without indicator match and saved queries]

(cherry picked from commit 939c87d)
jpdjere referenced this issue Oct 26, 2023
…ess tests and enable Serverless run (#169218) (#169681)

# Backport

This will backport the following commits from `main` to `8.11`:
- [[Security Solution] Unskip tests for `detection_response` Cypress
tests and enable Serverless run
(#169218)](#169218)

<!--- Backport version: 8.9.8 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Juan Pablo
Djeredjian","email":"[email protected]"},"sourceCommit":{"committedDate":"2023-10-24T07:18:42Z","message":"[Security
Solution] Unskip tests for `detection_response` Cypress tests and enable
Serverless run (#169218)\n\n## Summary\r\n\r\nFlaky test runner for
Cypress tests in:\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/detection_alerts`
\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/value_lists`
\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation`
\r\n\r\n## Changes\r\n\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/custom_query_rule_data_view.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/event_correlation_rule.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/machine_learning_rule.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/new_terms_rule.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/override.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/rule_actions/deletion/rule_delete.cy.ts`
**added disableAutorefresh() to avoid flakiness**\r\n\r\n## Related
failing-test issues\r\n\r\n###
`detection_response/detection_alerts`\r\n1.
https://github.com/elastic/kibana/issues/169091\r\n**Marked as
`legit-flake` and assigned to @elastic/security-detection-engine**
\r\n3. https://github.com/elastic/kibana/issues/163885\r\n\r\n###
`detection_response/value_lists`\r\n4.
https://github.com/elastic/kibana/issues/165699\r\n\r\n###
`detection_response/rule_creation`\r\n5.
https://github.com/elastic/kibana/issues/163691\r\n\r\n## Flaky test
runner\r\n\r\n-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3583\r\n-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3615
[SERVERLESS ONLY]\r\n-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3656
[Serverless without indicator match and saved
queries]","sha":"939c87d95342c61f03c853cb3b34c9f412c303b0","branchLabelMapping":{"^v8.12.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","backport:skip","Team:Detections
and Resp","Team:Detection Rule Management","Team:Detection
Engine","v8.12.0"],"number":169218,"url":"https://github.com/elastic/kibana/pull/169218","mergeCommit":{"message":"[Security
Solution] Unskip tests for `detection_response` Cypress tests and enable
Serverless run (#169218)\n\n## Summary\r\n\r\nFlaky test runner for
Cypress tests in:\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/detection_alerts`
\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/value_lists`
\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation`
\r\n\r\n## Changes\r\n\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/custom_query_rule_data_view.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/event_correlation_rule.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/machine_learning_rule.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/new_terms_rule.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/override.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/rule_actions/deletion/rule_delete.cy.ts`
**added disableAutorefresh() to avoid flakiness**\r\n\r\n## Related
failing-test issues\r\n\r\n###
`detection_response/detection_alerts`\r\n1.
https://github.com/elastic/kibana/issues/169091\r\n**Marked as
`legit-flake` and assigned to @elastic/security-detection-engine**
\r\n3. https://github.com/elastic/kibana/issues/163885\r\n\r\n###
`detection_response/value_lists`\r\n4.
https://github.com/elastic/kibana/issues/165699\r\n\r\n###
`detection_response/rule_creation`\r\n5.
https://github.com/elastic/kibana/issues/163691\r\n\r\n## Flaky test
runner\r\n\r\n-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3583\r\n-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3615
[SERVERLESS ONLY]\r\n-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3656
[Serverless without indicator match and saved
queries]","sha":"939c87d95342c61f03c853cb3b34c9f412c303b0"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v8.12.0","labelRegex":"^v8.12.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/169218","number":169218,"mergeCommit":{"message":"[Security
Solution] Unskip tests for `detection_response` Cypress tests and enable
Serverless run (#169218)\n\n## Summary\r\n\r\nFlaky test runner for
Cypress tests in:\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/detection_alerts`
\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/value_lists`
\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation`
\r\n\r\n## Changes\r\n\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/custom_query_rule_data_view.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/event_correlation_rule.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/machine_learning_rule.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/new_terms_rule.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/override.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/rule_actions/deletion/rule_delete.cy.ts`
**added disableAutorefresh() to avoid flakiness**\r\n\r\n## Related
failing-test issues\r\n\r\n###
`detection_response/detection_alerts`\r\n1.
https://github.com/elastic/kibana/issues/169091\r\n**Marked as
`legit-flake` and assigned to @elastic/security-detection-engine**
\r\n3. https://github.com/elastic/kibana/issues/163885\r\n\r\n###
`detection_response/value_lists`\r\n4.
https://github.com/elastic/kibana/issues/165699\r\n\r\n###
`detection_response/rule_creation`\r\n5.
https://github.com/elastic/kibana/issues/163691\r\n\r\n## Flaky test
runner\r\n\r\n-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3583\r\n-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3615
[SERVERLESS ONLY]\r\n-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3656
[Serverless without indicator match and saved
queries]","sha":"939c87d95342c61f03c853cb3b34c9f412c303b0"}}]}]
BACKPORT-->

Co-authored-by: Kibana Machine <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
failed-test A test failure on a tracked branch, potentially flaky-test Team:Detection Engine Security Solution Detection Engine Area Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. triage_needed
Projects
None yet
Development

No branches or pull requests

7 participants