Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution][Flaky] /x-pack/plugins/security_solution/cypress/e2e/detection_rules/related_integrations.cy.ts #153684

Closed
Tracked by #153667 ...
MadameSheema opened this issue Mar 24, 2023 · 2 comments · Fixed by #164933
Assignees
Labels
8.10 candidate failed-test A test failure on a tracked branch, potentially flaky-test Team:Detection Rule Management Security Detection Rule Management Team Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. test test-failure-flaky

Comments

@MadameSheema
Copy link
Member

Job

1) Related integrations
--
  | proc [cypress]        installed integrations: Amazon CloudFront, AWS CloudTrail, System, enabled integrations: Amazon CloudFront, Aws Cloudfront, System
  | proc [cypress]          "before all" hook for "should display a badge with the installed integrations on the rule management page":
  | proc [cypress]      AssertionError: Timed out retrying after 150000ms: Expected to find element: `[data-test-subj="skipAgentInstallation"]`, but never found it.

Related integrations -- installed integrations Amazon CloudFront, AWS CloudTrail, System, enabled integrations Amazon CloudFront, Aws Cloudfront, System -- should display a badge with the installed integrations on the rule management page -

@MadameSheema MadameSheema added test-failure-flaky Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. labels Mar 24, 2023
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-detections-response (Team:Detections and Resp)

@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

@MadameSheema MadameSheema added failed-test A test failure on a tracked branch, potentially flaky-test triage_needed labels May 23, 2023
@banderror banderror changed the title [Security Solution][Flaky test] /x-pack/plugins/security_solution/cypress/e2e/detection_rules/related_integrations.cy.ts [Security Solution][Flaky] /x-pack/plugins/security_solution/cypress/e2e/detection_rules/related_integrations.cy.ts May 25, 2023
@banderror banderror added test Team:Detection Rule Management Security Detection Rule Management Team 8.10 candidate and removed triage_needed labels May 25, 2023
maximpn added a commit that referenced this issue Aug 31, 2023
**Fixes: #154663
**Fixes: #153684

## Summary

This PR unskips Cypress rule related integration tests ([related_integrations.cy.ts](https://github.com/elastic/kibana/blob/main/x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/related_integrations/related_integrations.cy.ts)).

## Details

Testing approach has changed. Instead of importing a rule and installing agent and package policies via Fleet UI the rule is created by mocking a prebuilt rule asset and Fleet API is used to install required integrations. Along the way it required to add and update some testing selectors as UI had changed while the tests were skipped.

## Flaky test runner

[related_integrations.cy.ts (150 runs)](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2995) 🟢
maximpn added a commit to maximpn/kibana that referenced this issue Aug 31, 2023
**Fixes: elastic#154663
**Fixes: elastic#153684

## Summary

This PR unskips Cypress rule related integration tests ([related_integrations.cy.ts](https://github.com/elastic/kibana/blob/main/x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/related_integrations/related_integrations.cy.ts)).

## Details

Testing approach has changed. Instead of importing a rule and installing agent and package policies via Fleet UI the rule is created by mocking a prebuilt rule asset and Fleet API is used to install required integrations. Along the way it required to add and update some testing selectors as UI had changed while the tests were skipped.

## Flaky test runner

[related_integrations.cy.ts (150 runs)](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2995) 🟢

(cherry picked from commit 2c3464c)

# Conflicts:
#	x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/related_integrations/related_integrations.cy.ts
#	x-pack/test/security_solution_cypress/cypress/tasks/api_calls/prebuilt_rules.ts
maximpn referenced this issue Aug 31, 2023
#165310)

# Backport

This will backport the following commits from `main` to `8.10`:
- [[Security Solution] Unskip related integrations tests
(#164933)](#164933)

<!--- Backport version: 8.9.8 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Maxim
Palenov","email":"[email protected]"},"sourceCommit":{"committedDate":"2023-08-31T10:28:16Z","message":"[Security
Solution] Unskip related integrations tests (#164933)\n\n**Fixes:
https://github.com/elastic/kibana/issues/154663**\r\n**Fixes:
https://github.com/elastic/kibana/issues/153684**\r\n\r\n##
Summary\r\n\r\nThis PR unskips Cypress rule related integration tests
([related_integrations.cy.ts](https://github.com/elastic/kibana/blob/main/x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/related_integrations/related_integrations.cy.ts)).\r\n\r\n##
Details\r\n\r\nTesting approach has changed. Instead of importing a rule
and installing agent and package policies via Fleet UI the rule is
created by mocking a prebuilt rule asset and Fleet API is used to
install required integrations. Along the way it required to add and
update some testing selectors as UI had changed while the tests were
skipped.\r\n\r\n## Flaky test runner\r\n\r\n[related_integrations.cy.ts
(150
runs)](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2995)
🟢","sha":"2c3464ca8457e09f51852c6c0d60ab7f6e21afc0","branchLabelMapping":{"^v8.11.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["test","release_note:skip","Team:Detections
and Resp","Team: SecuritySolution","Feature:Rule
Management","Team:Detection Rule
Management","v8.10.0","v8.11.0","flake-docs"],"number":164933,"url":"https://github.com/elastic/kibana/pull/164933","mergeCommit":{"message":"[Security
Solution] Unskip related integrations tests (#164933)\n\n**Fixes:
https://github.com/elastic/kibana/issues/154663**\r\n**Fixes:
https://github.com/elastic/kibana/issues/153684**\r\n\r\n##
Summary\r\n\r\nThis PR unskips Cypress rule related integration tests
([related_integrations.cy.ts](https://github.com/elastic/kibana/blob/main/x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/related_integrations/related_integrations.cy.ts)).\r\n\r\n##
Details\r\n\r\nTesting approach has changed. Instead of importing a rule
and installing agent and package policies via Fleet UI the rule is
created by mocking a prebuilt rule asset and Fleet API is used to
install required integrations. Along the way it required to add and
update some testing selectors as UI had changed while the tests were
skipped.\r\n\r\n## Flaky test runner\r\n\r\n[related_integrations.cy.ts
(150
runs)](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2995)
🟢","sha":"2c3464ca8457e09f51852c6c0d60ab7f6e21afc0"}},"sourceBranch":"main","suggestedTargetBranches":["8.10"],"targetPullRequestStates":[{"branch":"8.10","label":"v8.10.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.11.0","labelRegex":"^v8.11.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/164933","number":164933,"mergeCommit":{"message":"[Security
Solution] Unskip related integrations tests (#164933)\n\n**Fixes:
https://github.com/elastic/kibana/issues/154663**\r\n**Fixes:
https://github.com/elastic/kibana/issues/153684**\r\n\r\n##
Summary\r\n\r\nThis PR unskips Cypress rule related integration tests
([related_integrations.cy.ts](https://github.com/elastic/kibana/blob/main/x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/related_integrations/related_integrations.cy.ts)).\r\n\r\n##
Details\r\n\r\nTesting approach has changed. Instead of importing a rule
and installing agent and package policies via Fleet UI the rule is
created by mocking a prebuilt rule asset and Fleet API is used to
install required integrations. Along the way it required to add and
update some testing selectors as UI had changed while the tests were
skipped.\r\n\r\n## Flaky test runner\r\n\r\n[related_integrations.cy.ts
(150
runs)](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2995)
🟢","sha":"2c3464ca8457e09f51852c6c0d60ab7f6e21afc0"}}]}] BACKPORT-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
8.10 candidate failed-test A test failure on a tracked branch, potentially flaky-test Team:Detection Rule Management Security Detection Rule Management Team Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. test test-failure-flaky
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants