Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failing test: Security Solution Cypress.x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/related_integrations/related_integrations·cy·ts - Related integrations related Integrations Advanced Setting is disabled rule details should display the integrations in the definition section should display the integrations in the definition section #173307

Closed
Tracked by #173731
kibanamachine opened this issue Dec 13, 2023 · 14 comments
Closed
Tracked by #173731

Failing test: Security Solution Cypress.x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/related_integrations/related_integrations·cy·ts - Related integrations related Integrations Advanced Setting is disabled rule details should display the integrations in the definition section should display the integrations in the definition section #173307

kibanamachine opened this issue Dec 13, 2023 · 14 comments
Assignees
@jpdjere
Labels
8.12 candidate failed-test A test failure on a tracked branch, potentially flaky-test Team:Detection Rule Management Security Detection Rule Management Team Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc.

Comments

@kibanamachine
Copy link
Contributor

kibanamachine commented Dec 13, 2023 •
edited
Loading

A test failed on a tracked branch

AssertionError: Timed out retrying after 150000ms: Not enough elements found. Found '2', expected '4'.
    at Context.eval (webpack:///./e2e/detection_response/rule_management/related_integrations/related_integrations.cy.ts:261:33)

First failure: CI Build - main
@kibanamachine kibanamachine added the failed-test A test failure on a tracked branch, potentially flaky-test label Dec 13, 2023
@botelastic botelastic bot added the needs-team Issues missing a team label label Dec 13, 2023
@MadameSheema MadameSheema added triage_needed Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:Detection Rule Management Security Detection Rule Management Team labels Dec 13, 2023
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-detections-response (Team:Detections and Resp)

@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

@botelastic botelastic bot removed the needs-team Issues missing a team label label Dec 13, 2023
@MindyRS
Copy link

MindyRS commented Dec 13, 2023

@banderror Can you check the CODEOWNERS file as well to see why this ticket didn’t get tagged automatically?

@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

mistic added a commit that referenced this issue Dec 13, 2023
@mistic
skip flaky suite (#173307)
047f6cf
@mistic
Copy link
Member

mistic commented Dec 13, 2023

Skipped.

main: 047f6cf

@banderror banderror assigned jpdjere and unassigned banderror Dec 14, 2023
@banderror
Copy link
Contributor

@banderror Can you check the CODEOWNERS file as well to see why this ticket didn’t get tagged automatically?

Yep, will do @MindyRS 👍

@banderror
Copy link
Contributor

banderror commented Dec 14, 2023 •
edited
Loading

@MindyRS @jpdjere Regarding CODEOWNERS:

This is the flaky test file's path: x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/related_integrations/related_integrations.cy.ts.

We do have the whole x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management folder assigned to our team in CODEOWNERS in main:

kibana/.github/CODEOWNERS

Line 1343 in 37af741

/x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management @elastic/security-detection-rule-management

To double-check that GitHub correctly identifies @elastic/security-detection-rule-management team as the code owner of this file, I changed this file and opened a test PR, where you can see that our team was automatically added as a mandatory reviewer based on CODEOWNERS: #173381

Screenshot 2023-12-14 at 14 26 33

@oatkiller So this looks like a bug in the automation to me. It fails to identify owners of test files in some cases. I'd assume that some lines in CODEOWNERS can "break" it. For example, we have these lines which are 100% valid and correct in terms of the CODEOWNERS file syntax:

kibana/.github/CODEOWNERS

Lines 1478 to 1479 in 99ab7a0

/x-pack/test/security_solution_cypress/* @elastic/security-engineering-productivity
/x-pack/test/security_solution_cypress/cypress/* @elastic/security-engineering-productivity

but these two lines are below this line:

kibana/.github/CODEOWNERS

Line 1343 in 37af741

/x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management @elastic/security-detection-rule-management

Could this be the reason?

@banderror banderror mentioned this issue Dec 14, 2023
Closed
@MindyRS
Copy link

MindyRS commented Dec 14, 2023

@banderror Yes, anything that matches lower in the file will take precedent.

@oatkiller
Copy link
Contributor

oatkiller commented Dec 14, 2023 •
edited
Loading

@banderror I did some debugging. I think the correct codeowner line is being referenced, but the script fails to parse the right team.

The script thinks its owned by the rule starting with : "x-pack/test/security_solution_cypress/cypress/*"

I'm fairly certain the error is in this file.

https://github.com/elastic/kibana-operations/blob/0e2eb70e3673d8845881b54064e53ba7d5b36296/triage/src/codePathsWithOwners.js#L15-L43

@MadameSheema to debug this you need to create a personal github access token and then follow instructions in /triage directory in the same repo.

@banderror banderror mentioned this issue Dec 18, 2023
Merged
@banderror
Copy link
Contributor

@MindyRS @MadameSheema I opened a PR with a workaround #173527 for the Eng Prod team, please take a look.

banderror added a commit that referenced this issue Dec 18, 2023
@banderror
[Security Solution] Lift up @elastic/security-engineering-productivity …
7b9888d 
…lines in CODEOWNERS (#173527)

## Summary

Context:
#173307 (comment)

Summary of the issue: the automation that assigns team labels to
`failed-test` tickets sometimes doesn't assign a team label or can
assign an incorrect label. This is because it scans the CODEOWNERS file
from bottom to top instead of properly parsing it as GitHub does.

This PR lifts some of the lines up, which should reduce the chance of
the above issue happening again. But that's a temporary workaround.
@banderror banderror mentioned this issue Dec 20, 2023
Closed
@banderror
Copy link
Contributor

The test has been fixed and unskipped in main and 8.12.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jpdjere jpdjere

Labels
8.12 candidate failed-test A test failure on a tracked branch, potentially flaky-test Team:Detection Rule Management Security Detection Rule Management Team Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
8 participants
@oatkiller @MindyRS @mistic @jpdjere @banderror @elasticmachine @MadameSheema @kibanamachine