Skip to content

Commit

Permalink
[8.11] [Security Solution] Unskip tests for detection_response Cypr…
Browse files Browse the repository at this point in the history
…ess tests and enable Serverless run (#169218) (#169681)

# Backport

This will backport the following commits from `main` to `8.11`:
- [[Security Solution] Unskip tests for `detection_response` Cypress
tests and enable Serverless run
(#169218)](#169218)

<!--- Backport version: 8.9.8 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Juan Pablo
Djeredjian","email":"[email protected]"},"sourceCommit":{"committedDate":"2023-10-24T07:18:42Z","message":"[Security
Solution] Unskip tests for `detection_response` Cypress tests and enable
Serverless run (#169218)\n\n## Summary\r\n\r\nFlaky test runner for
Cypress tests in:\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/detection_alerts`
\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/value_lists`
\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation`
\r\n\r\n## Changes\r\n\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/custom_query_rule_data_view.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/event_correlation_rule.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/machine_learning_rule.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/new_terms_rule.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/override.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/rule_actions/deletion/rule_delete.cy.ts`
**added disableAutorefresh() to avoid flakiness**\r\n\r\n## Related
failing-test issues\r\n\r\n###
`detection_response/detection_alerts`\r\n1.
https://github.com/elastic/kibana/issues/169091\r\n**Marked as
`legit-flake` and assigned to @elastic/security-detection-engine**
\r\n3. https://github.com/elastic/kibana/issues/163885\r\n\r\n###
`detection_response/value_lists`\r\n4.
https://github.com/elastic/kibana/issues/165699\r\n\r\n###
`detection_response/rule_creation`\r\n5.
https://github.com/elastic/kibana/issues/163691\r\n\r\n## Flaky test
runner\r\n\r\n-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3583\r\n-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3615
[SERVERLESS ONLY]\r\n-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3656
[Serverless without indicator match and saved
queries]","sha":"939c87d95342c61f03c853cb3b34c9f412c303b0","branchLabelMapping":{"^v8.12.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","backport:skip","Team:Detections
and Resp","Team:Detection Rule Management","Team:Detection
Engine","v8.12.0"],"number":169218,"url":"https://github.com/elastic/kibana/pull/169218","mergeCommit":{"message":"[Security
Solution] Unskip tests for `detection_response` Cypress tests and enable
Serverless run (#169218)\n\n## Summary\r\n\r\nFlaky test runner for
Cypress tests in:\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/detection_alerts`
\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/value_lists`
\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation`
\r\n\r\n## Changes\r\n\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/custom_query_rule_data_view.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/event_correlation_rule.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/machine_learning_rule.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/new_terms_rule.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/override.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/rule_actions/deletion/rule_delete.cy.ts`
**added disableAutorefresh() to avoid flakiness**\r\n\r\n## Related
failing-test issues\r\n\r\n###
`detection_response/detection_alerts`\r\n1.
https://github.com/elastic/kibana/issues/169091\r\n**Marked as
`legit-flake` and assigned to @elastic/security-detection-engine**
\r\n3. https://github.com/elastic/kibana/issues/163885\r\n\r\n###
`detection_response/value_lists`\r\n4.
https://github.com/elastic/kibana/issues/165699\r\n\r\n###
`detection_response/rule_creation`\r\n5.
https://github.com/elastic/kibana/issues/163691\r\n\r\n## Flaky test
runner\r\n\r\n-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3583\r\n-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3615
[SERVERLESS ONLY]\r\n-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3656
[Serverless without indicator match and saved
queries]","sha":"939c87d95342c61f03c853cb3b34c9f412c303b0"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v8.12.0","labelRegex":"^v8.12.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/169218","number":169218,"mergeCommit":{"message":"[Security
Solution] Unskip tests for `detection_response` Cypress tests and enable
Serverless run (#169218)\n\n## Summary\r\n\r\nFlaky test runner for
Cypress tests in:\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/detection_alerts`
\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/value_lists`
\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation`
\r\n\r\n## Changes\r\n\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/custom_query_rule_data_view.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/event_correlation_rule.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/machine_learning_rule.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/new_terms_rule.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/override.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/rule_actions/deletion/rule_delete.cy.ts`
**added disableAutorefresh() to avoid flakiness**\r\n\r\n## Related
failing-test issues\r\n\r\n###
`detection_response/detection_alerts`\r\n1.
https://github.com/elastic/kibana/issues/169091\r\n**Marked as
`legit-flake` and assigned to @elastic/security-detection-engine**
\r\n3. https://github.com/elastic/kibana/issues/163885\r\n\r\n###
`detection_response/value_lists`\r\n4.
https://github.com/elastic/kibana/issues/165699\r\n\r\n###
`detection_response/rule_creation`\r\n5.
https://github.com/elastic/kibana/issues/163691\r\n\r\n## Flaky test
runner\r\n\r\n-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3583\r\n-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3615
[SERVERLESS ONLY]\r\n-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3656
[Serverless without indicator match and saved
queries]","sha":"939c87d95342c61f03c853cb3b34c9f412c303b0"}}]}]
BACKPORT-->

Co-authored-by: Kibana Machine <[email protected]>
  • Loading branch information
jpdjere and kibanamachine authored Oct 26, 2023
1 parent 1d35c7b commit 9f1ce69
Show file tree
Hide file tree
Showing 6 changed files with 8 additions and 14 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -69,8 +69,7 @@ import { getDetails, waitForTheRuleToBeExecuted } from '../../../tasks/rule_deta

import { CREATE_RULE_URL } from '../../../urls/navigation';

// TODO: https://github.com/elastic/kibana/issues/161539
describe('Custom query rules', { tags: ['@ess', '@serverless', '@brokenInServerless'] }, () => {
describe('Custom query rules', { tags: ['@ess', '@serverless'] }, () => {
describe('Custom detection rules creation with data views', () => {
const rule = getDataViewRule();
const expectedUrls = rule.references?.join('');
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -57,8 +57,7 @@ import { visit } from '../../../tasks/navigation';
import { openRuleManagementPageViaBreadcrumbs } from '../../../tasks/rules_management';
import { CREATE_RULE_URL } from '../../../urls/navigation';

// TODO: https://github.com/elastic/kibana/issues/161539
describe('EQL rules', { tags: ['@ess', '@serverless', '@brokenInServerless'] }, () => {
describe('EQL rules', { tags: ['@ess', '@serverless'] }, () => {
before(() => {
cleanKibana();
});
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -54,19 +54,15 @@ import { visit } from '../../../tasks/navigation';
import { openRuleManagementPageViaBreadcrumbs } from '../../../tasks/rules_management';
import { CREATE_RULE_URL } from '../../../urls/navigation';

// TODO: https://github.com/elastic/kibana/issues/161539
describe('Machine Learning rules', { tags: ['@ess', '@serverless', '@brokenInServerless'] }, () => {
describe('Machine Learning rules', { tags: ['@ess', '@serverless'] }, () => {
const expectedUrls = (getMachineLearningRule().references ?? []).join('');
const expectedFalsePositives = (getMachineLearningRule().false_positives ?? []).join('');
const expectedTags = (getMachineLearningRule().tags ?? []).join('');
const expectedMitre = formatMitreAttackDescription(getMachineLearningRule().threat ?? []);
const expectedNumberOfRules = 1;

before(() => {
cleanKibana();
});

beforeEach(() => {
cleanKibana();
login();
visit(CREATE_RULE_URL);
});
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -59,8 +59,7 @@ import { visit } from '../../../tasks/navigation';
import { CREATE_RULE_URL } from '../../../urls/navigation';
import { openRuleManagementPageViaBreadcrumbs } from '../../../tasks/rules_management';

// TODO: https://github.com/elastic/kibana/issues/161539
describe('New Terms rules', { tags: ['@ess', '@serverless', '@brokenInServerless'] }, () => {
describe('New Terms rules', { tags: ['@ess', '@serverless'] }, () => {
before(() => {
cleanKibana();
login();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -62,8 +62,7 @@ import { getDetails, waitForTheRuleToBeExecuted } from '../../../tasks/rule_deta
import { CREATE_RULE_URL } from '../../../urls/navigation';
import { openRuleManagementPageViaBreadcrumbs } from '../../../tasks/rules_management';

// TODO: https://github.com/elastic/kibana/issues/161539
describe('Rules override', { tags: ['@ess', '@serverless', '@brokenInServerless'] }, () => {
describe('Rules override', { tags: ['@ess', '@serverless'] }, () => {
const rule = getNewOverrideRule();
const expectedUrls = rule.references?.join('');
const expectedFalsePositives = rule.false_positives?.join('');
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import { RULE_SWITCH } from '../../../../../screens/alerts_detection_rules';

import {
deleteFirstRule,
disableAutoRefresh,
getRulesManagementTableRows,
selectRulesByName,
} from '../../../../../tasks/alerts_detection_rules';
Expand All @@ -33,6 +34,7 @@ describe('Rule deletion', { tags: ['@ess', '@serverless'] }, () => {
createRule(testRules[2]);
login();
visitRulesManagementTable();
disableAutoRefresh();
});

it('User can delete an individual rule', () => {
Expand Down

0 comments on commit 9f1ce69

Please sign in to comment.