Skip to content

Commit

Permalink
[Security Solution] Unskip tests for detection_response Cypress tes…
Browse files Browse the repository at this point in the history
…ts and enable Serverless run (elastic#169218)

## Summary

Flaky test runner for Cypress tests in:
- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/detection_alerts` 
- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/value_lists` 
- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation` 

## Changes

- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/custom_query_rule_data_view.cy.ts` **tagged to run on Serverless**
- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/event_correlation_rule.cy.ts` **tagged to run on Serverless**
- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/machine_learning_rule.cy.ts` **tagged to run on Serverless**
- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/new_terms_rule.cy.ts` **tagged to run on Serverless**
- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/override.cy.ts`  **tagged to run on Serverless**
- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/rule_actions/deletion/rule_delete.cy.ts` **added disableAutorefresh() to avoid flakiness**

## Related failing-test issues

### `detection_response/detection_alerts`
1. elastic#169091
**Marked as `legit-flake` and assigned to @elastic/security-detection-engine** 
3. elastic#163885

### `detection_response/value_lists`
4. elastic#165699

### `detection_response/rule_creation`
5. elastic#163691

## Flaky test runner

- https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3583
- https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3615 [SERVERLESS ONLY]
- https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3656 [Serverless without indicator match and saved queries]
  • Loading branch information
jpdjere authored Oct 24, 2023
1 parent 52b9445 commit 939c87d
Show file tree
Hide file tree
Showing 6 changed files with 8 additions and 14 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -69,8 +69,7 @@ import { getDetails, waitForTheRuleToBeExecuted } from '../../../tasks/rule_deta

import { CREATE_RULE_URL } from '../../../urls/navigation';

// TODO: https://github.com/elastic/kibana/issues/161539
describe('Custom query rules', { tags: ['@ess', '@serverless', '@brokenInServerless'] }, () => {
describe('Custom query rules', { tags: ['@ess', '@serverless'] }, () => {
describe('Custom detection rules creation with data views', () => {
const rule = getDataViewRule();
const expectedUrls = rule.references?.join('');
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -57,8 +57,7 @@ import { visit } from '../../../tasks/navigation';
import { openRuleManagementPageViaBreadcrumbs } from '../../../tasks/rules_management';
import { CREATE_RULE_URL } from '../../../urls/navigation';

// TODO: https://github.com/elastic/kibana/issues/161539
describe('EQL rules', { tags: ['@ess', '@serverless', '@brokenInServerless'] }, () => {
describe('EQL rules', { tags: ['@ess', '@serverless'] }, () => {
before(() => {
cleanKibana();
});
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -54,19 +54,15 @@ import { visit } from '../../../tasks/navigation';
import { openRuleManagementPageViaBreadcrumbs } from '../../../tasks/rules_management';
import { CREATE_RULE_URL } from '../../../urls/navigation';

// TODO: https://github.com/elastic/kibana/issues/161539
describe('Machine Learning rules', { tags: ['@ess', '@serverless', '@brokenInServerless'] }, () => {
describe('Machine Learning rules', { tags: ['@ess', '@serverless'] }, () => {
const expectedUrls = (getMachineLearningRule().references ?? []).join('');
const expectedFalsePositives = (getMachineLearningRule().false_positives ?? []).join('');
const expectedTags = (getMachineLearningRule().tags ?? []).join('');
const expectedMitre = formatMitreAttackDescription(getMachineLearningRule().threat ?? []);
const expectedNumberOfRules = 1;

before(() => {
cleanKibana();
});

beforeEach(() => {
cleanKibana();
login();
visit(CREATE_RULE_URL);
});
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -59,8 +59,7 @@ import { visit } from '../../../tasks/navigation';
import { CREATE_RULE_URL } from '../../../urls/navigation';
import { openRuleManagementPageViaBreadcrumbs } from '../../../tasks/rules_management';

// TODO: https://github.com/elastic/kibana/issues/161539
describe('New Terms rules', { tags: ['@ess', '@serverless', '@brokenInServerless'] }, () => {
describe('New Terms rules', { tags: ['@ess', '@serverless'] }, () => {
before(() => {
cleanKibana();
login();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -62,8 +62,7 @@ import { getDetails, waitForTheRuleToBeExecuted } from '../../../tasks/rule_deta
import { CREATE_RULE_URL } from '../../../urls/navigation';
import { openRuleManagementPageViaBreadcrumbs } from '../../../tasks/rules_management';

// TODO: https://github.com/elastic/kibana/issues/161539
describe('Rules override', { tags: ['@ess', '@serverless', '@brokenInServerless'] }, () => {
describe('Rules override', { tags: ['@ess', '@serverless'] }, () => {
const rule = getNewOverrideRule();
const expectedUrls = rule.references?.join('');
const expectedFalsePositives = rule.false_positives?.join('');
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import { RULE_SWITCH } from '../../../../../screens/alerts_detection_rules';

import {
deleteFirstRule,
disableAutoRefresh,
getRulesManagementTableRows,
selectRulesByName,
} from '../../../../../tasks/alerts_detection_rules';
Expand All @@ -33,6 +34,7 @@ describe('Rule deletion', { tags: ['@ess', '@serverless'] }, () => {
createRule(testRules[2]);
login();
visitRulesManagementTable();
disableAutoRefresh();
});

it('User can delete an individual rule', () => {
Expand Down

0 comments on commit 939c87d

Please sign in to comment.