[Security Solution] Labels adjustments (elastic#181371)

Following up from elastic#180773

## Summary
 
In this PR we are introducing the `@skipServerlessMKI` label, with it, a
test will be excluded from the execution on any MKI environment but it
will be executed as part of the CI check if the `@serverless` tag is
present.
 
 With the new changes the serverless labels will work as follows:
 
* `@serverless`: The test is executed as part of the PR check process
and in the periodic pipeline (MKI environment).
 
* `@serverlessQA`: The test is executed as part of the kibana release
process in the QA environment (MKI environment).
 
* `@skipInServerless`: The test is skipped from being executed in CI as
part of the PR check and is skipped from being executed in any MKI
environment.
 
`@skipInServerlessMKI`: The test is skipped from being executed in any
MKI environment but it will continue being executed as part of the PR
process if the `@serverless` tag is present.
 
 **IMPORTANT:**
 
The skip labels have been created for those tests that use `@serverless`
or `@serverlessQA` labels. The absence of them (`@serverless` or
`@serverlessQA`) will exclude automatically the execution of the test in
the targeted environments.
 
I.E: A test without `@serverlessQA` will never be executed as part of
the Kibana release process. A test without `@serverless` will never be
executed as part of the PR CI check neither the periodic pipeline.

x-pack/plugins/security_solution/scripts/run_cypress/parallel_serverless.ts

@@ -475,7 +475,8 @@ ${JSON.stringify(argv, null, 2)}
         !process.env.KIBANA_MKI_USE_LATEST_COMMIT ||
         process.env.KIBANA_MKI_USE_LATEST_COMMIT !== '1'
       ) {
-        cypressConfigFile.env.grepTags = '@serverlessQA --@skipInServerless';
+        cypressConfigFile.env.grepTags =
+          '@serverlessQA --@skipInServerless --@skipInServerlessMKI ';
       }
       const tier: string = argv.tier;
       const endpointAddon: boolean = argv.endpointAddon;

x-pack/test/security_solution_cypress/cypress/README.md

@@ -48,6 +48,7 @@ Note that we use tags in order to select which tests we want to execute:
 - `@serverlessQA` includes a test in the Serverless test suite for the Kibana release process of serverless. You need to explicitly add this tag to any test you want yo run in CI for the second quality gate. These tests should be stable, otherviswe they will be blocking the release pipeline. They should be alsy critical enough, so that when they fail, there's a high chance of an SDH or blocker issue to be reported. 
 - `@ess` includes a test in the normal, non-Serverless test suite. You need to explicitly add this tag to any test you want to run against a non-Serverless environment.
 - `@skipInEss` excludes a test from the non-Serverless test suite. The test will not be executed as part for the PR process. All the skipped tests should have a link to a ticket describing the reason why the test got skipped.
+- `@skipInServerlessMKI` excludes a test from the execution on any MKI environment (even if it's tagged as `@serverless` or `@serverlessQA`). Could indicate many things, e.g. "the test is flaky in Serverless MKI", "the test has been temporarily excluded, see the comment above why". All the skipped tests should have a link to a ticket describing the reason why the test got skipped.
 - `@skipInServerless` excludes a test from the Serverless test suite and Serverless QA environment for both, periodic pipeline and second quality gate (even if it's tagged as `@serverless`). Could indicate many things, e.g. "the test is flaky in Serverless", "the test is Flaky in any type of environment", "the test has been temporarily excluded, see the comment above why". All the skipped tests should have a link to a ticket describing the reason why the test got skipped.
 
 Please, before opening a PR with a new test, make sure that the test fails. If you never see your test fail you don’t know if your test is actually testing the right thing, or testing anything at all.

x-pack/test/security_solution_cypress/cypress/cypress_ci_serverless_qa.config.ts

@@ -19,7 +19,7 @@ export default defineCypressConfig({
   env: {
     grepFilterSpecs: true,
     grepOmitFiltered: true,
-    grepTags: '@serverless --@skipInServerless',
+    grepTags: '@serverless --@skipInServerless --@skipInServerlessMKI',
   },
   execTimeout: 300000,
   pageLoadTimeout: 300000,

x-pack/test/security_solution_cypress/cypress/e2e/detection_response/detection_engine/rule_creation/common_flows_suppression_serverless_essentials.cy.ts

@@ -21,7 +21,7 @@ import { CREATE_RULE_URL } from '../../../../urls/navigation';
 describe(
   'Detection rules, Alert Suppression for Essentials tier',
   {
-    tags: ['@serverless'],
+    tags: ['@serverless', '@skipServerlessMKI'],
     env: {
       ftrConfig: {
         productTypes: [

x-pack/test/security_solution_cypress/cypress/e2e/detection_response/detection_engine/rule_creation/event_correlation_rule_suppression.cy.ts

@@ -37,7 +37,7 @@ const SUPPRESS_BY_FIELDS = ['agent.type'];
 describe(
   'Detection Rule Creation - EQL Rules - With Alert Suppression',
   {
-    tags: ['@ess', '@serverless', '@skipInServerless'],
+    tags: ['@ess', '@serverless', '@skipServerlessMKI'],
     // alertSuppressionForNonSequenceEqlRuleEnabled feature flag is also enabled in a global config
     env: {
       ftrConfig: {

x-pack/test/security_solution_cypress/cypress/e2e/detection_response/detection_engine/rule_creation/event_correlation_rule_suppression_sequence.cy.ts

@@ -21,7 +21,7 @@ import {
 describe(
   'Detection Rule Creation - EQL Rules - With Alert Suppression',
   {
-    tags: ['@ess', '@serverless', '@skipInServerless'],
+    tags: ['@ess', '@serverless', '@skipServerlessMKI'],
     // alertSuppressionForNonSequenceEqlRuleEnabled feature flag is also enabled in a global config
     env: {
       ftrConfig: {

x-pack/test/security_solution_cypress/cypress/e2e/detection_response/detection_engine/rule_creation/event_correlation_rule_suppression_serverless_essentials.cy.ts

@@ -28,7 +28,7 @@ const SUPPRESS_BY_FIELDS = ['agent.type'];
 describe(
   'Detection Rule Creation - EQL Rules - With Alert Suppression  - Serverless Essentials License',
   {
-    tags: ['@serverless', '@skipInServerless'],
+    tags: ['@serverless', '@skipServerlessMKI'],
     // alertSuppressionForNonSequenceEqlRuleEnabled feature flag is also enabled in a global config
     env: {
       ftrConfig: {

x-pack/test/security_solution_cypress/cypress/e2e/detection_response/detection_engine/rule_creation/new_terms_rule.cy.ts

@@ -74,7 +74,7 @@ import { openRuleManagementPageViaBreadcrumbs } from '../../../../tasks/rules_ma
 describe(
   'New Terms rules',
   {
-    tags: ['@ess', '@serverless'],
+    tags: ['@ess', '@serverless', '@skipServerlessMKI'],
     env: {
       // alertSuppressionForNewTermsRuleEnabled feature flag is also enabled in a global config
       kbnServerArgs: [

x-pack/test/security_solution_cypress/cypress/e2e/detection_response/detection_engine/rule_edit/new_terms_rule.cy.ts

@@ -44,7 +44,7 @@ const rule = getNewTermsRule();
 describe(
   'Detection rules, New terms, Edit',
   {
-    tags: ['@ess', '@serverless'],
+    tags: ['@ess', '@serverless', '@skipServerlessMKI'],
     env: {
       // alertSuppressionForNewTermsRuleEnabled feature flag is also enabled in a global config
       kbnServerArgs: [

x-pack/test/security_solution_cypress/cypress/e2e/detection_response/detection_engine/value_lists/value_list_items.cy.ts

@@ -43,7 +43,7 @@ import { RULES_MANAGEMENT_URL } from '../../../../urls/rules_management';
 describe(
   'Value list items',
   {
-    tags: ['@ess', '@serverless', '@skipInServerless'],
+    tags: ['@ess', '@serverless', '@skipServerlessMKI'],
     env: {
       ftrConfig: {
         kbnServerArgs: [

x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_update_authorization.cy.ts

@@ -68,7 +68,7 @@ const loginPageAsWriteAuthorizedUser = (url: string) => {
 // https://github.com/elastic/kibana/issues/179965
 describe(
   'Detection rules, Prebuilt Rules Installation and Update - Authorization/RBAC',
-  { tags: ['@ess', '@serverless', '@skipInServerless'] },
+  { tags: ['@ess', '@serverless', '@skipInServerlessMKI'] },
   () => {
     beforeEach(() => {
       preventPrebuiltRulesPackageInstallation();

x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_update_error_handling.cy.ts

@@ -39,7 +39,7 @@ import { visitRulesManagementTable } from '../../../../tasks/rules_management';
 // https://github.com/elastic/kibana/issues/179970
 describe(
   'Detection rules, Prebuilt Rules Installation and Update - Error handling',
-  { tags: ['@ess', '@serverless', '@skipInServerless'] },
+  { tags: ['@ess', '@serverless', '@skipInServerlessMKI'] },
   () => {
     beforeEach(() => {
       preventPrebuiltRulesPackageInstallation();

x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_workflow.cy.ts

@@ -32,7 +32,7 @@ import { deleteAlertsAndRules } from '../../../../tasks/api_calls/common';
 
 describe(
   'Detection rules, Prebuilt Rules Installation and Update workflow',
-  { tags: ['@ess', '@serverless', '@skipInServerless'] },
+  { tags: ['@ess', '@serverless', '@skipInServerlessMKI'] },
   () => {
     describe('Installation of prebuilt rules', () => {
       const RULE_1 = createRuleAssetSavedObject({

x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/management.cy.ts

@@ -51,7 +51,7 @@ const rules = Array.from(Array(5)).map((_, i) => {
 });
 
 // https://github.com/elastic/kibana/issues/179973
-describe('Prebuilt rules', { tags: ['@ess', '@serverless', '@skipInServerless'] }, () => {
+describe('Prebuilt rules', { tags: ['@ess', '@serverless', '@skipInServerlessMKI'] }, () => {
   beforeEach(() => {
     login();
     deleteAlertsAndRules();

x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/notifications.cy.ts

@@ -57,7 +57,7 @@ describe(
       // https://github.com/elastic/kibana/issues/179967
       it(
         'should NOT display install or update notifications when latest rules are installed',
-        { tags: ['@skipInServerless'] },
+        { tags: ['@skipInServerlessMKI'] },
         () => {
           visitRulesManagementTable();
           createAndInstallMockedPrebuiltRules([RULE_1]);
@@ -72,7 +72,7 @@ describe(
     });
 
     // https://github.com/elastic/kibana/issues/179968
-    describe('Notifications', { tags: ['@skipInServerless'] }, () => {
+    describe('Notifications', { tags: ['@skipInServerlessMKI'] }, () => {
       beforeEach(() => {
         installPrebuiltRuleAssets([RULE_1]);
       });

x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/related_integrations/related_integrations.cy.ts

@@ -47,7 +47,7 @@ import {
 
 // https://github.com/elastic/kibana/issues/179943
 
-describe('Related integrations', { tags: ['@ess', '@serverless', '@skipInServerless'] }, () => {
+describe('Related integrations', { tags: ['@ess', '@serverless', '@skipInServerlessMKI'] }, () => {
   const DATA_STREAM_NAME = 'logs-related-integrations-test';
   const PREBUILT_RULE_NAME = 'Prebuilt rule with related integrations';
   const RULE_RELATED_INTEGRATIONS: IntegrationDefinition[] = [

x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/rule_actions/bulk_actions/bulk_edit_rules.cy.ts

@@ -183,7 +183,7 @@ describe('Detection rules, bulk edit', { tags: ['@ess', '@serverless'] }, () =>
     });
 
     // github.com/elastic/kibana/issues/179954
-    it('Only prebuilt rules selected', { tags: ['@skipInServerless'] }, () => {
+    it('Only prebuilt rules selected', { tags: ['@skipInServerlessMKI'] }, () => {
       createAndInstallMockedPrebuiltRules(PREBUILT_RULES);
 
       // select Elastic(prebuilt) rules, check if we can't proceed further, as Elastic rules are not editable
@@ -204,7 +204,7 @@ describe('Detection rules, bulk edit', { tags: ['@ess', '@serverless'] }, () =>
     // https://github.com/elastic/kibana/issues/179955
     it(
       'Prebuilt and custom rules selected: user proceeds with custom rules editing',
-      { tags: ['@skipInServerless'] },
+      { tags: ['@skipInServerlessMKI'] },
       () => {
         getRulesManagementTableRows().then((existedRulesRows) => {
           createAndInstallMockedPrebuiltRules(PREBUILT_RULES);
@@ -235,7 +235,7 @@ describe('Detection rules, bulk edit', { tags: ['@ess', '@serverless'] }, () =>
     // https://github.com/elastic/kibana/issues/179956
     it(
       'Prebuilt and custom rules selected: user cancels action',
-      { tags: ['@skipInServerless'] },
+      { tags: ['@skipInServerlessMKI'] },
       () => {
         createAndInstallMockedPrebuiltRules(PREBUILT_RULES);
 

x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/rule_actions/bulk_actions/bulk_edit_rules_actions.cy.ts

@@ -75,7 +75,7 @@ const expectedSlackMessage = 'Slack action test message';
 // https://github.com/elastic/kibana/issues/179958
 describe(
   'Detection rules, bulk edit of rule actions',
-  { tags: ['@ess', '@serverless', '@skipInServerless'] },
+  { tags: ['@ess', '@serverless', '@skipInServerlessMKI'] },
   () => {
     beforeEach(() => {
       login();

x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/rule_actions/import_export/export_rule.cy.ts

@@ -96,7 +96,7 @@ describe('Export rules', { tags: ['@ess', '@serverless'] }, () => {
   // https://github.com/elastic/kibana/issues/179959
   it(
     'shows a modal saying that no rules can be exported if all the selected rules are prebuilt',
-    { tags: ['@skipInServerless'] },
+    { tags: ['@skipInServerlessMKI'] },
     function () {
       createAndInstallMockedPrebuiltRules(prebuiltRules);
 
@@ -164,7 +164,7 @@ describe('Export rules', { tags: ['@ess', '@serverless'] }, () => {
     });
 
     // https://github.com/elastic/kibana/issues/180029
-    it('exports custom rules with exceptions', { tags: ['@skipInServerless'] }, function () {
+    it('exports custom rules with exceptions', { tags: ['@skipInServerlessMKI'] }, function () {
       // one rule with exception, one without it
       const expectedNumberCustomRulesToBeExported = 2;
 

x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/rules_table/rules_table_selection.cy.ts

@@ -35,65 +35,69 @@ const RULE_2 = createRuleAssetSavedObject({
 });
 
 // https://github.com/elastic/kibana/issues/179961
-describe('Rules table: selection', { tags: ['@ess', '@serverless', '@skipInServerless'] }, () => {
-  beforeEach(() => {
-    login();
-    /* Create and install two mock rules */
-    createAndInstallMockedPrebuiltRules([RULE_1, RULE_2]);
-    visit(RULES_MANAGEMENT_URL);
-    waitForPrebuiltDetectionRulesToBeLoaded();
-    disableAutoRefresh();
-  });
+describe(
+  'Rules table: selection',
+  { tags: ['@ess', '@serverless', '@skipInServerlessMKI'] },
+  () => {
+    beforeEach(() => {
+      login();
+      /* Create and install two mock rules */
+      createAndInstallMockedPrebuiltRules([RULE_1, RULE_2]);
+      visit(RULES_MANAGEMENT_URL);
+      waitForPrebuiltDetectionRulesToBeLoaded();
+      disableAutoRefresh();
+    });
 
-  it('should correctly update the selection label when rules are individually selected and unselected', () => {
-    waitForPrebuiltDetectionRulesToBeLoaded();
+    it('should correctly update the selection label when rules are individually selected and unselected', () => {
+      waitForPrebuiltDetectionRulesToBeLoaded();
 
-    selectRulesByName(['Test rule 1', 'Test rule 2']);
+      selectRulesByName(['Test rule 1', 'Test rule 2']);
 
-    cy.get(SELECTED_RULES_NUMBER_LABEL).should('contain.text', '2');
+      cy.get(SELECTED_RULES_NUMBER_LABEL).should('contain.text', '2');
 
-    unselectRulesByName(['Test rule 1', 'Test rule 2']);
+      unselectRulesByName(['Test rule 1', 'Test rule 2']);
 
-    cy.get(SELECTED_RULES_NUMBER_LABEL).should('contain.text', '0');
-  });
+      cy.get(SELECTED_RULES_NUMBER_LABEL).should('contain.text', '0');
+    });
 
-  it('should correctly update the selection label when rules are bulk selected and then bulk un-selected', () => {
-    waitForPrebuiltDetectionRulesToBeLoaded();
+    it('should correctly update the selection label when rules are bulk selected and then bulk un-selected', () => {
+      waitForPrebuiltDetectionRulesToBeLoaded();
 
-    cy.get(SELECT_ALL_RULES_BTN).click();
+      cy.get(SELECT_ALL_RULES_BTN).click();
 
-    getAvailablePrebuiltRulesCount().then((availablePrebuiltRulesCount) => {
-      cy.get(SELECTED_RULES_NUMBER_LABEL).should('contain.text', availablePrebuiltRulesCount);
-    });
+      getAvailablePrebuiltRulesCount().then((availablePrebuiltRulesCount) => {
+        cy.get(SELECTED_RULES_NUMBER_LABEL).should('contain.text', availablePrebuiltRulesCount);
+      });
 
-    // Un-select all rules via the Bulk Selection button from the Utility bar
-    cy.get(SELECT_ALL_RULES_BTN).click();
+      // Un-select all rules via the Bulk Selection button from the Utility bar
+      cy.get(SELECT_ALL_RULES_BTN).click();
 
-    // Current selection should be 0 rules
-    cy.get(SELECTED_RULES_NUMBER_LABEL).should('contain.text', '0');
-    // Bulk selection button should be back to displaying all rules
-    getAvailablePrebuiltRulesCount().then((availablePrebuiltRulesCount) => {
-      cy.get(SELECT_ALL_RULES_BTN).should('contain.text', availablePrebuiltRulesCount);
+      // Current selection should be 0 rules
+      cy.get(SELECTED_RULES_NUMBER_LABEL).should('contain.text', '0');
+      // Bulk selection button should be back to displaying all rules
+      getAvailablePrebuiltRulesCount().then((availablePrebuiltRulesCount) => {
+        cy.get(SELECT_ALL_RULES_BTN).should('contain.text', availablePrebuiltRulesCount);
+      });
     });
-  });
 
-  it('should correctly update the selection label when rules are bulk selected and then unselected via the table select all checkbox', () => {
-    waitForPrebuiltDetectionRulesToBeLoaded();
+    it('should correctly update the selection label when rules are bulk selected and then unselected via the table select all checkbox', () => {
+      waitForPrebuiltDetectionRulesToBeLoaded();
 
-    cy.get(SELECT_ALL_RULES_BTN).click();
+      cy.get(SELECT_ALL_RULES_BTN).click();
 
-    getAvailablePrebuiltRulesCount().then((availablePrebuiltRulesCount) => {
-      cy.get(SELECTED_RULES_NUMBER_LABEL).should('contain.text', availablePrebuiltRulesCount);
-    });
+      getAvailablePrebuiltRulesCount().then((availablePrebuiltRulesCount) => {
+        cy.get(SELECTED_RULES_NUMBER_LABEL).should('contain.text', availablePrebuiltRulesCount);
+      });
 
-    // Un-select all rules via the Un-select All checkbox from the table
-    cy.get(SELECT_ALL_RULES_ON_PAGE_CHECKBOX).click();
+      // Un-select all rules via the Un-select All checkbox from the table
+      cy.get(SELECT_ALL_RULES_ON_PAGE_CHECKBOX).click();
 
-    // Current selection should be 0 rules
-    cy.get(SELECTED_RULES_NUMBER_LABEL).should('contain.text', '0');
-    // Bulk selection button should be back to displaying all rules
-    getAvailablePrebuiltRulesCount().then((availablePrebuiltRulesCount) => {
-      cy.get(SELECT_ALL_RULES_BTN).should('contain.text', availablePrebuiltRulesCount);
+      // Current selection should be 0 rules
+      cy.get(SELECTED_RULES_NUMBER_LABEL).should('contain.text', '0');
+      // Bulk selection button should be back to displaying all rules
+      getAvailablePrebuiltRulesCount().then((availablePrebuiltRulesCount) => {
+        cy.get(SELECT_ALL_RULES_BTN).should('contain.text', availablePrebuiltRulesCount);
+      });
     });
-  });
-});
+  }
+);

x-pack/test/security_solution_cypress/cypress/e2e/entity_analytics/entity_flyout.cy.ts

@@ -144,7 +144,7 @@ describe(
       });
 
       // https://github.com/elastic/kibana/issues/179248
-      describe('Managed data section', { tags: ['@skipInServerless'] }, () => {
+      describe('Managed data section', { tags: ['@skipInServerlessMKI'] }, () => {
         beforeEach(() => {
           mockFleetInstalledIntegrations([
             {

x-pack/test/security_solution_cypress/cypress/e2e/explore/inspect/inspect_button.cy.ts

@@ -27,7 +27,7 @@ const DATA_VIEW = 'auditbeat-*';
 // FLAKY: https://github.com/elastic/kibana/issues/178367
 describe.skip(
   'Inspect Explore pages',
-  { tags: ['@ess', '@serverless', '@skipInServerless'] },
+  { tags: ['@ess', '@serverless', '@skipInServerlessMKI'] },
   () => {
     before(() => {
       // illegal_argument_exception: unknown setting [index.lifecycle.name]

x-pack/test/security_solution_cypress/cypress/e2e/explore/ml/ml_conditional_links.cy.ts

@@ -27,7 +27,7 @@ import {
 } from '../../../urls/ml_conditional_links';
 
 // FLAKY: https://github.com/elastic/kibana/issues/180748
-describe.skip('ml conditional links', { tags: ['@ess', '@skipInServerless'] }, () => {
+describe.skip('ml conditional links', { tags: ['@ess', '@skipInServerlessMKI'] }, () => {
   beforeEach(() => {
     login();
   });

x-pack/test/security_solution_cypress/cypress/e2e/explore/overview/overview.cy.ts

@@ -66,7 +66,7 @@ describe('Overview Page', { tags: ['@ess', '@serverless'] }, () => {
   });
 });
 
-describe('Overview page with no data', { tags: '@skipInServerless' }, () => {
+describe('Overview page with no data', { tags: '@skipInServerlessMKI' }, () => {
   it('Splash screen should be here', () => {
     login();
     visitWithTimeRange(OVERVIEW_URL);

x-pack/test/security_solution_cypress/cypress/e2e/investigations/timelines/unified_components/query_tab.cy.ts

@@ -28,7 +28,7 @@ import { ALERTS_URL } from '../../../../urls/navigation';
 describe(
   'Unsaved Timeline query tab',
   {
-    tags: ['@ess', '@serverless', '@skipInServerless'],
+    tags: ['@ess', '@serverless', '@skipInServerlessMKI'],
     env: {
       ftrConfig: {
         kbnServerArgs: [