Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution] Adds serverlessQA tag to the API tests #180773

Merged
merged 19 commits into from
Apr 23, 2024
Merged

[Security Solution] Adds serverlessQA tag to the API tests #180773

Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
973d5da
adds serverlessQA tag to API tests
MadameSheema Apr 15, 2024
ec2c07b
aling tags with Cypress tests
MadameSheema Apr 15, 2024
77fa125
Merge branch 'main' into api/serverlessQA
MadameSheema Apr 15, 2024
08b8ebc
Merge branch 'main' into api/serverlessQA
MadameSheema Apr 15, 2024
b052b7f
adds missing file
MadameSheema Apr 15, 2024
dae1ce9
fixes
MadameSheema Apr 16, 2024
2137e9e
more fixes
MadameSheema Apr 16, 2024
3980615
adjustments
MadameSheema Apr 16, 2024
39270f7
adds tags information to the readme
MadameSheema Apr 17, 2024
0cc8b13
Applying Yara's comment: we need to add serverlessQA to the suites t…
MadameSheema Apr 17, 2024
37d3b3a
Merge branch 'main' into api/serverlessQA
MadameSheema Apr 17, 2024
cfdaefb
Update .buildkite/pipelines/security_solution/api_integration_serverl…
MadameSheema Apr 18, 2024
dc14e41
Update x-pack/test/security_solution_api_integration/README.md
MadameSheema Apr 18, 2024
6ff7457
Update .buildkite/pipelines/security_solution/api_integration_serverl…
MadameSheema Apr 18, 2024
78760a6
Merge branch 'main' into api/serverlessQA
MadameSheema Apr 18, 2024
1a6dc9a
updates readme
MadameSheema Apr 22, 2024
7dae577
updates package.json
MadameSheema Apr 22, 2024
3e1dd97
updating tags
MadameSheema Apr 22, 2024
0ac31e6
Merge branch 'main' into api/serverlessQA
MadameSheema Apr 22, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
389 changes: 389 additions & 0 deletions .buildkite/pipelines/security_solution/api_integration_serverless_release.yml
View file
Open in desktop

Large diffs are not rendered by default.

17 changes: 13 additions & 4 deletions x-pack/test/security_solution_api_integration/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,16 +15,25 @@ we have introduced the `detection_response` directory to consolidate all the int

- In this directory, Mocha tagging is utilized to assign tags to specific test suites and individual test cases. This tagging system enables the ability to selectively apply tags to test suites and test cases, facilitating the exclusion of specific test cases within a test suite as needed.

- There are three primary tags that have been defined: @ess, @serverless, and @brokenInServerless

- Test suites and cases are prefixed with specific tags to determine their execution in particular environments or to exclude them from specific environments.

- We are using the following tags:
* `@ess`: Runs in an ESS environment (on-prem installation) as part of the CI validation on PRs.

* `@serverless`: Runs in the first quality gate and in the periodic pipeline.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only other mention of "first quality gate" is in the cypress README; I think it's worth defining these terms in some detail since they're not going to be self-evident to everyone.

Same thing for "periodic pipeline," "second quality gate," and "all quality gates."

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @rylnd everything is already documented.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MadameSheema it's wonderful that it's documented, but how does someone reading this README find those definitions? You shared links to quality gates and periodic pipeline with me individually, but that process does not scale, so I'm proposing that we somehow reference that documentation here.

If we are reluctant/unable to share those docs because they're internal, I think that might be indication that this README either covers too much or is in the wrong place; our public documentation shouldn't reference internal processes that the reader has no insight on.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

++ to @rylnd's comment

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, share it with you in private due to being internal documentation, tbh I don't know what is best, thoughts? What do you prefer?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the ideal approach in my mind would be:

  1. Internal documentation associating the cypress tags to our CI/QA infrastructure/processes
  2. Public documentation with definitions appropriate to those not involved in our operations

I'm imagining a contributor who's written a cypress test for a feature they'd like to add; I would not expect them to know/care about any of these tags. As part of the review process, an Elastician would tell them which tags need to be added to their tests.

Generally: if we can't provide the reader with enough information to determine whether they should use the tag, I don't think that it's helpful to mention it at all.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe a passing mention of the tags as "related to internal release processes" would be a way to both define the tags but also deemphasize them, here?


* `@serverlessQA`: Runs in the second quality gate.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This tag seems to be strongly tied to the concept of a "release" (as evidenced in the command names, and this PR's description), but that isn't really reflected in either the README here nor in the tag name itself.

Would @serverlessRelease be a more accurate and descriptive tag name?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As part of the release, the tests might be executed as part of 2 other quality gates and we might have some on the second one that we don't want to execute it on the third one.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO referencing the "gate order" might be the most straightforward way to express this; it's not intuitive that QA represents "second gate," nor what the other gates are.


* `@skipInEss`: Skipped for ESS environment.

* `@skipInServerless`: Skipped for all quality gates and periodic pipeline.

ex:
```
describe('@serverless @ess create_rules', () => { ==> tests in this suite will run in both Ess and Serverless
describe('creating rules', () => {});

describe('@brokenInServerless missing timestamps', () => {}); ==> tests in this suite will be excluded in Serverless
describe('@skipInServerless missing timestamps', () => {}); ==> tests in this suite will be excluded in Serverless
rylnd marked this conversation as resolved.
Show resolved Hide resolved

```

Expand Down Expand Up @@ -86,7 +95,7 @@ In this project, you can run various commands to execute tests and workflows, ea
```
3. **Run tests for "exception_workflows" using the serverless runner in the "qaEnv" environment:**
```shell
npm run run-tests:dr:default exceptions/workflows serverless qaEnv
npm run run-tests:dr:default exceptions/workflows serverless qaPeriodicEnv
```
4. **Run the server for "exception_workflows" in the "essEnv" environment:**
```shell
Expand Down
2 changes: 1 addition & 1 deletion x-pack/test/security_solution_api_integration/config/ess/config.base.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ export function createTestConfig(options: CreateTestConfigOptions, testFiles?: s
],
},
mochaOpts: {
grep: '/^(?!.*@brokenInEss).*@ess.*/',
grep: '/^(?!.*@skipInEss).*@ess.*/',
},
};
};
Expand Down
2 changes: 1 addition & 1 deletion x-pack/test/security_solution_api_integration/config/serverless/config.base.essentials.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ export function createTestConfig(options: CreateTestConfigOptions) {

mochaOpts: {
...svlSharedConfig.get('mochaOpts'),
grep: '/^(?!.*@brokenInServerless).*@serverless.*/',
grep: '/^(?!.*@skipInServerless).*@serverless.*/',
},
};
};
Expand Down
2 changes: 1 addition & 1 deletion x-pack/test/security_solution_api_integration/config/serverless/config.base.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ export function createTestConfig(options: CreateTestConfigOptions) {

mochaOpts: {
...svlSharedConfig.get('mochaOpts'),
grep: '/^(?!.*@brokenInServerless).*@serverless.*/',
grep: '/^(?!.*@skipInServerless).*@serverless.*/',
},
};
};
Expand Down
114 changes: 76 additions & 38 deletions x-pack/test/security_solution_api_integration/package.json
View file
Open in desktop

Large diffs are not rendered by default.

10 changes: 7 additions & 3 deletions x-pack/test/security_solution_api_integration/scripts/index.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,15 +21,19 @@ let grepArgs = [];
if (type !== 'server') {
switch (environment) {
case 'serverlessEnv':
grepArgs = ['--grep', '/^(?!.*@brokenInServerless).*@serverless.*/'];
grepArgs = ['--grep', '/^(?!.*@skipInServerless).*@serverless.*/'];
break;

case 'essEnv':
grepArgs = ['--grep', '/^(?!.*@brokenInEss).*@ess.*/'];
grepArgs = ['--grep', '/^(?!.*@skipInEss).*@ess.*/'];
break;

case 'qaPeriodicEnv':
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do I get it right all Serverless tests besides skipped ones will run periodically against MKI env?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exactly :)

grepArgs = ['--grep', '/^(?!.*@skipInServerless).*@serverless.*/'];
break;

case 'qaEnv':
grepArgs = ['--grep', '/^(?!.*@brokenInServerless|.*@skipInQA).*@serverless.*/'];
grepArgs = ['--grep', '/^(?!.*@skipInServerless).*@serverlessQA.*/'];
break;

default:
Expand Down
2 changes: 1 addition & 1 deletion ...s/detections_response/detection_engine/actions/trial_license_complete_tier/add_actions.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ export default ({ getService }: FtrProviderContext) => {
const log = getService('log');
const es = getService('es');

describe('@serverless @ess add_actions', () => {
describe('@serverless @serverlessQA @ess add_actions', () => {
describe('adding actions', () => {
beforeEach(async () => {
await es.indices.delete({ index: 'logs-test', ignore_unavailable: true });
Expand Down
2 changes: 1 addition & 1 deletion ...ections_response/detection_engine/actions/trial_license_complete_tier/check_privileges.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ export default ({ getService }: FtrProviderContext) => {
const log = getService('log');
const es = getService('es');

describe('@ess @serverless @brokenInServerless check_privileges', () => {
describe('@ess @serverless @skipInServerless check_privileges', () => {
before(async () => {
await esArchiver.load('x-pack/test/functional/es_archives/auditbeat/hosts');
await esArchiver.load('x-pack/test/functional/es_archives/security_solution/alias');
Expand Down
10 changes: 6 additions & 4 deletions ...etections_response/detection_engine/actions/trial_license_complete_tier/update_actions.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,8 @@ export default ({ getService }: FtrProviderContext) => {
expect(body?.execution_summary?.last_execution?.status).toBe('succeeded');
});

it('@skipInQA expects an updated rule with a webhook action and meta field runs successfully', async () => {
// Broken in MKI environment, needs triage
it('@skipInServerless expects an updated rule with a webhook action and meta field runs successfully', async () => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a comment here:

// Broken in MKI environment, needs triage

Just so we know it needs follow up.

const webhookAction = await createWebHookRuleAction(supertest);

await supertest
Expand Down Expand Up @@ -187,7 +188,8 @@ export default ({ getService }: FtrProviderContext) => {
expect(body?.execution_summary?.last_execution?.status).toBe('succeeded');
});

it('@skipInQA adds a webhook to an immutable rule', async () => {
// Broken in MKI environment, needs triage
it('@skipInServerless adds a webhook to an immutable rule', async () => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a comment here:

// Broken in MKI environment, needs triage

Just so we know it needs follow up.

const immutableRule = await getImmutableRule();
const webhookAction = await createWebHookRuleAction(supertest);
const ruleAction = {
Expand All @@ -210,7 +212,7 @@ export default ({ getService }: FtrProviderContext) => {
expect(updatedRule.throttle).toEqual(immutableRule.throttle);
});

it('@skipInQA should be able to create a new webhook action, attach it to an immutable rule and the count of prepackaged rules should not increase. If this fails, suspect the immutable tags are not staying on the rule correctly.', async () => {
it('@skipInServerless should be able to create a new webhook action, attach it to an immutable rule and the count of prepackaged rules should not increase. If this fails, suspect the immutable tags are not staying on the rule correctly.', async () => {
const immutableRule = await getImmutableRule();
const hookAction = await createWebHookRuleAction(supertest);
const ruleToUpdate = getRuleWithWebHookAction(
Expand All @@ -224,7 +226,7 @@ export default ({ getService }: FtrProviderContext) => {
expect(status.rules_not_installed).toBe(0);
});

it('@skipInQA should be able to create a new webhook action, attach it to an immutable rule and the rule should stay immutable when searching against immutable tags', async () => {
it('@skipInServerless should be able to create a new webhook action, attach it to an immutable rule and the rule should stay immutable when searching against immutable tags', async () => {
const immutableRule = await getImmutableRule();
const webhookAction = await createWebHookRuleAction(supertest);
const ruleAction = {
Expand Down
2 changes: 1 addition & 1 deletion ...etections_response/detection_engine/alerts/basic_license_essentials_tier/field_aliases.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ export default ({ getService }: FtrProviderContext) => {
name: string;
}

describe('@ess @serverless Tests involving aliases of source indexes and the alerts index', () => {
describe('@ess @serverless @serverlessQA Tests involving aliases of source indexes and the alerts index', () => {
before(async () => {
await esArchiver.load('x-pack/test/functional/es_archives/security_solution/alias');
});
Expand Down
2 changes: 1 addition & 1 deletion ...detections_response/detection_engine/alerts/basic_license_essentials_tier/query_alerts.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ export default ({ getService }: FtrProviderContext) => {
const log = getService('log');
const es = getService('es');

describe('@ess @serverless query_signals_route and find_alerts_route', () => {
describe('@ess @serverless @serverlessQA query_signals_route and find_alerts_route', () => {
describe('validation checks', () => {
it('should not give errors when querying and the alerts index does exist and is empty', async () => {
await createAlertsIndex(supertest, log);
Expand Down
4 changes: 2 additions & 2 deletions ...tections_response/detection_engine/alerts/basic_license_essentials_tier/set_alert_tags.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,8 @@ export default ({ getService }: FtrProviderContext) => {
const dataPathBuilder = new EsArchivePathBuilder(isServerless);
const path = dataPathBuilder.getPath('auditbeat/hosts');

// Intentionally setting as @skipInQA, keeping tests running in MKI that should block release
describe('@ess @serverless @skipInQA set_alert_tags', () => {
// Intentionally setting as @skipInServerless, keeping tests running in MKI that should block release
describe('@ess @serverless @skipInServerless set_alert_tags', () => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one's a bit tricky - we technically need to add serverlessQA to any test suites that do not have @skipInQA and remove @skipInQA from test suites that have this comment about intentionally setting @skipInQA, because we do want those run in serverless, just not in MKI.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

because we do want those run in serverless, just not in MKI.

Per the agreement we did on the Cypress PR: #179737 this is not possible without having an extra tag. Currently for both API and cypress:

  • A serverless tests is executed in CI, periodic pipeline (MKI) and release process (MKI)
  • A serverless tests is executed in CI and periodic pipeline (MKI)
  • A serverless test is not executed

describe('validation checks', () => {
it('should give errors when no alert ids are provided', async () => {
const { body } = await supertest
Expand Down
3 changes: 1 addition & 2 deletions ...s_response/detection_engine/alerts/trial_license_complete_tier/assignments/assignments.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,7 @@ export default ({ getService }: FtrProviderContext) => {
const dataPathBuilder = new EsArchivePathBuilder(isServerless);
const path = dataPathBuilder.getPath('auditbeat/hosts');

// Intentionally setting as @skipInQA, keeping tests running in MKI that should block release
describe('@ess @serverless @skipInQA Alert User Assignment - ESS & Serverless', () => {
describe('@ess @serverless Alert User Assignment - ESS & Serverless', () => {
describe('validation checks', () => {
it('should give errors when no alert ids are provided', async () => {
const { body } = await supertest
Expand Down
3 changes: 1 addition & 2 deletions ...detection_engine/alerts/trial_license_complete_tier/assignments/assignments_serverless.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,7 @@ export default ({ getService }: FtrProviderContext) => {
const dataPathBuilder = new EsArchivePathBuilder(isServerless);
const path = dataPathBuilder.getPath('auditbeat/hosts');

// Intentionally setting as @skipInQA, keeping tests running in MKI that should block release
describe('@serverless @skipInQA Alert User Assignment - Serverless', () => {
describe('@serverless Alert User Assignment - Serverless', () => {
before(async () => {
await esArchiver.load(path);
});
Expand Down
2 changes: 1 addition & 1 deletion ...s_response/detection_engine/alerts/trial_license_complete_tier/document_level_security.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ export default ({ getService }: FtrProviderContext) => {
const esArchiver = getService('esArchiver');
const security = getService('security');

describe('@ess @serverless @brokenInServerless find alert with/without doc level security', () => {
describe('@ess @serverless @skipInServerless find alert with/without doc level security', () => {
before(async () => {
await security.role.create(
roleToAccessSecuritySolution.name,
Expand Down
2 changes: 1 addition & 1 deletion .../exceptions/operators_data_types/date_numeric_types/basic_license_essentials_tier/date.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ export default ({ getService }: FtrProviderContext) => {
const log = getService('log');
const es = getService('es');

describe('@serverless @ess Rule exception operators for data type date', () => {
describe('@serverless @serverlessQA @ess Rule exception operators for data type date', () => {
before(async () => {
await esArchiver.load('x-pack/test/functional/es_archives/rule_exceptions/date');
});
Expand Down
2 changes: 1 addition & 1 deletion ...xceptions/operators_data_types/date_numeric_types/basic_license_essentials_tier/double.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ export default ({ getService }: FtrProviderContext) => {
const log = getService('log');
const es = getService('es');

describe('@serverless @ess Rule exception operators for data type double', () => {
describe('@serverless @serverlessQA @ess Rule exception operators for data type double', () => {
before(async () => {
await esArchiver.load('x-pack/test/functional/es_archives/rule_exceptions/double');
await esArchiver.load('x-pack/test/functional/es_archives/rule_exceptions/double_as_string');
Expand Down
2 changes: 1 addition & 1 deletion ...exceptions/operators_data_types/date_numeric_types/basic_license_essentials_tier/float.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ export default ({ getService }: FtrProviderContext) => {
const log = getService('log');
const es = getService('es');

describe('@serverless @ess Rule exception operators for data type float', () => {
describe('@serverless @serverlessQA @ess Rule exception operators for data type float', () => {
before(async () => {
await esArchiver.load('x-pack/test/functional/es_archives/rule_exceptions/float');
await esArchiver.load('x-pack/test/functional/es_archives/rule_exceptions/float_as_string');
Expand Down
2 changes: 1 addition & 1 deletion ...ceptions/operators_data_types/date_numeric_types/basic_license_essentials_tier/integer.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ export default ({ getService }: FtrProviderContext) => {
const log = getService('log');
const es = getService('es');

describe('@serverless @ess Rule exception operators for data type integer', () => {
describe('@serverless @serverlessQA @ess Rule exception operators for data type integer', () => {
before(async () => {
await esArchiver.load('x-pack/test/functional/es_archives/rule_exceptions/integer');
await esArchiver.load('x-pack/test/functional/es_archives/rule_exceptions/integer_as_string');
Expand Down
2 changes: 1 addition & 1 deletion .../detection_engine/exceptions/operators_data_types/ips/basic_license_essentials_tier/ip.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ export default ({ getService }: FtrProviderContext) => {
const log = getService('log');
const es = getService('es');

describe('@serverless @ess Rule exception operators for data type ip', () => {
describe('@serverless @serverless @ess Rule exception operators for data type ip', () => {
before(async () => {
await esArchiver.load('x-pack/test/functional/es_archives/rule_exceptions/ip');
});
Expand Down
2 changes: 1 addition & 1 deletion ...tion_engine/exceptions/operators_data_types/ips/basic_license_essentials_tier/ip_array.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ export default ({ getService }: FtrProviderContext) => {
const log = getService('log');
const es = getService('es');

describe('@serverless @ess Rule exception operators for data type ip', () => {
describe('@serverless @serverlessQA @ess Rule exception operators for data type ip', () => {
before(async () => {
await esArchiver.load('x-pack/test/functional/es_archives/rule_exceptions/ip_as_array');
});
Expand Down
2 changes: 1 addition & 1 deletion ...n_engine/exceptions/operators_data_types/keyword/basic_license_essentials_tier/keyword.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ export default ({ getService }: FtrProviderContext) => {
const log = getService('log');
const es = getService('es');

describe('@serverless @ess Rule exception operators for data type keyword', () => {
describe('@serverless @serverlessQA @ess Rule exception operators for data type keyword', () => {
before(async () => {
await esArchiver.load('x-pack/test/functional/es_archives/rule_exceptions/keyword');
});
Expand Down
2 changes: 1 addition & 1 deletion ...ne/exceptions/operators_data_types/keyword/basic_license_essentials_tier/keyword_array.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ export default ({ getService }: FtrProviderContext) => {
const log = getService('log');
const es = getService('es');

describe('@serverless @ess Rule exception operators for data type keyword', () => {
describe('@serverles @serverlessQA @ess Rule exception operators for data type keyword', () => {
before(async () => {
await esArchiver.load('x-pack/test/functional/es_archives/rule_exceptions/keyword_as_array');
});
Expand Down
2 changes: 1 addition & 1 deletion ...tection_engine/exceptions/operators_data_types/long/basic_license_essentials_tier/long.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ export default ({ getService }: FtrProviderContext) => {
const log = getService('log');
const es = getService('es');

describe('@serverless @ess Rule exception operators for data type long', () => {
describe('@serverless @serverlessQA @ess Rule exception operators for data type long', () => {
before(async () => {
await esArchiver.load('x-pack/test/functional/es_archives/rule_exceptions/long');
await esArchiver.load('x-pack/test/functional/es_archives/rule_exceptions/long_as_string');
Expand Down
2 changes: 1 addition & 1 deletion ...tection_engine/exceptions/operators_data_types/text/basic_license_essentials_tier/text.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ export default ({ getService }: FtrProviderContext) => {
const log = getService('log');
const es = getService('es');

describe('@serverless @ess Rule exception operators for data type text', () => {
describe('@serverless @serverlessQA @ess Rule exception operators for data type text', () => {
before(async () => {
await esArchiver.load('x-pack/test/functional/es_archives/rule_exceptions/text');
await esArchiver.load('x-pack/test/functional/es_archives/rule_exceptions/text_no_spaces');
Expand Down
2 changes: 1 addition & 1 deletion ...n_engine/exceptions/operators_data_types/text/basic_license_essentials_tier/text_array.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ export default ({ getService }: FtrProviderContext) => {
const log = getService('log');
const es = getService('es');

describe('@serverless @ess Rule exception operators for data type text', () => {
describe('@serverless @serverlessQA @ess Rule exception operators for data type text', () => {
before(async () => {
await esArchiver.load('x-pack/test/functional/es_archives/rule_exceptions/text_as_array');
});
Expand Down
2 changes: 1 addition & 1 deletion ...n_engine/exceptions/workflows/basic_license_essentials_tier/create_endpoint_exceptions.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ export default ({ getService }: FtrProviderContext) => {
const log = getService('log');
const es = getService('es');

describe('@serverless @ess create_endpoint_exceptions', () => {
describe('@serverless @serverlessQA @ess create_endpoint_exceptions', () => {
before(async () => {
await esArchiver.load(
'x-pack/test/functional/es_archives/rule_exceptions/endpoint_without_host_type'
Expand Down
2 changes: 1 addition & 1 deletion ...ction_engine/exceptions/workflows/basic_license_essentials_tier/create_rule_exceptions.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ export default ({ getService }: FtrProviderContext) => {
const config = getService('config');
const ELASTICSEARCH_USERNAME = config.get('servers.kibana.username');

describe('@serverless @ess create "rule_default" exceptions', () => {
describe('@serverless @serverlessQA @ess create "rule_default" exceptions', () => {
before(async () => {
await createAlertsIndex(supertest, log);
});
Expand Down
3 changes: 1 addition & 2 deletions ...ngine/exceptions/workflows/basic_license_essentials_tier/exception_comments_serverless.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,7 @@ export default ({ getService }: FtrProviderContext) => {
const log = getService('log');
const supertestWithoutAuth = getService('supertestWithoutAuth');

// @skipInQA purposefully - only running tests in MKI whose failure should block release
describe('@serverless @skipInQA exception item comments - serverless specific behavior', () => {
describe('@serverless exception item comments - serverless specific behavior', () => {
describe('Rule Exceptions', () => {
afterEach(async () => {
await deleteAllExceptions(supertest, log);
Expand Down
3 changes: 1 addition & 2 deletions ...on_engine/exceptions/workflows/basic_license_essentials_tier/exceptions_data_integrity.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,7 @@ export default ({ getService }: FtrProviderContext) => {
const log = getService('log');
const es = getService('es');

// @skipInQA purposefully - only running tests in MKI whose failure should block release
describe('@serverless @ess @skipInQA exceptions data integrity', () => {
describe('@serverless @ess exceptions data integrity', () => {
afterEach(async () => {
await deleteAllAlerts(supertest, log, es);
await deleteAllRules(supertest, log);
Expand Down
2 changes: 1 addition & 1 deletion ...gine/exceptions/workflows/basic_license_essentials_tier/find_rule_exception_references.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ export default ({ getService }: FtrProviderContext) => {
const log = getService('log');
const es = getService('es');

describe('@serverless @ess find_rule_exception_references', () => {
describe('@serverless @serverlessQA @ess find_rule_exception_references', () => {
before(async () => {
await createAlertsIndex(supertest, log);
});
Expand Down
3 changes: 1 addition & 2 deletions ...nse/detection_engine/exceptions/workflows/basic_license_essentials_tier/prebuilt_rules.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,7 @@ export default ({ getService }: FtrProviderContext) => {
const log = getService('log');
const es = getService('es');

// @skipInQA purposefully - only running tests in MKI whose failure should block release
describe('@serverless @ess @skipInQA exceptions workflows for prebuilt rules', () => {
describe('@serverless @ess exceptions workflows for prebuilt rules', () => {
describe('creating rules with exceptions', () => {
beforeEach(async () => {
await createAlertsIndex(supertest, log);
Expand Down
2 changes: 1 addition & 1 deletion ...on_engine/exceptions/workflows/basic_license_essentials_tier/rule_exceptions_execution.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ export default ({ getService }: FtrProviderContext) => {
const dataPathBuilder = new EsArchivePathBuilder(isServerless);
const path = dataPathBuilder.getPath('auditbeat/hosts');

describe('@serverless @ess rule exceptions execution', () => {
describe('@serverless @serverlessQA @ess rule exceptions execution', () => {
before(async () => {
await esArchiver.load(path);
});
Expand Down
2 changes: 1 addition & 1 deletion .../detection_engine/rule_execution_logic/trial_license_complete_tier/execution_logic/eql.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ export default ({ getService }: FtrProviderContext) => {
const auditPath = dataPathBuilder.getPath('auditbeat/hosts');

// FLAKY: https://github.com/elastic/kibana/issues/180641
describe.skip('@ess @serverless EQL type rules', () => {
describe.skip('@ess @serverless @serverlessQA EQL type rules', () => {
const { indexListOfDocuments } = dataGeneratorFactory({
es,
index: 'ecs_compliant',
Expand Down
Loading