Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution] Adds tests for coverage overview page #168058

Merged
merged 52 commits into from
Jan 9, 2024
Merged
Show file tree
Hide file tree
Changes from 47 commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
fffc5f5
adds initial tests
dplumlee Oct 5, 2023
c5063f4
Merge remote-tracking branch 'upstream/main' into coverage-overview-t…
dplumlee Oct 30, 2023
146e173
fixes cypress merge
dplumlee Oct 31, 2023
2c1b76f
Merge remote-tracking branch 'upstream/main' into coverage-overview-t…
dplumlee Nov 1, 2023
053909a
Merge remote-tracking branch 'upstream/main' into coverage-overview-t…
dplumlee Nov 2, 2023
0058dc6
adds remaining cypress tests
dplumlee Nov 6, 2023
d456e10
removes duplicated ftr tests
dplumlee Nov 7, 2023
7f102aa
removes unnecessary util
dplumlee Nov 7, 2023
d845cbe
adds error cases
dplumlee Nov 7, 2023
4ecf9a9
Merge remote-tracking branch 'upstream/main' into coverage-overview-t…
dplumlee Nov 7, 2023
85008c5
moves ftr tests into new folder structure
dplumlee Nov 7, 2023
1bcf6c0
add config file
dplumlee Nov 7, 2023
4c58bcd
Merge remote-tracking branch 'upstream/main' into coverage-overview-t…
dplumlee Nov 7, 2023
e9f8e15
adds serverless config file
dplumlee Nov 7, 2023
e585ede
fixes types
dplumlee Nov 7, 2023
eeb1d23
adds back rule test
dplumlee Nov 8, 2023
7539e30
adds scripts
dplumlee Nov 9, 2023
ab56cd7
Merge remote-tracking branch 'upstream/main' into coverage-overview-t…
dplumlee Nov 13, 2023
a0f1cc4
refactors some utils
dplumlee Nov 15, 2023
7067309
Merge remote-tracking branch 'upstream/main' into coverage-overview-t…
dplumlee Nov 15, 2023
51f4bbd
addresses cypress comments
dplumlee Nov 20, 2023
b64e55a
Merge remote-tracking branch 'upstream/main' into coverage-overview-t…
dplumlee Nov 20, 2023
d8b2620
switches package.json for flaky test runner
dplumlee Nov 21, 2023
b48dc43
resets package.json
dplumlee Nov 21, 2023
72aca4d
resets package.json
dplumlee Nov 21, 2023
ad2d829
Merge remote-tracking branch 'upstream/main' into coverage-overview-t…
dplumlee Nov 21, 2023
a0343bd
removes non-security test
dplumlee Nov 21, 2023
aea805b
fixes ftr tests
dplumlee Nov 21, 2023
f248349
Merge remote-tracking branch 'upstream/main' into coverage-overview-t…
dplumlee Nov 29, 2023
e2dc4b3
fixes rest of merge conflicts
dplumlee Nov 29, 2023
e66c647
fix import path updates
dplumlee Nov 29, 2023
a240bc9
Merge remote-tracking branch 'upstream/main' into coverage-overview-t…
dplumlee Dec 1, 2023
a37bbc3
fixes non-security rule test
dplumlee Dec 1, 2023
a06fb5e
fixes import bug
dplumlee Dec 1, 2023
2172170
add alerting api service to ess config
dplumlee Dec 1, 2023
2c79610
Merge remote-tracking branch 'upstream/main' into coverage-overview-t…
dplumlee Dec 5, 2023
6d471f6
addresses comments
dplumlee Dec 6, 2023
089ee4b
adds back ess non security rule util
dplumlee Dec 6, 2023
f24d4f8
Merge remote-tracking branch 'upstream/main' into coverage-overview-t…
dplumlee Dec 13, 2023
879594d
merge fixes
dplumlee Dec 13, 2023
00b5f5f
Merge remote-tracking branch 'upstream/main' into coverage-overview-t…
dplumlee Dec 13, 2023
e57b430
Merge remote-tracking branch 'upstream/main' into coverage-overview-t…
dplumlee Dec 18, 2023
72d4166
changes script
dplumlee Dec 20, 2023
76b4469
Merge remote-tracking branch 'upstream/main' into coverage-overview-t…
dplumlee Jan 3, 2024
2cfc275
modifies mitre mock data and generation script
dplumlee Jan 4, 2024
4b13726
fix unused code
dplumlee Jan 4, 2024
a8b6d95
splits ftr test tags
dplumlee Jan 4, 2024
e18f573
fixes script conflict
dplumlee Jan 4, 2024
19282d8
Merge remote-tracking branch 'upstream/main' into coverage-overview-t…
dplumlee Jan 5, 2024
6cd40c3
addresses comments
dplumlee Jan 5, 2024
4442f7c
Merge remote-tracking branch 'upstream/main' into coverage-overview-t…
dplumlee Jan 8, 2024
636311c
merge conflicts
dplumlee Jan 8, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .buildkite/ftr_configs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -496,6 +496,8 @@ enabled:
- x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/telemetry/configs/ess.config.ts
- x-pack/test/security_solution_api_integration/test_suites/detections_response/basic_essentials_license/detection_engine/configs/serverless.config.ts
- x-pack/test/security_solution_api_integration/test_suites/detections_response/basic_essentials_license/detection_engine/configs/ess.config.ts
- x-pack/test/security_solution_api_integration/test_suites/detections_response/basic_essentials_license/rule_management/configs/serverless.config.ts
- x-pack/test/security_solution_api_integration/test_suites/detections_response/basic_essentials_license/rule_management/configs/ess.config.ts
banderror marked this conversation as resolved.
Show resolved Hide resolved
- x-pack/test/security_solution_api_integration/test_suites/lists_and_exception_lists/default_license/exception_lists_items/configs/serverless.config.ts
- x-pack/test/security_solution_api_integration/test_suites/lists_and_exception_lists/default_license/exception_lists_items/configs/ess.config.ts
- x-pack/test/security_solution_api_integration/test_suites/lists_and_exception_lists/default_license/lists_items/configs/serverless.config.ts
Expand Down
1 change: 1 addition & 0 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -1377,6 +1377,7 @@ x-pack/test/security_solution_cypress/cypress/tasks/expandable_flyout @elastic/
/x-pack/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules @elastic/security-detection-rule-management
/x-pack/plugins/security_solution/docs/testing/test_plans/detection_response/rule_management @elastic/security-detection-rule-management
/x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/prebuilt_rules @elastic/security-detection-rule-management
/x-pack/test/security_solution_api_integration/test_suites/detections_response/basic_essentials_license/rule_management @elastic/security-detection-rule-management
/x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/rule_delete @elastic/security-detection-rule-management
x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/rule_update @elastic/security-detection-rule-management
/x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/rule_patch @elastic/security-detection-rule-management
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

import { left } from 'fp-ts/lib/Either';
import { exactCheck, foldLeftRight, getPaths } from '@kbn/securitysolution-io-ts-utils';
import {
CoverageOverviewRequestBody,
CoverageOverviewRuleActivity,
CoverageOverviewRuleSource,
} from './coverage_overview_route';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Obviously not part of this PR, but these types need to be migrated to Zod. It makes testing these request schemas SO much easier and understandable.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, I saw the validation tests Dmitrii wrote with zod while writing these and thought the same. In a later PR these types definitely need to be switched over.


describe('Coverage overview request schema', () => {
test('empty object validates', () => {
const payload: CoverageOverviewRequestBody = {};

const decoded = CoverageOverviewRequestBody.decode(payload);
const checked = exactCheck(payload, decoded);
const message = foldLeftRight(checked);

expect(getPaths(left(message.errors))).toEqual([]);
expect(message.schema).toEqual(payload);
});

test('validates with all fields populated', () => {
const payload: CoverageOverviewRequestBody = {
filter: {
activity: [CoverageOverviewRuleActivity.Enabled, CoverageOverviewRuleActivity.Disabled],
source: [CoverageOverviewRuleSource.Custom, CoverageOverviewRuleSource.Prebuilt],
search_term: 'search term',
},
};

const decoded = CoverageOverviewRequestBody.decode(payload);
const checked = exactCheck(payload, decoded);
const message = foldLeftRight(checked);

expect(getPaths(left(message.errors))).toEqual([]);
expect(message.schema).toEqual(payload);
});

test('does NOT validate with extra fields', () => {
const payload: CoverageOverviewRequestBody & { invalid_field: string } = {
filter: {
activity: [CoverageOverviewRuleActivity.Enabled, CoverageOverviewRuleActivity.Disabled],
source: [CoverageOverviewRuleSource.Custom, CoverageOverviewRuleSource.Prebuilt],
search_term: 'search term',
},
invalid_field: 'invalid field',
};

const decoded = CoverageOverviewRequestBody.decode(payload);
const checked = exactCheck(payload, decoded);
const message = foldLeftRight(checked);

expect(getPaths(left(message.errors))).toEqual(['invalid keys "invalid_field"']);
expect(message.schema).toEqual({});
});

test('does NOT validate with invalid filter values', () => {
const payload: CoverageOverviewRequestBody = {
filter: {
// @ts-expect-error
activity: ['Wrong activity field'],
// @ts-expect-error
source: ['Wrong source field'],
search_term: 'search term',
},
};

const decoded = CoverageOverviewRequestBody.decode(payload);
const checked = exactCheck(payload, decoded);
const message = foldLeftRight(checked);

expect(getPaths(left(message.errors))).toEqual([
'Invalid value "Wrong activity field" supplied to "filter,activity"',
'Invalid value "Wrong source field" supplied to "filter,source"',
]);
expect(message.schema).toEqual({});
});

test('does NOT validate with empty filter arrays', () => {
const payload: CoverageOverviewRequestBody = {
filter: {
activity: [],
source: [],
search_term: 'search term',
},
};

const decoded = CoverageOverviewRequestBody.decode(payload);
const checked = exactCheck(payload, decoded);
const message = foldLeftRight(checked);

expect(getPaths(left(message.errors))).toEqual([
'Invalid value "[]" supplied to "filter,activity"',
'Invalid value "[]" supplied to "filter,source"',
]);
expect(message.schema).toEqual({});
});

test('does NOT validate with empty search_term', () => {
const payload: CoverageOverviewRequestBody = {
filter: {
search_term: '',
},
};

const decoded = CoverageOverviewRequestBody.decode(payload);
const checked = exactCheck(payload, decoded);
const message = foldLeftRight(checked);

expect(getPaths(left(message.errors))).toEqual([
'Invalid value "" supplied to "filter,search_term"',
]);
expect(message.schema).toEqual({});
});
});
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,9 @@ Status: `in progress`. The current test plan matches `Milestone 1 - MVP` of the

- **Rule Source**: The filter type defining rule type, current options are `prebuilt`(from elastic prebuilt rules package) and `custom`(created by user)

-**Initial filter state**: The filters present on initial page load. Rule activity will be set to `enabled`, rule source will be set to `prebuilt` and `custom` simultaneously.
- **Initial filter state**: The filters present on initial page load. Rule activity will be set to `enabled`, rule source will be set to `prebuilt` and `custom` simultaneously.

-**Dashboard containing the rule data**: The normal render of the coverage overview dashboard. Any returned rule data mapped correctly to the tile layout of all the MITRE data in a colored grid
- **Dashboard containing the rule data**: The normal render of the coverage overview dashboard. Any returned rule data mapped correctly to the tile layout of all the MITRE data in a colored grid

### Assumptions

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,14 @@
import type { UseQueryOptions } from '@tanstack/react-query';
import { useQuery, useQueryClient } from '@tanstack/react-query';
import { useCallback } from 'react';
import * as i18n from '../../../rule_management_ui/pages/coverage_overview/translations';
import type { CoverageOverviewFilter } from '../../../../../common/api/detection_engine';
import { RULE_MANAGEMENT_COVERAGE_OVERVIEW_URL } from '../../../../../common/api/detection_engine';
import { fetchCoverageOverview } from '../api';
import { buildCoverageOverviewDashboardModel } from '../../logic/coverage_overview/build_coverage_overview_dashboard_model';
import type { CoverageOverviewDashboard } from '../../model/coverage_overview/dashboard';
import { DEFAULT_QUERY_OPTIONS } from './constants';
import { useAppToasts } from '../../../../common/hooks/use_app_toasts';

const COVERAGE_OVERVIEW_QUERY_KEY = ['POST', RULE_MANAGEMENT_COVERAGE_OVERVIEW_URL];

Expand All @@ -29,6 +31,8 @@ export const useFetchCoverageOverviewQuery = (
filter: CoverageOverviewFilter = {},
options?: UseQueryOptions<CoverageOverviewDashboard>
) => {
const { addError } = useAppToasts();

return useQuery<CoverageOverviewDashboard>(
[...COVERAGE_OVERVIEW_QUERY_KEY, filter],
async ({ signal }) => {
Expand All @@ -39,6 +43,11 @@ export const useFetchCoverageOverviewQuery = (
{
...DEFAULT_QUERY_OPTIONS,
...options,
onError: (error) => {
addError(error, {
title: i18n.COVERAGE_OVERVIEW_FETCH_ERROR_TITLE,
});
},
}
);
};
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ const RuleSourceFilterComponent = ({
`}
>
<EuiPopover
id="ruleActivityFilterPopover"
id="ruleSourceFilterPopover"
button={button}
isOpen={isPopoverOpen}
closePopover={closePopover}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ const CoverageOverviewMitreTechniquePanelComponent = ({
>
<EuiFlexGroup css={{ height: '100%' }} direction="column" justifyContent="spaceBetween">
<EuiFlexItem>
<EuiText size="xs">
<EuiText data-test-subj={`coverageOverviewTechniqueTitle-${technique.id}`} size="xs">
<h4>{technique.name}</h4>
</EuiText>
{SubtechniqueInfo}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -177,3 +177,10 @@ export const CoverageOverviewDashboardInformation = i18n.translate(
"Your current coverage of MITRE ATT&CK\u00AE tactics and techniques, based on installed rules. Click a cell to view and enable a technique's rules. Rules must be mapped to the MITRE ATT&CK\u00AE framework to be displayed.",
}
);

export const COVERAGE_OVERVIEW_FETCH_ERROR_TITLE = i18n.translate(
'xpack.securitySolution.coverageOverviewDashboard.fetchErrorTitle',
{
defaultMessage: 'Failed to fetch coverage overview data',
}
);
Original file line number Diff line number Diff line change
Expand Up @@ -7251,27 +7251,123 @@ export const subtechniques: MitreSubTechnique[] = [
];

/**
* A full object of Mitre Attack Threat data that is taken directly from the `mitre_tactics_techniques.ts` file
* An array of full Mitre Attack Threat objects that are taken directly from the `mitre_tactics_techniques.ts` file
*
* Is built alongside and sampled from the data in the file so to always be valid with the most up to date MITRE ATT&CK data
*/
export const getMockThreatData = () => ({
tactic: {
name: 'Credential Access',
id: 'TA0006',
reference: 'https://attack.mitre.org/tactics/TA0006',
},
technique: {
name: 'OS Credential Dumping',
id: 'T1003',
reference: 'https://attack.mitre.org/techniques/T1003',
tactics: ['credential-access'],
export const getMockThreatData = () => [
jpdjere marked this conversation as resolved.
Show resolved Hide resolved
{
tactic: {
name: 'Credential Access',
id: 'TA0006',
reference: 'https://attack.mitre.org/tactics/TA0006',
},
technique: {
name: 'OS Credential Dumping',
id: 'T1003',
reference: 'https://attack.mitre.org/techniques/T1003',
tactics: ['credential-access'],
},
subtechnique: {
name: '/etc/passwd and /etc/shadow',
id: 'T1003.008',
reference: 'https://attack.mitre.org/techniques/T1003/008',
tactics: ['credential-access'],
techniqueId: 'T1003',
},
},
{
tactic: {
name: 'Credential Access',
id: 'TA0006',
reference: 'https://attack.mitre.org/tactics/TA0006',
},
technique: {
name: 'Steal or Forge Kerberos Tickets',
id: 'T1558',
reference: 'https://attack.mitre.org/techniques/T1558',
tactics: ['credential-access'],
},
subtechnique: {
name: 'AS-REP Roasting',
id: 'T1558.004',
reference: 'https://attack.mitre.org/techniques/T1558/004',
tactics: ['credential-access'],
techniqueId: 'T1558',
},
},
{
tactic: {
name: 'Persistence',
id: 'TA0003',
reference: 'https://attack.mitre.org/tactics/TA0003',
},
technique: {
name: 'Boot or Logon Autostart Execution',
id: 'T1547',
reference: 'https://attack.mitre.org/techniques/T1547',
tactics: ['persistence', 'privilege-escalation'],
},
subtechnique: {
name: 'Active Setup',
id: 'T1547.014',
reference: 'https://attack.mitre.org/techniques/T1547/014',
tactics: ['persistence', 'privilege-escalation'],
techniqueId: 'T1547',
},
},
{
tactic: {
name: 'Persistence',
id: 'TA0003',
reference: 'https://attack.mitre.org/tactics/TA0003',
},
technique: {
name: 'Account Manipulation',
id: 'T1098',
reference: 'https://attack.mitre.org/techniques/T1098',
tactics: ['persistence'],
},
subtechnique: {
name: 'Additional Cloud Credentials',
id: 'T1098.001',
reference: 'https://attack.mitre.org/techniques/T1098/001',
tactics: ['persistence'],
techniqueId: 'T1098',
},
},
subtechnique: {
name: '/etc/passwd and /etc/shadow',
id: 'T1003.008',
reference: 'https://attack.mitre.org/techniques/T1003/008',
tactics: ['credential-access'],
techniqueId: 'T1003',
];

/**
* An array of specifically chosen Mitre Attack Threat objects that is taken directly from the `mitre_tactics_techniques.ts` file
*
* These objects have identical technique fields but are assigned to different tactics
*/
export const getDuplicateTechniqueThreatData = () => [
{
tactic: {
name: 'Privilege Escalation',
id: 'TA0004',
reference: 'https://attack.mitre.org/tactics/TA0004',
},
technique: {
name: 'Event Triggered Execution',
id: 'T1546',
reference: 'https://attack.mitre.org/techniques/T1546',
tactics: ['privilege-escalation', 'persistence'],
},
},
{
tactic: {
name: 'Persistence',
id: 'TA0003',
reference: 'https://attack.mitre.org/tactics/TA0003',
},
technique: {
name: 'Event Triggered Execution',
id: 'T1546',
reference: 'https://attack.mitre.org/techniques/T1546',
tactics: ['privilege-escalation', 'persistence'],
},
},
});
];
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
import type { Threats } from '@kbn/securitysolution-io-ts-alerting-types';
import { getMockThreatData } from './mitre_tactics_techniques';

const { tactic, technique, subtechnique } = getMockThreatData();
const { tactic, technique, subtechnique } = getMockThreatData()[0];
const { tactics, ...mockTechnique } = technique;
const { tactics: subtechniqueTactics, ...mockSubtechnique } = subtechnique;

Expand Down
Loading