Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RAC][Security Solution] Add base Security Rule Type #105096

Merged
merged 100 commits into from
Aug 3, 2021
Merged
Changes from 1 commit
Commits
Show all changes
100 commits
Select commit Hold shift + click to select a range
e2467ac
injects bulkCreate and wrapHits to individual rule executors
marshallmain May 11, 2021
ae41613
WIP create_security_rule_type_factory based on Marshall's work in #d3…
ecezalp Jun 4, 2021
83a2f0f
removes ruleStatusService from old rule executors, fixes executor uni…
ecezalp Jun 7, 2021
5fd3f60
fixes rebase
ecezalp Jun 30, 2021
b95b6b6
Merge branch 'master' of github.com:elastic/kibana into security-rule…
madirey Jul 6, 2021
2c0236e
Rename reference_rules to rule_types
madirey Jul 6, 2021
637246c
Fix type errors
madirey Jul 6, 2021
c336531
Fix type errors in base security rule factory
madirey Jul 7, 2021
0645902
Additional improvements to types and interfaces
madirey Jul 7, 2021
72032d7
More type alignment
madirey Jul 8, 2021
7ea0928
Fix remaining type errors in query rule
madirey Jul 8, 2021
c92dbe6
Add validation / inject lists plugin
madirey Jul 9, 2021
849a428
Merge branch 'master' of github.com:elastic/kibana into security-rule…
madirey Jul 9, 2021
6fb0fc8
Formatting
madirey Jul 9, 2021
b302674
Improvements to typing
madirey Jul 9, 2021
3cd9ee7
Static typing on executors
madirey Jul 9, 2021
44eb2de
cleanup
madirey Jul 11, 2021
b4b7b56
Hook up params for query/threshold rules... includes exceptionsList a…
madirey Jul 12, 2021
14b0b6b
Merge branch 'master' of github.com:elastic/kibana into security-rule…
madirey Jul 12, 2021
f9922fa
Scaffolding for wrapHits and bulkCreate
madirey Jul 12, 2021
78601da
Add error handling / status reporting
madirey Jul 12, 2021
47f0f9c
Fixup alert type state
madirey Jul 13, 2021
5450d1b
Begin threshold
madirey Jul 13, 2021
a6a9efc
Begin work on threshold state
madirey Jul 13, 2021
a22c321
Organize rule types
madirey Jul 13, 2021
dc4f5bf
Export base security rule types
madirey Jul 13, 2021
b8185f2
Fixup lifecycle static typing
madirey Jul 13, 2021
a8c0b4e
WrapHits / bulk changes
madirey Jul 14, 2021
e25b32b
Field mappings (partial)
madirey Jul 14, 2021
a7771bd
whoops
madirey Jul 14, 2021
3daa823
Merge branch 'master' of github.com:elastic/kibana into security-rule…
madirey Jul 15, 2021
87aff9c
Remove redundant params
madirey Jul 15, 2021
5f64f3b
More flexibile implementation of bulkCreateFactory
madirey Jul 15, 2021
aa60279
Add mappings
madirey Jul 16, 2021
bc50b42
Merge branch 'master' of github.com:elastic/kibana into security-rule…
madirey Jul 19, 2021
cf13ad6
Finish query rule
madirey Jul 21, 2021
1a3393c
Revert "Remove redundant params"
madirey Jul 21, 2021
0570a37
Revert "whoops"
madirey Jul 21, 2021
ccbc66a
Fixup return types
madirey Jul 21, 2021
978984e
Use alertWithPersistence
madirey Jul 21, 2021
d6d5025
Merge branch 'master' of github.com:elastic/kibana into security-rule…
madirey Jul 21, 2021
f384e49
Fix import
madirey Jul 21, 2021
e7ee2a7
End-to-end rule mostly working
madirey Jul 22, 2021
edc4578
Fix bulkCreate
madirey Jul 22, 2021
3f1dfe0
Bug fixes
madirey Jul 24, 2021
f44c2cf
Merge branch 'master' of github.com:elastic/kibana into security-rule…
madirey Jul 24, 2021
66fdd74
Bug fixes and mapping changes
madirey Jul 24, 2021
058e576
Fix indexing
madirey Jul 25, 2021
f8ed661
cleanup
madirey Jul 25, 2021
7be4690
Fix type errors
madirey Jul 26, 2021
677659c
Merge branch 'master' of github.com:elastic/kibana into security-rule…
madirey Jul 26, 2021
484cc00
Test fixes
madirey Jul 26, 2021
69b0007
Fix query tests
madirey Jul 27, 2021
c5eca53
Merge branch 'master' of github.com:elastic/kibana into security-rule…
madirey Jul 27, 2021
15d671e
cleanup / rename kibana.rac to kibana
madirey Jul 27, 2021
95d70dc
Remove eql/threshold (for now)
madirey Jul 27, 2021
7617a3b
Move technical fields to package
madirey Jul 27, 2021
6ee2f85
Add indexAlias and buildRuleMessageFactory
madirey Jul 28, 2021
794f029
imports
madirey Jul 28, 2021
6b9cf53
type errors
madirey Jul 28, 2021
0328fe4
Change 'kibana.rac.*' to 'kibana.*'
madirey Jul 28, 2021
5bfb66c
Fix lifecycle tests
madirey Jul 28, 2021
d9b2d4c
Single alert instance
madirey Jul 28, 2021
774277e
Merge branch 'master' of github.com:elastic/kibana into security-rule…
madirey Jul 28, 2021
9b16d22
fix import
madirey Jul 28, 2021
cb46500
Fix type error
madirey Jul 28, 2021
164f549
Merge branch 'master' of github.com:elastic/kibana into security-rule…
madirey Jul 28, 2021
07a2f7e
Fix more type errors
madirey Jul 28, 2021
4bafdda
Fix query rule type test
madirey Jul 28, 2021
4c83aa3
Merge branch 'master' of github.com:elastic/kibana into security-rule…
madirey Jul 28, 2021
d115a95
revert to previous ts-expect-error
madirey Jul 28, 2021
aeb052f
type errors again
madirey Jul 28, 2021
b93ed2c
types / linting
madirey Jul 28, 2021
44ce886
Merge branch 'master' of github.com:elastic/kibana into security-rule…
madirey Jul 28, 2021
3284812
General readability improvements
madirey Jul 29, 2021
f7dfd3f
Add invariant function from Dmitrii's branch
madirey Jul 29, 2021
2ccc6c2
Use invariant and constants
madirey Jul 29, 2021
8c12651
Merge branch 'master' of github.com:elastic/kibana into security-rule…
madirey Jul 29, 2021
7af0175
Improvements to field mappings
madirey Jul 29, 2021
aa1a49f
More test failure fixes
madirey Jul 29, 2021
1a65e63
Merge branch 'master' of github.com:elastic/kibana into security-rule…
madirey Jul 29, 2021
7646f5e
Add refresh param for bulk create
madirey Jul 29, 2021
3175fed
Update more field refs
madirey Jul 29, 2021
dcae14d
Actually use refresh param
madirey Jul 29, 2021
5e3e3bb
cleanup
madirey Jul 29, 2021
eb1b900
test fixes
madirey Jul 30, 2021
c44bd32
Merge branch 'master' of github.com:elastic/kibana into security-rule…
madirey Jul 30, 2021
d76eecd
changes to rule creation script
madirey Jul 30, 2021
01529fc
Fix created signals count
madirey Jul 30, 2021
668dddc
Use ruleId
madirey Jul 30, 2021
8d19387
Updates to bulk indexing
madirey Jul 30, 2021
44d0a2a
Mapping updates
madirey Jul 30, 2021
091b64a
Cannot use 'strict' for dynamic setting
madirey Jul 30, 2021
7f519e8
Merge branch 'master' of github.com:elastic/kibana into security-rule…
madirey Jul 30, 2021
3ef3de9
Merge branch 'master' into security-rule-type
kibanamachine Aug 2, 2021
cfbdd20
Merge branch 'master' into security-rule-type
kibanamachine Aug 2, 2021
75260cf
Merge branch 'master' of github.com:elastic/kibana into security-rule…
madirey Aug 3, 2021
a4b37b0
Merge branch 'security-rule-type' of github.com:madirey/kibana into s…
madirey Aug 3, 2021
9c02627
Merge branch 'master' of github.com:elastic/kibana into security-rule…
madirey Aug 3, 2021
1882710
Fix type errors from master
madirey Aug 3, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
cleanup
  • Loading branch information
madirey committed Jul 11, 2021
commit 44eb2de2169edf977002386b56eaa967706e47a1
2 changes: 1 addition & 1 deletion x-pack/plugins/rule_registry/server/types.ts
Original file line number Diff line number Diff line change
@@ -24,7 +24,7 @@ export type AlertTypeExecutor<
TParams extends AlertTypeParams = {},
TAlertInstanceContext extends AlertInstanceContext = {},
TServices extends Record<string, any> = {},
TState extends AlertTypeState = {},
TState extends AlertTypeState = {}
> = (
options: Parameters<SimpleAlertType<TParams, TAlertInstanceContext>['executor']>[0] & {
services: TServices;
Original file line number Diff line number Diff line change
@@ -19,6 +19,11 @@ import { SetupPlugins } from '../../../../target/types/server/plugin';
import { eqlRuleParams, EqlRuleParams } from '../schemas/rule_schemas';
import { BaseSignalHit, EqlSignalSearchResponse } from '../signals/types';
import { createSecurityRuleTypeFactory } from './create_security_rule_type_factory';
import { createResultObject } from './utils';

interface EqlAlertState {
[key: string]: never;
}

export const createEqlAlertType = (createOptions: {
lists: SetupPlugins['lists'];
@@ -66,6 +71,8 @@ export const createEqlAlertType = (createOptions: {
services: { alertWithPersistence, scopedClusterClient },
params: { index, query },
}) {
const result = createResultObject<EqlAlertState>({});

const from = moment(startedAt).subtract(moment.duration(5, 'm')).toISOString(); // hardcoded 5-minute rule interval
const to = startedAt.toISOString();

@@ -128,9 +135,7 @@ export const createEqlAlertType = (createOptions: {
});
}

return {
lastChecked: new Date(),
};
return result;
},
});
};
Original file line number Diff line number Diff line change
@@ -19,11 +19,10 @@ import { SetupPlugins } from '../../../../target/types/server/plugin';
import { queryRuleParams, QueryRuleParams } from '../schemas/rule_schemas';

import { createSecurityRuleTypeFactory } from './create_security_rule_type_factory';
import { SecurityAlertTypeReturnValue } from './types';
import { createResultObject } from './utils';

type QueryStateFields = 'previousStartedAt';
type QueryAlertState = Record<QueryStateFields, unknown>;
interface QueryAlertState {
[key: string]: never;
}

export const createQueryAlertType = (createOptions: {
lists: SetupPlugins['lists'];
@@ -36,7 +35,7 @@ export const createQueryAlertType = (createOptions: {
logger,
ruleDataClient,
});
return createSecurityRuleType<QueryRuleParams, {}, PersistenceServices, QueryAlertState>({
return createSecurityRuleType<QueryRuleParams, {}, PersistenceServices, {}>({
id: CUSTOM_ALERT_TYPE_ID,
name: 'Custom Query Rule',
validate: {
@@ -67,9 +66,7 @@ export const createQueryAlertType = (createOptions: {
isExportable: false,
producer: 'security-solution',
async executor(execOptions) {
const result = createResultObject<QueryAlertState>({
previousStartedAt: new Date().toISOString(),
});
const result = createResultObject<QueryAlertState>({});
const {
params: { index, query },
services: { alertWithPersistence, savedObjectsClient },
Original file line number Diff line number Diff line change
@@ -12,7 +12,7 @@ import { Logger } from '@kbn/logging';
import { validateNonExact } from '@kbn/securitysolution-io-ts-utils';

import { AlertServices } from '../../../../../alerting/server';
import { RuleDataClient } from '../../../../../rule_registry/server';
import { PersistenceServices, RuleDataClient } from '../../../../../rule_registry/server';
import { THRESHOLD_ALERT_TYPE_ID } from '../../../../common/constants';
import { SetupPlugins } from '../../../../target/types/server/plugin';
import { thresholdRuleParams, ThresholdRuleParams } from '../schemas/rule_schemas';
@@ -26,20 +26,10 @@ import {
import { getFilter } from '../signals/get_filter';
import { BuildRuleMessage } from '../signals/rule_messages';
import { createSecurityRuleTypeFactory } from './create_security_rule_type_factory';
import { createResultObject } from './utils';

/*
interface RuleParams {
indexPatterns: string[];
customQuery: string;
thresholdFields: string[];
thresholdValue: number;
thresholdCardinality: Array<{
field: string;
value: number;
}>;
}
*/

type ThresholdStateFields = 'TODO';
type ThresholdAlertState = Record<ThresholdStateFields, unknown>;
interface BulkCreateThresholdSignalParams {
results: SignalSearchResponse;
ruleParams: ThresholdRuleParams;
@@ -90,7 +80,7 @@ export const createThresholdAlertType = (createOptions: {
logger,
ruleDataClient,
});
return createSecurityRuleType({
return createSecurityRuleType<ThresholdRuleParams, {}, PersistenceServices, ThresholdAlertState>({
id: THRESHOLD_ALERT_TYPE_ID,
name: 'Threshold Rule',
validate: {
@@ -107,21 +97,6 @@ export const createThresholdAlertType = (createOptions: {
},
},
},
/*
params: schema.object({
indexPatterns: schema.arrayOf(schema.string()),
customQuery: schema.string(),
thresholdFields: schema.arrayOf(schema.string()),
thresholdValue: schema.number(),
thresholdCardinality: schema.arrayOf(
schema.object({
field: schema.string(),
value: schema.number(),
})
),
}),
},
*/
actionGroups: [
{
id: 'default',
@@ -136,6 +111,7 @@ export const createThresholdAlertType = (createOptions: {
isExportable: false,
producer: 'security-solution',
async executor({ startedAt, services, params, alertId }) {
const result = createResultObject<ThresholdAlertState>({ TODO: 'test' });
const { index, query, threshold } = params;
const fromDate = moment(startedAt).subtract(moment.duration(5, 'm')); // hardcoded 5-minute rule interval
const from = fromDate.toISOString();
@@ -220,9 +196,7 @@ export const createThresholdAlertType = (createOptions: {
throw new Error(errors.join('\n'));
}

return {
lastChecked: new Date(),
};
return result;
},
});
};
Original file line number Diff line number Diff line change
@@ -8,12 +8,13 @@
import { AlertTypeState } from '../../../../../alerting/common';
import { AlertInstance } from '../../../../../alerting/server';

export interface SecurityRuleState<TState extends AlertTypeState> {
export type SecurityAlertTypeState = AlertTypeState & { previousStartedAt: string };
export interface SecurityRuleState<TState extends AlertTypeState = {}> {
alertTypeState: TState;
alertInstances: AlertInstance[];
previousStartedAt: Date;
}
export interface SecurityAlertTypeReturnValue<TState extends AlertTypeState> {
export interface SecurityAlertTypeReturnValue<TState extends AlertTypeState = {}> {
bulkCreateTimes: string[];
createdSignals: unknown[];
createdSignalsCount: number;
@@ -23,4 +24,4 @@ export interface SecurityAlertTypeReturnValue<TState extends AlertTypeState> {
state: TState;
success: boolean;
warningMessages: string[];
}
}
Original file line number Diff line number Diff line change
@@ -8,7 +8,7 @@
import { AlertTypeState } from '../../../../../../alerting/server';
import { SecurityAlertTypeReturnValue } from '../types';

export const createResultObject = <TState extends AlertTypeState>(state: TState) => {
export const createResultObject = <TState extends AlertTypeState = {}>(state: TState) => {
const result: SecurityAlertTypeReturnValue<TState> = {
bulkCreateTimes: [],
createdSignals: [],
Original file line number Diff line number Diff line change
@@ -666,7 +666,6 @@ export const createSearchAfterReturnTypeFromResponse = ({

export const createSearchAfterReturnType = ({
success,
warning,
searchAfterTimes,
bulkCreateTimes,
lastLookBackDate,
@@ -676,7 +675,6 @@ export const createSearchAfterReturnType = ({
warningMessages,
}: {
success?: boolean | undefined;
warning?: boolean;
searchAfterTimes?: string[] | undefined;
bulkCreateTimes?: string[] | undefined;
lastLookBackDate?: Date | undefined;
@@ -687,7 +685,6 @@ export const createSearchAfterReturnType = ({
} = {}): SearchAfterAndBulkCreateReturnType => {
return {
success: success ?? true,
warning: warning ?? false,
searchAfterTimes: searchAfterTimes ?? [],
bulkCreateTimes: bulkCreateTimes ?? [],
lastLookBackDate: lastLookBackDate ?? null,