Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Detection Engine] Adds Alert Suppression to ML Rules #181926

Merged
merged 111 commits into from
Jul 2, 2024
Merged
Changes from 1 commit
Commits
Show all changes
111 commits
Select commit Hold shift + click to select a range
ad461bb
Add outline of integration test scenarios
rylnd Apr 26, 2024
8c0f6c1
Fleshing out more of our suppression execution tests
rylnd Apr 26, 2024
01bcf8e
Declare alert suppression fields as optional for ML rules
rylnd Apr 26, 2024
c42b339
Generated new types from new schema
rylnd Apr 26, 2024
b78c531
First legitimately failing test
rylnd Apr 26, 2024
cad4183
Extract executor params to interface
rylnd May 1, 2024
5377c6d
Merge branch 'main' into ml_rule_alert_suppression
rylnd May 3, 2024
6dbb88f
Adding more ML suppression functionality as typescript and tests dictate
rylnd May 3, 2024
e29c3d7
Declare ML rule to be suppressible
rylnd May 3, 2024
12ad5f5
Add ML rule to general suppression schema tests
rylnd May 3, 2024
c8b7c6a
Add placeholder for ML executor functionality
rylnd May 3, 2024
7f317cf
Declare our new executor parameters needed for rule suppression
rylnd May 3, 2024
13408a3
Call bulkCreateSuppressedAlertsInMemory to ML rules
rylnd May 3, 2024
703084f
Enable feature flag in FTR tests
rylnd May 3, 2024
b9de69e
Handle ML suppression params in rule converters
rylnd May 3, 2024
7e63b4d
First passing integration test
rylnd May 3, 2024
0caa342
Implementing most of our FTR test functionality
rylnd May 7, 2024
99aaffe
Flesh out remaining API integration tests
rylnd May 8, 2024
ee86fb1
Update test description in response to feedback
rylnd May 8, 2024
b5e809d
Add non-destructive form filling task for ML rules
rylnd May 8, 2024
10a9a42
Remove unused helper
rylnd May 8, 2024
f010c1c
Add cypress tests around creating/editing ML rules with suppression
rylnd May 8, 2024
5358bc4
Add TODO for later
rylnd May 8, 2024
4804eef
Add missing frontend logic to enable ML alert suppression
rylnd May 8, 2024
03833d1
Fix ML executor unit tests
rylnd May 8, 2024
184375d
Fix type error
rylnd May 9, 2024
982649f
Add alert suppression fields to alerting integration snapshot
rylnd May 9, 2024
241deaf
Allow cypress task to fill combobox with varying values
rylnd May 9, 2024
651806f
Get most of the rule editing UI tests green
rylnd May 9, 2024
c738d51
Merge branch 'main' into ml_rule_alert_suppression
rylnd May 9, 2024
c9a371e
Fix type errors in prebuilt rule cypress test
rylnd May 10, 2024
a39c6ba
Merge branch 'main' into ml_rule_alert_suppression
rylnd May 10, 2024
fe2635f
Merge branch 'main' into ml_rule_alert_suppression
rylnd May 16, 2024
fc40364
Fix nondeterministic ordering of alerts
rylnd May 16, 2024
afa7db6
Merge branch 'main' into ml_rule_alert_suppression
rylnd May 30, 2024
4a172ee
Fix missed merge conflicts
rylnd May 31, 2024
1a92d58
Remove unused dependency from test helper
rylnd May 31, 2024
60149d1
Merge branch 'main' into ml_rule_alert_suppression
rylnd Jun 4, 2024
eacb0b0
Remove unnecessary linting exception
rylnd Jun 4, 2024
845a217
Fix type error in helper fn
rylnd Jun 4, 2024
3c68939
Use empty array as input index for wrapSuppressedHits
rylnd Jun 6, 2024
3914d74
Test that some timing information is used in the ML executor
rylnd Jun 6, 2024
3f5d9c2
Remove TODO that we won't touch
rylnd Jun 6, 2024
3d8a11e
Remove placeholder comment
rylnd Jun 6, 2024
7901b5c
Remove unused variable
rylnd Jun 6, 2024
3b25273
Merge branch 'main' into ml_rule_alert_suppression
rylnd Jun 7, 2024
f8145a2
Update test following upstream behavioral change
rylnd Jun 7, 2024
7aa764a
Merge branch 'main' into ml_rule_alert_suppression
rylnd Jun 10, 2024
c54e3e9
Merge branch 'main' into ml_rule_alert_suppression
rylnd Jun 10, 2024
9c79daf
Merge branch 'main' into ml_rule_alert_suppression
rylnd Jun 10, 2024
922d8b0
Merge branch 'main' into ml_rule_alert_suppression
rylnd Jun 11, 2024
56de5f6
Add copy for new ML rule warnings
rylnd Jun 6, 2024
661a195
Add UX messaging for ML suppression edge cases
rylnd Jun 6, 2024
dfeb857
style: sort StepDefineRule arguments
rylnd Jun 6, 2024
5273d1f
Provide ML fields on Define step of rule creation
rylnd Jun 6, 2024
4955831
WIP: adding cypress tests for ML warning
rylnd Jun 11, 2024
eefa1c0
Install test ML jobs with a known/compatible group
rylnd Jun 12, 2024
cf5ceae
Use correct job ID in cypress tests
rylnd Jun 12, 2024
87c079d
Replace duplicated, magic-stringed function for existing one
rylnd Jun 12, 2024
cf71e6d
ML Suppression Cypress tests are green
rylnd Jun 12, 2024
65e16e8
More direct copy for user action
rylnd Jun 17, 2024
e6d85e2
Update ML warning copy per docs' suggestions
rylnd Jun 18, 2024
9c2d143
Replace display names with actual Job IDs
rylnd Jun 17, 2024
1aa170a
Extract ML rule validation logic into hook
rylnd Jun 17, 2024
fedb755
Relax requirement on number of rules
rylnd Jun 18, 2024
61d24d4
Clean up existing ML cypress test
rylnd Jun 18, 2024
99deac3
Bring back job display names for assertions
rylnd Jun 18, 2024
b0970bf
Re-enable ML rule edit tests
rylnd Jun 18, 2024
1a99fd1
Robustify ML cypress tasks
rylnd Jun 18, 2024
742503b
Ensure test has clean setup
rylnd Jun 18, 2024
f5cbaa5
Remove exclusivity from FTR tests, add TODO for re-skipping
rylnd Jun 18, 2024
8e9d4c5
Fix suppression fields for non-ML cases
rylnd Jun 18, 2024
e6aae21
Merge pull request #9 from rylnd/ml_rule_suppression_warnings
rylnd Jun 18, 2024
0cf3a51
Merge branch 'main' into ml_rule_alert_suppression
rylnd Jun 18, 2024
792373e
[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…
kibanamachine Jun 18, 2024
a3117a1
Revert change to form field types
rylnd Jun 18, 2024
9e77939
Update cypress tests to reflect latest copy
rylnd Jun 18, 2024
a42b966
Better assertion
rylnd Jun 18, 2024
df8e8a2
Fix mappings/aliases for v3 ML job archive
rylnd Jun 18, 2024
d71d5b0
Fix test assertion by loosening constraints
rylnd Jun 18, 2024
5ab6506
Stop ALL potential jobs at the start of our ML tests
rylnd Jun 18, 2024
5f1e4fa
Fix type error in test helper
rylnd Jun 19, 2024
87c3331
Reset combobox state after every option "fill"
rylnd Jun 20, 2024
a789766
Merge branch 'main' into ml_rule_alert_suppression
rylnd Jun 21, 2024
8f18a8e
Stop both datafeeds and jobs between ML tests
rylnd Jun 21, 2024
f451762
Don't fail if no jobs found during setup
rylnd Jun 21, 2024
32a55fb
Disable ML suppression fields until they're loaded and available
rylnd Jun 21, 2024
072e2f7
Add ML feature flag for serverless cypress
rylnd Jun 21, 2024
e78404c
Add read access to ML indices for security roles
rylnd Jun 24, 2024
240e998
Skip FF-dependent cypress tests in MKI builds
rylnd Jun 24, 2024
61ea144
Merge branch 'main' into ml_rule_alert_suppression
rylnd Jun 26, 2024
550bc8e
Fix type error in integration test
rylnd Jun 26, 2024
798fcb0
Add missing ML privileges from other role definitions
rylnd Jun 27, 2024
8462043
Re-skip flaky ML API tests
rylnd Jun 27, 2024
33ad209
Merge branch 'main' into ml_rule_alert_suppression
rylnd Jun 27, 2024
3168eac
Extract hairy inline logic to named variables
rylnd Jun 27, 2024
307e964
Update copy to be more direct
rylnd Jun 27, 2024
6ce66d1
More precise instruction
rylnd Jun 28, 2024
98ac972
Inline function-specific parameter
rylnd Jun 28, 2024
d152289
Add ML rules to Basic/Essentials workflow tests
rylnd Jun 28, 2024
ce395da
Add additional non-suppressible alert to better demonstrate suppression
rylnd Jun 28, 2024
f56840d
Merge branch 'main' into ml_rule_alert_suppression
rylnd Jun 28, 2024
d666eb0
Remove unused helper
rylnd Jun 28, 2024
112dafe
Add comment indicating why tests are skipped in MKI
rylnd Jul 1, 2024
006c88c
DRY up helper predicates
rylnd Jul 1, 2024
dd897d6
Send suppressed ML alerts to timeline as normal alerts
rylnd Jul 1, 2024
1924596
Merge branch 'main' into ml_rule_alert_suppression
rylnd Jul 1, 2024
96141dd
Test prebuilt rule workflows with ML Alert Suppression
rylnd Jul 1, 2024
3aa105d
Abstract ML-related logic into helper hook
rylnd Jul 1, 2024
a9b410d
Remove unused parameter
rylnd Jul 1, 2024
8e07763
Skip FTR suppression tests in MKI
rylnd Jul 2, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add UX messaging for ML suppression edge cases
* Disables suppression fields if no relevant ML jobs are running (as we
  cannot retrieve field info)
* Adds a warning message if not all relevant ML jobs are running (as we
  may be missing some field info)

Next step is testing this; we don't currently have a way to run ML rules
in cypress, but I'm going to attempt to copy the logic in our FTR to
accomplish this.
rylnd committed Jun 11, 2024
commit 661a195e362f08db89bdfe7697b164d48e236178
Original file line number Diff line number Diff line change
@@ -35,7 +35,7 @@ import type { DataViewBase } from '@kbn/es-query';
import { FormattedMessage } from '@kbn/i18n-react';
import { useSetFieldValueWithCallback } from '../../../../common/utils/use_set_field_value_cb';
import { useRuleFromTimeline } from '../../../../detections/containers/detection_engine/rules/use_rule_from_timeline';
import { isMlRule } from '../../../../../common/machine_learning/helpers';
import { isJobStarted, isMlRule } from '../../../../../common/machine_learning/helpers';
import { hasMlAdminPermissions } from '../../../../../common/machine_learning/has_ml_admin_permissions';
import { hasMlLicense } from '../../../../../common/machine_learning/has_ml_license';
import { useMlCapabilities } from '../../../../common/components/ml/hooks/use_ml_capabilities';
@@ -93,6 +93,7 @@ import { NewTermsFields } from '../new_terms_fields';
import { ScheduleItem } from '../../../rule_creation/components/schedule_item_form';
import { RequiredFields } from '../../../rule_creation/components/required_fields';
import { DocLink } from '../../../../common/components/links_to_docs/doc_link';
import { useInstalledSecurityJobs } from '../../../../common/components/ml/hooks/use_installed_security_jobs';
import { defaultCustomQuery } from '../../../../detections/pages/detection_engine/rules/utils';
import { MultiSelectFieldsAutocomplete } from '../multi_select_fields';
import { useLicense } from '../../../../common/hooks/use_license';
@@ -196,12 +197,24 @@ const StepDefineRuleComponent: FC<StepDefineRuleProps> = ({
const queryClient = useQueryClient();

const { isSuppressionEnabled: isAlertSuppressionEnabled } = useAlertSuppression(ruleType);
const mlCapabilities = useMlCapabilities();
const [openTimelineSearch, setOpenTimelineSearch] = useState(false);
const [indexModified, setIndexModified] = useState(false);
const [threatIndexModified, setThreatIndexModified] = useState(false);
const license = useLicense();

const mlCapabilities = useMlCapabilities();
const installedMlJobs = useInstalledSecurityJobs();
const [{ machineLearningJobId }] = useFormData<DefineStepRule>({
form,
watch: ['machineLearningJobId'],
});
const ruleMlJobs = installedMlJobs.jobs.filter((job) => machineLearningJobId.includes(job.id));
const numberOfRuleMlJobsStarted = ruleMlJobs.filter((job) =>
isJobStarted(job.jobState, job.datafeedState)
).length;
const noMlJobsStarted = numberOfRuleMlJobsStarted === 0;
const someMlJobsStarted = !noMlJobsStarted && numberOfRuleMlJobsStarted !== ruleMlJobs.length;

const esqlQueryRef = useRef<DefineStepRule['queryBar'] | undefined>(undefined);

const isAlertSuppressionLicenseValid = license.isAtLeast(MINIMUM_LICENSE_FOR_SUPPRESSION);
@@ -1068,22 +1081,32 @@ const StepDefineRuleComponent: FC<StepDefineRuleProps> = ({
</EuiText>
}
>
<UseField
path="groupByFields"
component={MultiSelectFieldsAutocomplete}
componentProps={{
browserFields: isEsqlRule(ruleType)
? esqlSuppressionFields
: termsAggregationFields,
isDisabled:
!isAlertSuppressionLicenseValid ||
areSuppressionFieldsDisabledBySequence ||
isEsqlSuppressionLoading,
disabledText: areSuppressionFieldsDisabledBySequence
? i18n.EQL_SEQUENCE_SUPPRESSION_DISABLE_TOOLTIP
: alertSuppressionUpsellingMessage,
}}
/>
<>
<UseField
path="groupByFields"
component={MultiSelectFieldsAutocomplete}
componentProps={{
browserFields: isEsqlRule(ruleType)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: personally I found ternary operators with more than 1 level hard to read, maybe moving this logic can simplify a little bit

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I applied 3168eac, but for the reasons stated there it's not going to be easy to move this logic into e.g. a helper function (that isn't defined within this component context, at least). LMK what you think.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the only solution to nested ternary operators in that case I see as either extracting it into helper(which does not look to helpful) or put it is useMemo(or Immediately Invoked Function Expression)

so,

  const suppressionGroupByDisabledText = areSuppressionFieldsDisabledBySequence
    ? i18n.EQL_SEQUENCE_SUPPRESSION_DISABLE_TOOLTIP
    : isMlRule(ruleType) && noMlJobsStarted
    ? i18n.MACHINE_LEARNING_SUPPRESSION_DISABLED_LABEL
    : alertSuppressionUpsellingMessage;

would become

const suppressionGroupByDisabledText = useMemo(() => {
if (areSuppressionFieldsDisabledBySequence) {
return i18n.EQL_SEQUENCE_SUPPRESSION_DISABLE_TOOLTIP
} 

if (isMlRule(ruleType) && noMlJobsStarted) {
return  i18n.MACHINE_LEARNING_SUPPRESSION_DISABLED_LABEL
}

return alertSuppressionUpsellingMessage
}, [
// put dependencies here
])

I would prefer a second approach. It is more readable to me + you would get a benefit of memoization(although not critical at all here)

? esqlSuppressionFields
: termsAggregationFields,
isDisabled:
!isAlertSuppressionLicenseValid ||
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: here is lot logic in this condition, can we move it to the variable?

areSuppressionFieldsDisabledBySequence ||
isEsqlSuppressionLoading ||
noMlJobsStarted,
disabledText: areSuppressionFieldsDisabledBySequence
? i18n.EQL_SEQUENCE_SUPPRESSION_DISABLE_TOOLTIP
: noMlJobsStarted
? i18n.MACHINE_LEARNING_SUPPRESSION_DISABLED_LABEL
: alertSuppressionUpsellingMessage,
}}
/>
{someMlJobsStarted && (
<EuiText size="xs" color="warning">
{i18n.MACHINE_LEARNING_SUPPRESSION_INCOMPLETE_LABEL}
</EuiText>
)}
</>
</RuleTypeEuiFormRow>

<IntendedRuleTypeEuiFormRow
Original file line number Diff line number Diff line change
@@ -237,15 +237,15 @@ export const EQL_SEQUENCE_SUPPRESSION_GROUPBY_VALIDATION_TEXT = i18n.translate(
export const MACHINE_LEARNING_SUPPRESSION_DISABLED_LABEL = i18n.translate(
'xpack.securitySolution.detectionEngine.createRule.stepDefineRule.machineLearningSuppressionDisabledLabel',
{
defaultMessage: 'Alert suppression will be disabled until the Machine Learning jobs are run.',
defaultMessage: 'Machine Learning jobs must be running to enable alert suppression.',
}
);

export const MACHINE_LEARNING_SUPPRESSION_INCOMPLETE_LABEL = i18n.translate(
'xpack.securitySolution.detectionEngine.createRule.stepDefineRule.machineLearningSuppressionIncompleteLabel',
{
defaultMessage:
'Alert suppression may have incomplete field options until all Machine Learning jobs are run.',
'This list of fields may be incomplete as some Machine Learning jobs are not running.',
}
);