-
Notifications
You must be signed in to change notification settings - Fork 8.3k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[Response Ops][Alerting] Adding group by options to ES query rule type (
#144689) Resolves #89481 ## Summary Adds group by options to the ES query rule type, both DSL and KQL options. This is the same limited group by options that are offered in the index threshold rule type so I used the same UI components and rule parameter names. I moved some aggregation building code to `common` so they could be reused. All existing ES query rules are migrated to be `count over all` rules. ## To Verify * Create the following types of rules and verify they work as expected. Verify for both DSL query and KQL query * `count over all` rule - this should run the same as before, where it counts the number of documents that matches the query and applies the threshold condition to that value. `{{context.hits}}` is all the documents that match the query if the threshold condition is met. * `<metric> over all` rule - this calculates the specific aggregation metric and applies the threshold condition to the aggregated metric (for example, `avg event.duration`). `{{context.hits}}` is all the documents that match the query if the threshold condition is met. * `count over top N terms` - this will apply a term aggregation to the query and matches the threshold condition to each term bucket (for example, `count over top 10 event.action` will apply the threshold condition to the count of documents within each `event.action` bucket). `{{context.hits}}` is the result of the top hits aggregation within each term bucket if the threshold condition is met for that bucket. * `<metric> over top N terms` - this will apply a term aggregation and a metric sub-aggregation to the query and matches the threshold condition to the metric value within each term bucket (for example, `avg event.duration over top 10 event.action` will apply the threshold condition to the average value of `event.duration` within each `event.action` bucket). `{{context.hits}}` is the result of the top hits aggregation within each term bucket if the threshold condition is met for that bucket. * Verify the migration by creating a DSL and KQL query in an older version of Kibana and then upgrading to this PR. The rules should still continue running successfully. ### Checklist - [x] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios Co-authored-by: Kibana Machine <[email protected]> Co-authored-by: Lisa Cawley <[email protected]>
- Loading branch information
3 people
authored
Dec 15, 2022
1 parent
e709523
commit fdf4dea
Showing
74 changed files
with
5,236 additions
and
691 deletions.
There are no files selected for viewing
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
39 changes: 39 additions & 0 deletions
39
x-pack/plugins/alerting/server/saved_objects/migrations/8.7/index.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,39 @@ | ||
| /* | ||
| * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one | ||
| * or more contributor license agreements. Licensed under the Elastic License | ||
| * 2.0; you may not use this file except in compliance with the Elastic License | ||
| * 2.0. | ||
| */ | ||
|
|
||
| import { SavedObjectUnsanitizedDoc } from '@kbn/core-saved-objects-server'; | ||
| import { EncryptedSavedObjectsPluginSetup } from '@kbn/encrypted-saved-objects-plugin/server'; | ||
| import { createEsoMigration, isEsQueryRuleType, pipeMigrations } from '../utils'; | ||
| import { RawRule } from '../../../types'; | ||
|
|
||
| function addGroupByToEsQueryRule( | ||
| doc: SavedObjectUnsanitizedDoc<RawRule> | ||
| ): SavedObjectUnsanitizedDoc<RawRule> { | ||
| // Adding another check in for isEsQueryRuleType in case we add more migrations | ||
| if (isEsQueryRuleType(doc)) { | ||
| return { | ||
| ...doc, | ||
| attributes: { | ||
| ...doc.attributes, | ||
| params: { | ||
| ...doc.attributes.params, | ||
| aggType: 'count', | ||
| groupBy: 'all', | ||
| }, | ||
| }, | ||
| }; | ||
| } | ||
|
|
||
| return doc; | ||
| } | ||
|
|
||
| export const getMigrations870 = (encryptedSavedObjects: EncryptedSavedObjectsPluginSetup) => | ||
| createEsoMigration( | ||
| encryptedSavedObjects, | ||
| (doc): doc is SavedObjectUnsanitizedDoc<RawRule> => isEsQueryRuleType(doc), | ||
| pipeMigrations(addGroupByToEsQueryRule) | ||
| ); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,81 @@ | ||
| /* | ||
| * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one | ||
| * or more contributor license agreements. Licensed under the Elastic License | ||
| * 2.0; you may not use this file except in compliance with the Elastic License | ||
| * 2.0. | ||
| */ | ||
|
|
||
| import { Comparator } from './comparator_types'; | ||
|
|
||
| export type ComparatorFn = (value: number, threshold: number[]) => boolean; | ||
|
|
||
| const humanReadableComparators = new Map<Comparator, string>([ | ||
| [Comparator.LT, 'less than'], | ||
| [Comparator.LT_OR_EQ, 'less than or equal to'], | ||
| [Comparator.GT_OR_EQ, 'greater than or equal to'], | ||
| [Comparator.GT, 'greater than'], | ||
| [Comparator.BETWEEN, 'between'], | ||
| [Comparator.NOT_BETWEEN, 'not between'], | ||
| ]); | ||
|
|
||
| export const ComparatorFns = new Map<Comparator, ComparatorFn>([ | ||
| [Comparator.LT, (value: number, threshold: number[]) => value < threshold[0]], | ||
| [Comparator.LT_OR_EQ, (value: number, threshold: number[]) => value <= threshold[0]], | ||
| [Comparator.GT_OR_EQ, (value: number, threshold: number[]) => value >= threshold[0]], | ||
| [Comparator.GT, (value: number, threshold: number[]) => value > threshold[0]], | ||
| [ | ||
| Comparator.BETWEEN, | ||
| (value: number, threshold: number[]) => value >= threshold[0] && value <= threshold[1], | ||
| ], | ||
| [ | ||
| Comparator.NOT_BETWEEN, | ||
| (value: number, threshold: number[]) => value < threshold[0] || value > threshold[1], | ||
| ], | ||
| ]); | ||
|
|
||
| export const getComparatorScript = ( | ||
| comparator: Comparator, | ||
| threshold: number[], | ||
| fieldName: string | ||
| ) => { | ||
| if (threshold.length === 0) { | ||
| throw new Error('Threshold value required'); | ||
| } | ||
|
|
||
| function getThresholdString(thresh: number) { | ||
| return Number.isInteger(thresh) ? `${thresh}L` : `${thresh}`; | ||
| } | ||
|
|
||
| switch (comparator) { | ||
| case Comparator.LT: | ||
| return `${fieldName} < ${getThresholdString(threshold[0])}`; | ||
| case Comparator.LT_OR_EQ: | ||
| return `${fieldName} <= ${getThresholdString(threshold[0])}`; | ||
| case Comparator.GT: | ||
| return `${fieldName} > ${getThresholdString(threshold[0])}`; | ||
| case Comparator.GT_OR_EQ: | ||
| return `${fieldName} >= ${getThresholdString(threshold[0])}`; | ||
| case Comparator.BETWEEN: | ||
| if (threshold.length < 2) { | ||
| throw new Error('Threshold values required'); | ||
| } | ||
| return `${fieldName} >= ${getThresholdString( | ||
| threshold[0] | ||
| )} && ${fieldName} <= ${getThresholdString(threshold[1])}`; | ||
| case Comparator.NOT_BETWEEN: | ||
| if (threshold.length < 2) { | ||
| throw new Error('Threshold values required'); | ||
| } | ||
| return `${fieldName} < ${getThresholdString( | ||
| threshold[0] | ||
| )} || ${fieldName} > ${getThresholdString(threshold[1])}`; | ||
| } | ||
| }; | ||
|
|
||
| export const ComparatorFnNames = new Set(ComparatorFns.keys()); | ||
|
|
||
| export function getHumanReadableComparator(comparator: Comparator) { | ||
| return humanReadableComparators.has(comparator) | ||
| ? humanReadableComparators.get(comparator) | ||
| : comparator; | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.