[ML] Adding anomaly score explanations #142999

jgowdyelastic · 2022-10-10T15:36:10Z

Adds the new anomaly explanation data to the expanded rows in the anomaly explorer and the single metric viewer.
If no anomaly_score_explanation record data is present, the section is no shown.

The elasticsearch changes are not in yet, but this can be tested by mocking the data by making these changes 1ab9b45
As soon as the backend changes are in, I will test throughly and raise a follow up PR to fix any issues.

Also refactors the AnomalyDetails component into typescript.

@elastic/kibana-design removes some now unused rules from a sass file.

cc @valeriy42

…-ref HEAD~1..HEAD --fix'

jgowdyelastic · 2022-10-11T16:21:07Z

x-pack/plugins/ml/public/application/components/anomalies_table/anomaly_details_utils.tsx

+          content={i18n.translate(
+            'xpack.ml.anomaliesTable.anomalyDetails.anomalyExplanationDetails.recordScoreTooltip',
+            {
+              defaultMessage: 'Record score normalized blah blah blah',


Description text needed

@szabosteve and @valeriy42 can you please help me with this tooltip text.

We have a lot of questions / SDHs where the final score is significantly different than the initial score. As well as the tooltip text here, I wonder if we need extra text (not behind a tooltip), possibly in the 'Impact on initial score' section clarifying why this is, giving reasons for 'why the actual score is so much lower than the initial score?'.

It looks like this is just a repeat of the "Initial record score" and the "Record score" from the left column, just with an arrow in between? If that's true, having "normalized" in the label here seems to falsely imply the other values aren't normalized. Unless there's something else going on here that you want to cover in the tooltip?

I was thinking that the label could be "Record score reduction" or something like that. With the tooltip explaining that normalization has caused the score to be reduced.

I wonder if we need extra text (not behind a tooltip), possibly in the 'Impact on initial score' section clarifying why this is, giving reasons for 'why the actual score is so much lower than the initial score?'.

Yes I think that would be helpful.

jgowdyelastic · 2022-10-11T16:21:24Z

x-pack/plugins/ml/public/application/components/anomalies_table/anomaly_details_utils.tsx

+          content={i18n.translate(
+            'xpack.ml.anomaliesTable.anomalyDetails.anomalyExplanationDetails.anomalyCharacteristicsTooltip',
+            {
+              defaultMessage: 'Anomaly characteristics impact blah blah blah',


Description text needed

@szabosteve and @valeriy42 can you please help me with this tooltip text.

The description we use in the API is: "Impact of the statistical properties of the detected anomalous interval", though hopefully we can make that more intelligible.

Updated in 45313cd

"The high impact from the statistical properties of the detected anomaly compared to the previous anomalies." or "impact from the shape of the detected anomaly distribution compared to the previous anomalies."

jgowdyelastic · 2022-10-11T16:21:35Z

x-pack/plugins/ml/public/application/components/anomalies_table/anomaly_details_utils.tsx

+          content={i18n.translate(
+            'xpack.ml.anomaliesTable.anomalyDetails.anomalyExplanationDetails.singleBucketTooltip',
+            {
+              defaultMessage: 'Single bucket impact blah blah blah',


Description text needed

@szabosteve and @valeriy42 can you please help me with this tooltip text.

The text we used in the API was: "Impact of the deviation between actual and typical in the current bucket." However, a clearer tip here might be: "The impact of the difference between actual and typical values in this bucket on the score."

Updated in 45313cd

jgowdyelastic · 2022-10-11T16:21:44Z

x-pack/plugins/ml/public/application/components/anomalies_table/anomaly_details_utils.tsx

+          content={i18n.translate(
+            'xpack.ml.anomaliesTable.anomalyDetails.anomalyExplanationDetails.multiBucketTooltip',
+            {
+              defaultMessage: 'Multi bucket impact blah blah blah',


Description text needed

@szabosteve and @valeriy42 can you please help me with this tooltip text.

The description in the API docs is " Impact of the deviation between actual and typical in the past 12 buckets." So to match my other suggestion, I'd suggest "The impact of the difference between actual and typical values in the past 12 buckets on the score."

Updated in 45313cd

I would skip putting "on the score." as the fields belong to the group "Impact on the initial score".

jgowdyelastic · 2022-10-11T16:21:50Z

x-pack/plugins/ml/public/application/components/anomalies_table/anomaly_details_utils.tsx

+          content={i18n.translate(
+            'xpack.ml.anomaliesTable.anomalyDetails.anomalyExplanationDetails.highVarianceTooltip',
+            {
+              defaultMessage: 'High variance interval blah blah blah',


Description text needed

@szabosteve and @valeriy42 can you please help me with this tooltip text.

The API docs for this one say "Indicates reduction of anomaly score for the bucket with large confidence intervals." However, I think that's another one we should try to make more intelligible if possible. For example, "If a bucket has large confidence intervals, the score is reduced." Though that begs the question of why this bucket has large confidence intervals.

Updated in 45313cd

The long and simplified answer is: We have detected a periodic pattern in the time series. The current bucket is in the part of the periodic interval where data can vary a lot. This leads to a large confidence interval since we have seen data all over the place in this part of the period before. Hence, we reduce the anomaly score for the current anomaly in this region.

Now, how to fit it in a tooltip?

jgowdyelastic · 2022-10-11T16:21:56Z

x-pack/plugins/ml/public/application/components/anomalies_table/anomaly_details_utils.tsx

+          content={i18n.translate(
+            'xpack.ml.anomaliesTable.anomalyDetails.anomalyExplanationDetails.incompleteBucketTooltip',
+            {
+              defaultMessage: 'Incomplete bucket blah blah blah',


Description text needed

@szabosteve and @valeriy42 can you please help me with this tooltip text.

The API docs for this one say "Indicates reduction of anomaly score if the bucket contains fewer samples than historically expected." It might be clearer to say something like "If the bucket contains fewer samples than expected, the score is reduced."

Updated in 45313cd

elasticmachine · 2022-10-11T16:42:02Z

Pinging @elastic/ml-ui (:ml)

…-ref HEAD~1..HEAD --fix'

peteharverson · 2022-10-19T13:45:57Z

I reckon we should not introduce another severity indicator. Did you consider using the one from the table row?
Good point. I'd vote for using the existing health indicator circle icon, unless there's a reason why you went with the underline style here?

Having seen the examples which use the health indicator in #142999 (comment), I agree the underline style works better in this case where we are showing how the score has dropped. Happy to go with the underline style in the explanation section.

peteharverson · 2022-10-19T14:18:20Z

x-pack/plugins/ml/public/application/components/anomalies_table/anomaly_details_utils.tsx

+      'xpack.ml.anomaliesTable.anomalyDetails.anomalyExplanationDetails.anomalyCharacteristicsTooltip.low',
+      {
+        defaultMessage:
+          'The statistical properties of the detected anomalous interval have a low impact on the score.',


@valeriy42 are there any more details we can provide for the 'statistical properties' info? This text is a bit unclear to me and leaves me wondering 'what statistical properties'.

@peteharverson: "The duration and magnitude of the current anomaly relative to the historical average among the detected anomalies."

magnitude, scale, extent, expanse, range of deviation

How about this?

Moderate / medium / high impact from the duration and magnitude of the detected anomaly relative to the historical average.

peteharverson · 2022-10-24T10:00:03Z

x-pack/plugins/ml/public/application/components/anomalies_table/anomaly_details_utils.tsx

+      'xpack.ml.anomaliesTable.anomalyDetails.anomalyExplanationDetails.singleBucketTooltip.low',
+      {
+        defaultMessage:
+          'The difference between actual and typical values in this bucket have a moderate impact.',


Should be has a moderate..

peteharverson · 2022-10-24T10:00:15Z

x-pack/plugins/ml/public/application/components/anomalies_table/anomaly_details_utils.tsx

+      'xpack.ml.anomaliesTable.anomalyDetails.anomalyExplanationDetails.singleBucketTooltip.medium',
+      {
+        defaultMessage:
+          'The difference between actual and typical values in this bucket have a significant impact.',


Should be has a significant..

peteharverson · 2022-10-24T10:00:23Z

x-pack/plugins/ml/public/application/components/anomalies_table/anomaly_details_utils.tsx

+      'xpack.ml.anomaliesTable.anomalyDetails.anomalyExplanationDetails.singleBucketTooltip.high',
+      {
+        defaultMessage:
+          'The difference between actual and typical values in this bucket have a high impact.',


Should be has a high..

peteharverson · 2022-10-24T10:00:48Z

x-pack/plugins/ml/public/application/components/anomalies_table/anomaly_details_utils.tsx

+      'xpack.ml.anomaliesTable.anomalyDetails.anomalyExplanationDetails.multiBucketTooltip.low',
+      {
+        defaultMessage:
+          'The difference between actual and typical values in the past 12 buckets have a moderate impact.',


Should be The differences or switch have to has

peteharverson · 2022-10-24T10:00:52Z

x-pack/plugins/ml/public/application/components/anomalies_table/anomaly_details_utils.tsx

+      'xpack.ml.anomaliesTable.anomalyDetails.anomalyExplanationDetails.multiBucketTooltip.medium',
+      {
+        defaultMessage:
+          'The difference between actual and typical values in the past 12 buckets have a significant impact.',


Should be The differences or switch have to has

peteharverson · 2022-10-24T10:00:55Z

x-pack/plugins/ml/public/application/components/anomalies_table/anomaly_details_utils.tsx

+      'xpack.ml.anomaliesTable.anomalyDetails.anomalyExplanationDetails.multiBucketTooltip.high',
+      {
+        defaultMessage:
+          'The difference between actual and typical values in the past 12 buckets have a high impact.',


Should be The differences or switch have to has

peteharverson

Latest changes LGTM

darnautov

LGTM

kibana-ci · 2022-10-24T11:04:05Z

💚 Build Succeeded

Buildkite Build
Commit: 48628be

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`ml`	3.4MB	3.4MB	+9.2KB

Unknown metric groups

API count

id	before	after	diff
`ml`	251	254	+3

History

💛 Build #81757 was flaky fcd6740
💚 Build #81691 succeeded 4fc0d32
💔 Build #81423 failed 2c0db82
💚 Build #81339 succeeded 79f3f6f
💚 Build #81298 succeeded 07fb3bb
💚 Build #81040 succeeded 7011f33

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @jgowdyelastic

…act (#146866) ## Summary Follow-up to #142999, using the `multi_bucket_impact` field from the `anomaly_score_explanation` data to populate the multi-bucket information in the anomaly chart tooltip using the same visualization style as used in the expanded table row. <img width="1174" alt="image" src="https://user-images.githubusercontent.com/7405507/205274876-b2007388-e37a-4f31-8c23-97fbac42f4d8.png"> The anomaly charts in the Anomaly Explorer and Single Metric Viewer, and the severity cell in the anomalies table now use a new `isMultiBucketAnomaly` check in `ml/common/util/anomaly_utils.ts`m to determine whether to indicate the anomaly as multi-bucket with the plus-shaped symbol. Note that this will result in some changes to whether anomalies are plotted as multi-bucket anomalies. Specifically anomalies with a 'moderate' multi-bucket impact are now likely to be marked with a plus symbol where before they were not: Before: <img width="775" alt="image" src="https://user-images.githubusercontent.com/7405507/205276861-613e0966-9041-4a77-a922-369bb0c6d3c3.png"> After: <img width="809" alt="image" src="https://user-images.githubusercontent.com/7405507/205277054-86895ab2-f5b1-42d4-a947-5d2cd5f3d82e.png"> ### Checklist Delete any items that are not applicable to this PR. - [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] This was checked for [cross-browser compatibility](https://www.elastic.co/support/matrix#matrix_browsers)

jgowdyelastic and others added 13 commits October 10, 2022 16:35

[ML] Adding anomaly score explanations

5400566

Merge branch 'main' into anomaly-score-explanation

f36547e

fixing types

3a4ca38

fixing jest test

10bb798

test title

0c769db

style improvements

a72bda9

test data

324d067

removing test data

93dd2c8

better test data

1ab9b45

removing test data

38f081d

Merge branch 'main' into anomaly-score-explanation

d47e1e2

[CI] Auto-commit changed files from 'node scripts/precommit_hook.js -…

3d2a4ef

…-ref HEAD~1..HEAD --fix'

using eui colors

3345cea

jgowdyelastic commented Oct 11, 2022

View reviewed changes

jgowdyelastic requested review from peteharverson, darnautov and alvarezmelissa87 October 11, 2022 16:32

jgowdyelastic added 2 commits October 11, 2022 17:32

Merge branch 'main' into anomaly-score-explanation

4ecd7f4

reverting auto changes

6c312ea

jgowdyelastic self-assigned this Oct 11, 2022

jgowdyelastic added release_note:enhancement :ml Feature:Anomaly Detection ML anomaly detection v8.6.0 labels Oct 11, 2022

jgowdyelastic marked this pull request as ready for review October 11, 2022 16:41

jgowdyelastic requested review from a team as code owners October 11, 2022 16:41

[CI] Auto-commit changed files from 'node scripts/precommit_hook.js -…

992c4a5

…-ref HEAD~1..HEAD --fix'

kibanamachine requested a review from a team as a code owner October 11, 2022 16:45

adding tooltip text

45313cd

valeriy42 requested review from valeriy42 and removed request for valeriy42 October 17, 2022 14:34

jgowdyelastic added 5 commits October 18, 2022 09:00

Merge branch 'main' into anomaly-score-explanation

d457144

adding comments

be8cfdf

text update

7011f33

Merge branch 'main' into anomaly-score-explanation

07fb3bb

adding conditional impact score text

79f3f6f

text and scoring changes

2c0db82

peteharverson reviewed Oct 19, 2022

View reviewed changes

jgowdyelastic added 2 commits October 20, 2022 09:20

translations

4fc0d32

Merge branch 'main' into anomaly-score-explanation

fcd6740

jgowdyelastic mentioned this pull request Oct 20, 2022

[ML] Anomaly detection multi-bucket impact discussion #143759

Closed

jgowdyelastic added 3 commits October 24, 2022 09:51

Merge branch 'main' into anomaly-score-explanation

2e5e5cd

updating text

078e43c

more text changes

48628be

peteharverson reviewed Oct 24, 2022

View reviewed changes

peteharverson approved these changes Oct 24, 2022

View reviewed changes

darnautov approved these changes Oct 24, 2022

View reviewed changes

jgowdyelastic merged commit 6cfdd9d into elastic:main Oct 24, 2022

kibanamachine added the backport:skip This commit does not require backporting label Oct 24, 2022

jgowdyelastic deleted the anomaly-score-explanation branch October 24, 2022 11:05

peteharverson added release_note:feature Makes this part of the condensed release notes and removed release_note:enhancement labels Nov 4, 2022

peteharverson mentioned this pull request Dec 2, 2022

[ML] Use anomaly score explanation for chart tooltip multi-bucket impact #146866

Merged

3 tasks

walterra mentioned this pull request Jul 4, 2024

[ML] Anomaly explanations missing for rare detectors. #187563

Open

[ML] Adding anomaly score explanations #142999

[ML] Adding anomaly score explanations #142999

Conversation

jgowdyelastic commented Oct 10, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

valeriy42 Oct 19, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

elasticmachine commented Oct 11, 2022

peteharverson commented Oct 19, 2022

Choose a reason for hiding this comment

valeriy42 Oct 19, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

peteharverson left a comment

Choose a reason for hiding this comment

darnautov left a comment

Choose a reason for hiding this comment

kibana-ci commented Oct 24, 2022

💚 Build Succeeded

Metrics [docs]

Async chunks

API count

History

jgowdyelastic commented Oct 10, 2022 •

edited

Loading

valeriy42 Oct 19, 2022 •

edited

Loading

valeriy42 Oct 19, 2022 •

edited

Loading