Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ML] Adding anomaly score explanations #142999

Merged
merged 35 commits into from
Oct 24, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
5400566
[ML] Adding anomaly score explanations
jgowdyelastic Oct 10, 2022
f36547e
Merge branch 'main' into anomaly-score-explanation
jgowdyelastic Oct 11, 2022
3a4ca38
fixing types
jgowdyelastic Oct 11, 2022
10bb798
fixing jest test
jgowdyelastic Oct 11, 2022
0c769db
test title
jgowdyelastic Oct 11, 2022
a72bda9
style improvements
jgowdyelastic Oct 11, 2022
324d067
test data
jgowdyelastic Oct 11, 2022
93dd2c8
removing test data
jgowdyelastic Oct 11, 2022
1ab9b45
better test data
jgowdyelastic Oct 11, 2022
38f081d
removing test data
jgowdyelastic Oct 11, 2022
d47e1e2
Merge branch 'main' into anomaly-score-explanation
jgowdyelastic Oct 11, 2022
3d2a4ef
[CI] Auto-commit changed files from 'node scripts/precommit_hook.js -…
kibanamachine Oct 11, 2022
3345cea
using eui colors
jgowdyelastic Oct 11, 2022
4ecd7f4
Merge branch 'main' into anomaly-score-explanation
jgowdyelastic Oct 11, 2022
6c312ea
reverting auto changes
jgowdyelastic Oct 11, 2022
992c4a5
[CI] Auto-commit changed files from 'node scripts/precommit_hook.js -…
kibanamachine Oct 11, 2022
3acbc7c
tiny change
jgowdyelastic Oct 11, 2022
96a224a
reverting test data
jgowdyelastic Oct 11, 2022
3c46afe
tiny change
jgowdyelastic Oct 11, 2022
6ce7bac
Merge branch 'main' into anomaly-score-explanation
jgowdyelastic Oct 14, 2022
872a12e
Merge branch 'main' into anomaly-score-explanation
jgowdyelastic Oct 14, 2022
8be0cd7
removing render functions
jgowdyelastic Oct 14, 2022
39cd2df
Merge branch 'main' into anomaly-score-explanation
jgowdyelastic Oct 17, 2022
45313cd
adding tooltip text
jgowdyelastic Oct 17, 2022
d457144
Merge branch 'main' into anomaly-score-explanation
jgowdyelastic Oct 18, 2022
be8cfdf
adding comments
jgowdyelastic Oct 18, 2022
7011f33
text update
jgowdyelastic Oct 18, 2022
07fb3bb
Merge branch 'main' into anomaly-score-explanation
jgowdyelastic Oct 19, 2022
79f3f6f
adding conditional impact score text
jgowdyelastic Oct 19, 2022
2c0db82
text and scoring changes
jgowdyelastic Oct 19, 2022
4fc0d32
translations
jgowdyelastic Oct 20, 2022
fcd6740
Merge branch 'main' into anomaly-score-explanation
jgowdyelastic Oct 20, 2022
2e5e5cd
Merge branch 'main' into anomaly-score-explanation
jgowdyelastic Oct 24, 2022
078e43c
updating text
jgowdyelastic Oct 24, 2022
48628be
more text changes
jgowdyelastic Oct 24, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 61 additions & 0 deletions x-pack/plugins/ml/common/types/anomalies.ts
Original file line number Diff line number Diff line change
Expand Up @@ -183,6 +183,52 @@ export interface AnomalyRecordDoc {
* purely single bucket and +5.0 means the anomaly is purely multi bucket.
*/
multi_bucket_impact?: number;

/**
* An explanation for the anomaly score
*/
anomaly_score_explanation?: {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this exported publicly? If so, could you add comments, to each of the props.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure yeah.
@valeriy42 Could you please help with the comments on each property?
I can guess, but I think you'd do a better job.

anomaly_type
anomaly_length
single_bucket_impact
multi_bucket_impact
anomaly_characteristics_impact
lower_confidence_bound
typical_value
upper_confidence_bound
high_variance_penalty
incomplete_bucket_penalty

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • anomaly_characteristics_impact Impact of the statistical properties of the detected anomalous interval.
  • anomaly_length Length of the detected anomaly in the number of buckets.
  • anomaly_type Type of the detected anomaly: spike or dip.
  • high_variance_penalty Indicates a reduction of anomaly score for the bucket with large confidence intervals.
  • incomplete_bucket_penalty Indicates a reduction of anomaly score if the bucket contains fewer samples than historically expected.
  • lower_confidence_bound Lower bound of the 95% confidence interval.
  • multi_bucket_impact Impact of the deviation between actual and typical in the past 12 buckets."
  • single_bucket_impact Impact of the deviation between actual and typical in the current bucket.
  • typical_value Typical (expected) value for this bucket.
  • upper_confidence_bound Upper bound of the 95% confidence interval.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @valeriy42
Updated in be8cfdf

/**
* Type of the detected anomaly: spike or dip.
*/
anomaly_type?: 'dip' | 'spike';
/**
* Length of the detected anomaly in the number of buckets.
*/
anomaly_length?: number;
/**
* Impact of the deviation between actual and typical in the current bucket.
*/
single_bucket_impact?: number;
/**
* Impact of the deviation between actual and typical in the past 12 buckets.
*/
multi_bucket_impact?: number;
/**
* Impact of the statistical properties of the detected anomalous interval.
*/
anomaly_characteristics_impact?: number;
/**
* Lower bound of the 95% confidence interval.
*/
lower_confidence_bound?: number;
/**
* Typical (expected) value for this bucket.
*/
typical_value?: number;
/**
* Upper bound of the 95% confidence interval.
*/
upper_confidence_bound?: number;
/**
* Indicates a reduction of anomaly score for the bucket with large confidence intervals.
*/
high_variance_penalty?: boolean;
/**
* Indicates a reduction of anomaly score if the bucket contains fewer samples than historically expected.
*/
incomplete_bucket_penalty?: boolean;
};
}

/**
Expand Down Expand Up @@ -283,6 +329,21 @@ export interface AnomaliesTableRecord {
* Returns true if the anomaly record represented by the table row can be shown in the maps plugin
*/
isGeoRecord?: boolean;

/**
* Returns true if the job has the model plot enabled
*/
modelPlotEnabled: boolean;
}

/**
* Customized version of AnomaliesTableRecord which inserts the detector description
* and rules length.
* Used by the AnomaliesTable component
*/
export interface AnomaliesTableRecordExtended extends AnomaliesTableRecord {
detector: string;
rulesLength?: number;
}

export type PartitionFieldsType = typeof PARTITION_FIELDS[number];
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -91,28 +91,6 @@
}

.ml-anomalies-table-details {
padding: $euiSizeXS $euiSizeXL;
max-height: 1000px;
overflow-y: auto;

.anomaly-description-list {

.euiDescriptionList__title {
margin-top: 0;
flex-basis: 15%;
font-size: inherit;
line-height: 1.5rem;
@include euiTextTruncate;
}

.euiDescriptionList__description {
margin-top: 0;
flex-basis: 85%;
font-size: inherit;
line-height: 1.5rem;
}

}
padding: $euiSizeM;
}

}
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,8 @@ export class AnomaliesTableInternal extends Component {
}
}

const job = this.props.selectedJobs.find(({ id }) => id === item.jobId);

itemIdToExpandedRowMap[item.rowId] = (
<AnomalyDetails
tabIndex={tab}
Expand All @@ -104,6 +106,7 @@ export class AnomaliesTableInternal extends Component {
filter={this.props.filter}
influencerFilter={this.props.influencerFilter}
influencersLimit={INFLUENCERS_LIMIT}
job={job}
/>
);
}
Expand Down Expand Up @@ -277,4 +280,5 @@ AnomaliesTableInternal.propTypes = {
tableState: PropTypes.object.isRequired,
updateTableState: PropTypes.func.isRequired,
sourceIndicesWithGeoFields: PropTypes.object.isRequired,
selectedJobs: PropTypes.array.isRequired,
};
Loading