[ML] Explain log rate spikes: Add field histograms to analysis result. #136295

walterra · 2022-07-13T14:20:35Z

Summary

Part of #136265.

Extends the analysis API to return histogram data.
Adds a column with MiniHistogram components to the analysis results table.
Moves/consolidates fetchHistogramsForFields to @kbn/ml-agg-utils (also used in Data Visualizer and Data Grid Mini Histograms).
So far fetchHistogramsForFields auto-identified the necessary interval and min/max. To be able to generate histogram data for the log rate spikes charts an options was added to use that information up front for the data to be fetched. This allows the buckets for the chart data for the overall (green bars) and the field/value-filtered (orange bars) histogram to have the exact same buckets.

Checklist

Unit or functional tests were updated or added to match the most common scenarios
This was checked for cross-browser compatibility
This was checked for breaking API changes and was labeled appropriately

elasticmachine · 2022-07-18T15:59:48Z

Pinging @elastic/ml-ui (:ml)

x-pack/packages/ml/agg_utils/src/fetch_histograms_for_fields.ts

x-pack/packages/ml/agg_utils/src/types.ts

x-pack/plugins/aiops/public/components/mini_histogram/mini_histogram.tsx

peteharverson · 2022-07-19T09:16:53Z

x-pack/plugins/aiops/server/routes/explain_log_rate_spikes.ts

+            updateLoadingStateAction({
+              ccsWarning: false,
+              loaded: 1,
+              loadingState: `Done.`,


Is this string of text displayed to the user in the UI?

Added an item to the meta issue to translate those messages, there are more from previous PRs too. #136265

peteharverson · 2022-07-19T09:19:59Z

x-pack/plugins/aiops/server/routes/explain_log_rate_spikes.ts

+                updateLoadingStateAction({
+                  ccsWarning: false,
+                  loaded,
+                  loadingState: `Loading histogram data.`,


Is this displayed in the UI?

Added an item to the meta issue to translate those messages, there are more from previous PRs too. #136265

peteharverson

Tested and LGTM. Just added a few minor comments on the code.

qn895 · 2022-07-19T15:18:12Z

x-pack/plugins/aiops/public/components/mini_histogram/mini_histogram.tsx

+          stackAccessors={[0]}
+        />
+        <BarSeries
+          id={`${label}`}


Since label is already a string, we can simplify it to id={label} here.

Fixed in 959edc2.

qn895 · 2022-07-19T15:20:19Z

x-pack/plugins/aiops/public/components/mini_histogram/mini_histogram.tsx

+      <Chart>
+        <Settings theme={theme} showLegend={false} />
+        <BarSeries
+          id="Other"


Super duper nit: this id should be lowercase to be consistent with others

Good catch, the "Other" in this BarSeries was wrong anyway, changed it to doc_count_overall. Fixed in 959edc2.

qn895 · 2022-07-19T15:24:32Z

x-pack/plugins/ml/server/models/data_visualizer/data_visualizer.ts

@@ -174,125 +115,11 @@ type BatchStats =
  | FieldExamples;

 // export for re-use by transforms plugin


I think we can remove this comment here now

Fixed in 959edc2.

qn895 · 2022-07-19T15:27:11Z

x-pack/plugins/aiops/public/components/spike_analysis_table/spike_analysis_table.tsx

@@ -119,7 +118,10 @@ export const SpikeAnalysisTable: FC<Props> = ({ changePointData, error, loading
    <EuiBasicTable
      compressed
      columns={columns}
-      items={pageOfItems ?? []}
+      // Temporary default sorting by ascending pValue until we add native table sorting
+      items={pageOfItems.sort((a, b) => {


We should move this temporary default sort into the useMemo hook for pageOfItems so it doesn't get re-sorted and re-rendered every time.

Fixed in 959edc2.

kibana-ci · 2022-07-19T18:03:53Z

💚 Build Succeeded

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`aiops`	284	285	+1
`dataVisualizer`	365	366	+1
`ml`	1624	1625	+1
`transform`	319	320	+1
total			+4

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`@kbn/ml-agg-utils`	18	35	+17
`ml`	78	71	-7
total			+10

Any counts in public APIs

Total count of every any typed public API. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats any for more detailed information.

id	before	after	diff
`@kbn/ml-agg-utils`	1	2	+1
`ml`	10	9	-1
total			-0

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`aiops`	424.5KB	425.2KB	+733.0B
`dataVisualizer`	545.2KB	547.4KB	+2.2KB
`ml`	3.3MB	3.3MB	+2.1KB
`transform`	383.2KB	385.4KB	+2.2KB
total			+7.2KB

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id	before	after	diff
`@kbn/ml-agg-utils`	0	4	+4
`ml`	31	30	-1
total			+3

Unknown metric groups

API count

id	before	after	diff
`@kbn/ml-agg-utils`	21	53	+32
`ml`	251	244	-7
total			+25

History

💚 Build #58626 succeeded 2e081b5
💔 Build #58320 failed 1488117
💔 Build #57474 failed 273cda741f8a7d15b05dee45a6b41abbe08a091e
💚 Build #57403 succeeded d6308097575ae23d1c41c8bcc4ffd9224b4c7cd9

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @walterra

walterra · 2022-07-20T10:41:15Z

If you have suggestions for follow ups please feel free to add them to the meta issue: #136265

walterra added :ml release_note:skip Skip the PR/issue when compiling release notes v8.4.0 labels Jul 13, 2022

walterra self-assigned this Jul 13, 2022

walterra added 9 commits July 18, 2022 10:53

[ML] Move getHistogramsForFields to package.

659c09e

[ML] Cleanup types.

3c257a1

[ML] Fetch histogram data for change points.

62c34af

[ML] Fix imports and types.

51183ec

[ML] Add histogram data to change point items.

d6aef05

[ML] Consolidate HistogramField.

c22b8d1

[ML] Consider predefined interval/min/max.

da388bb

[ML] Update returned field histograms data format.

f79f9fc

[ML] Mini histogram charts.

1488117

walterra force-pushed the ml-aiops-field-histograms-api branch from 273cda7 to 1488117 Compare July 18, 2022 15:05

walterra mentioned this pull request Jul 18, 2022

[ML] Explain log rate spikes: MVP 8.4 improvements. #136265

Closed

34 tasks

walterra marked this pull request as ready for review July 18, 2022 15:59

walterra requested a review from a team as a code owner July 18, 2022 15:59

walterra requested review from peteharverson, qn895 and alvarezmelissa87 July 18, 2022 15:59

Merge branch 'main' into ml-aiops-field-histograms-api

7ae4585

peteharverson reviewed Jul 19, 2022

View reviewed changes

peteharverson approved these changes Jul 19, 2022

View reviewed changes

walterra added 3 commits July 19, 2022 12:51

[ML] Add comments to exported types.

d3796b5

[ML] Remove commented code.

38ef121

[ML] Update functional tests.

2e081b5

qn895 reviewed Jul 19, 2022

View reviewed changes

walterra added 2 commits July 19, 2022 17:31

[ML] Simplify id prop.

e29aadc

[ML] Review feedback tweaks.

959edc2

walterra merged commit 092fb35 into elastic:main Jul 20, 2022

kibanamachine added the backport:skip This commit does not require backporting label Jul 20, 2022

walterra deleted the ml-aiops-field-histograms-api branch July 20, 2022 09:44

walterra mentioned this pull request Jul 29, 2022

[ML] Adds explain log rate spikes feature to the ML plugin #135948

Merged

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ML] Explain log rate spikes: Add field histograms to analysis result. #136295

[ML] Explain log rate spikes: Add field histograms to analysis result. #136295

walterra commented Jul 13, 2022 •

edited

Loading

elasticmachine commented Jul 18, 2022

peteharverson Jul 19, 2022

walterra Jul 19, 2022

peteharverson Jul 19, 2022

walterra Jul 19, 2022

peteharverson left a comment

qn895 Jul 19, 2022

walterra Jul 19, 2022

qn895 Jul 19, 2022

walterra Jul 19, 2022

qn895 Jul 19, 2022

walterra Jul 19, 2022

qn895 Jul 19, 2022

walterra Jul 19, 2022 •

edited

Loading

kibana-ci commented Jul 19, 2022

API count

walterra commented Jul 20, 2022

		@@ -174,125 +115,11 @@ type BatchStats =
		\| FieldExamples;

		// export for re-use by transforms plugin

[ML] Explain log rate spikes: Add field histograms to analysis result. #136295

[ML] Explain log rate spikes: Add field histograms to analysis result. #136295

Conversation

walterra commented Jul 13, 2022 • edited Loading

Summary

Checklist

elasticmachine commented Jul 18, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

peteharverson left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

walterra Jul 19, 2022 • edited Loading

Choose a reason for hiding this comment

kibana-ci commented Jul 19, 2022

💚 Build Succeeded

Metrics [docs]

Module Count

Public APIs missing comments

Any counts in public APIs

Async chunks

Public APIs missing exports

API count

History

walterra commented Jul 20, 2022

walterra commented Jul 13, 2022 •

edited

Loading

walterra Jul 19, 2022 •

edited

Loading