[ML] Transforms: Support sub-aggregations #68306

darnautov · 2020-06-04T19:29:35Z

Summary

Follow-up on #67591. Adds sub-aggregations support to the transforms wizard.

Please bear in mind data-grid preview does not work properly at the moment because it can't handle keys containing . in the property name. If you change the aggregation name to the one without dots, you should see counters updated in the data-grid table accordingly. This issue will be resolved in another PR.

Checklist

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Unit or functional tests were updated or added to match the most common scenarios
This was checked for keyboard-only and screenreader accessibility
This renders correctly on smaller devices using a responsive layout. (You can test this in your browser
This was checked for cross-browser compatibility, including a check against IE11

darnautov · 2020-06-08T10:29:06Z

...s/create_transform/components/step_define/common/filter_agg/components/filter_range_form.tsx

+              // @ts-ignore
+              step="any"


elastic/eui#3550 EUI related issue

elasticmachine · 2020-06-08T12:10:26Z

Pinging @elastic/ml-ui (:ml)

…-aggs

alvarezmelissa87

LGTM ⚡

walterra

Code LGTM! f triggered this propTypes related error:

Looks like we need to cast to a number for the EUI component for the number input.

darnautov · 2020-06-09T08:23:15Z

Code LGTM! f triggered this propTypes related error:

Looks like we need to cast to a number for the EUI component for the number input.

@walterra https://github.com/elastic/kibana/pull/68306/files#r436599588 I have left a comment about it, in case you missed 😉 it's a EUI issue, hope it's going to be resolved soon.

walterra

LGTM (error screenshot I posted relates to EUI issue)

peteharverson · 2020-06-09T09:58:51Z

With the default dot . separators for the nested sub-aggs, the column that gets added to the preview data grid is blank. If we can't fix the grid to work with the dot separators, then I think we need to replace the dot separator used for the default aggregation name to e.g. underscores. Otherwise the user is going to be very confused as to why the column they add to the table is blank.

...nsform/public/app/sections/create_transform/components/aggregation_list/sub_aggs_section.tsx

darnautov · 2020-06-09T11:01:02Z

With the default dot . separators for the nested sub-aggs, the column that gets added to the preview data grid is blank. If we can't fix the grid to work with the dot separators, then I think we need to replace the dot separator used for the default aggregation name to e.g. underscores. Otherwise the user is going to be very confused as to why the column they add to the table is blank.

This bug took place before this PR, so we decided to fix it in a follow-up. It's a part of the meta issue for the data grid enhancements in DFA and Transforms #66383

…-aggs

peteharverson

Tested and LGTM.

Noted that the handling of fields with dot separators in the data grid will be addressed in #66383.

x-pack/test/functional/apps/transform/creation_index_pattern.ts

pheyos

Functional tests LGTM

kibanamachine · 2020-06-09T14:11:30Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: 7f51f7b

History

💚 Build #52679 succeeded efe2265
💚 Build #52655 succeeded 3005944
💚 Build #52652 succeeded 5bf217c
💚 Build #52440 succeeded 157ff2e
💛 Build #52399 was flaky 83e7477

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

* [ML] WIP UI support for the sug-aggs * [ML] extract sug-aggs from es config * [ML] fix for range filter * [ML] delete sub-aggregation * [ML] isConfigInvalid * [ML] fix actions, support parent agg * [ML] update getAggConfigFromEsAgg * [ML] unit tests * [ML] SubAggsSection * [ML] fix snapshot, add validation * [ML] fix label capping * [ML] rename Filter query label * [ML] fix updateSubAggregation validation * [ML] fix state update warning * [ML] increase MAX_NESTING_SUB_AGGS * [ML] functional tests for sub-aggregations * [ML] change the tooltip message * [ML] update functional tests * [ML] fix typo

* master: (37 commits) [DOCS] Adds documentation for drilldowns (elastic#68122) [Telemetry] telemetry.sendUsageFrom: "server" by default (elastic#68071) [ML] Transforms: Support sub-aggregations (elastic#68306) Fixed pre-configured docs link points to the wrong page and functional tests configs (elastic#68606) [Ingest Manager] Update queries from `stream.*` to `dataset.*` (elastic#68322) Enable Watcher by default to fix bug in which Watcher doesn't render in the side nav (elastic#68602) Convert Index Templates API routes to snakecase. (elastic#68463) [SECURITY SOLUTION] [Detections] Add / Update e2e tests to ensure initial rule runs are successful (elastic#68441) [Ingest] OpenAPI spec file (elastic#68323) chore(NA): skip apis Endpoint plugin Endpoint policy api (elastic#68638) bumping makelogs version to v6.0.0 (elastic#66163) [SIEM] Add create template button (elastic#66613) Bump websocket-extensions from 0.1.3 to 0.1.4 (elastic#68414) [ML] Sample data modules - use event.dataset instead of index name (elastic#68538) [ML] Functional tests - fix job validation API test with maxModelMemoryLimit (elastic#68501) [ML] Functional tests - stabilize DFA job creation (elastic#68495) [TSVB] Allows the user to change the tooltip mode (elastic#67775) chore(NA): skip apis Endpoint plugin Endpoint alert API when data is in elasticsearch (elastic#68613) chore(NA): skip endpoint Endpoint Alert Page: when es has data and user has navigated to the page (elastic#68596) [SIEM][Detection Engine] Converts from joi to use io-ts and moves the types to common (elastic#68127) ...

* [ML] WIP UI support for the sug-aggs * [ML] extract sug-aggs from es config * [ML] fix for range filter * [ML] delete sub-aggregation * [ML] isConfigInvalid * [ML] fix actions, support parent agg * [ML] update getAggConfigFromEsAgg * [ML] unit tests * [ML] SubAggsSection * [ML] fix snapshot, add validation * [ML] fix label capping * [ML] rename Filter query label * [ML] fix updateSubAggregation validation * [ML] fix state update warning * [ML] increase MAX_NESTING_SUB_AGGS * [ML] functional tests for sub-aggregations * [ML] change the tooltip message * [ML] update functional tests * [ML] fix typo

darnautov added 3 commits June 4, 2020 17:15

[ML] WIP UI support for the sug-aggs

5e9ec0d

[ML] extract sug-aggs from es config

afb2171

[ML] fix for range filter

7986db0

darnautov changed the title ~~[ML] Transforms: Support sug-aggregations~~ [ML] Transforms: Support sub-aggregations Jun 4, 2020

darnautov added 6 commits June 5, 2020 09:56

[ML] delete sub-aggregation

1262caa

[ML] isConfigInvalid

ad507f3

[ML] fix actions, support parent agg

5902051

[ML] update getAggConfigFromEsAgg

8b121db

[ML] unit tests

8e63f25

[ML] SubAggsSection

147897e

darnautov commented Jun 8, 2020

View reviewed changes

[ML] fix snapshot, add validation

5b6c5ce

darnautov requested review from walterra, peteharverson and alvarezmelissa87 June 8, 2020 11:58

darnautov self-assigned this Jun 8, 2020

darnautov added :ml Feature:Transforms ML transforms v7.9.0 v8.0.0 release_note:enhancement labels Jun 8, 2020

[ML] fix label capping

d338ec0

darnautov marked this pull request as ready for review June 8, 2020 12:10

darnautov requested a review from a team as a code owner June 8, 2020 12:10

Merge remote-tracking branch 'upstream/master' into ML-transforms-sub…

a4edc96

…-aggs

darnautov requested a review from hendrikmuhs June 8, 2020 12:17

[ML] rename Filter query label

355b76a

elastic deleted a comment from kibanamachine Jun 8, 2020

darnautov added 2 commits June 8, 2020 16:01

[ML] fix updateSubAggregation validation

83e7477

[ML] fix state update warning

157ff2e

alvarezmelissa87 approved these changes Jun 8, 2020

View reviewed changes

walterra reviewed Jun 9, 2020

View reviewed changes

walterra approved these changes Jun 9, 2020

View reviewed changes

darnautov added 2 commits June 9, 2020 10:53

[ML] increase MAX_NESTING_SUB_AGGS

5bf217c

[ML] functional tests for sub-aggregations

3005944

darnautov requested a review from pheyos June 9, 2020 09:23

peteharverson reviewed Jun 9, 2020

View reviewed changes

...nsform/public/app/sections/create_transform/components/aggregation_list/sub_aggs_section.tsx Outdated Show resolved Hide resolved

darnautov added 2 commits June 9, 2020 13:05

[ML] change the tooltip message

b71fd7b

Merge remote-tracking branch 'upstream/master' into ML-transforms-sub…

efe2265

…-aggs

darnautov requested a review from peteharverson June 9, 2020 11:09

peteharverson approved these changes Jun 9, 2020

View reviewed changes

pheyos reviewed Jun 9, 2020

View reviewed changes

x-pack/test/functional/apps/transform/creation_index_pattern.ts Outdated Show resolved Hide resolved

[ML] update functional tests

0e90be9

darnautov requested a review from pheyos June 9, 2020 12:23

[ML] fix typo

7f51f7b

pheyos approved these changes Jun 9, 2020

View reviewed changes

darnautov merged commit 1160d08 into elastic:master Jun 9, 2020

darnautov deleted the ML-transforms-sub-aggs branch June 9, 2020 14:13

darnautov mentioned this pull request Jun 9, 2020

[7.x] [ML] Transforms: Support sub-aggregations (#68306) #68655

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ML] Transforms: Support sub-aggregations #68306

[ML] Transforms: Support sub-aggregations #68306

darnautov commented Jun 4, 2020 •

edited

Loading

darnautov Jun 8, 2020

elasticmachine commented Jun 8, 2020

alvarezmelissa87 left a comment

walterra left a comment •

edited

Loading

darnautov commented Jun 9, 2020

walterra left a comment

peteharverson commented Jun 9, 2020

darnautov commented Jun 9, 2020

peteharverson left a comment

pheyos left a comment

kibanamachine commented Jun 9, 2020

[ML] Transforms: Support sub-aggregations #68306

[ML] Transforms: Support sub-aggregations #68306

Conversation

darnautov commented Jun 4, 2020 • edited Loading

Summary

Checklist

darnautov Jun 8, 2020

Choose a reason for hiding this comment

elasticmachine commented Jun 8, 2020

alvarezmelissa87 left a comment

Choose a reason for hiding this comment

walterra left a comment • edited Loading

Choose a reason for hiding this comment

darnautov commented Jun 9, 2020

walterra left a comment

Choose a reason for hiding this comment

peteharverson commented Jun 9, 2020

darnautov commented Jun 9, 2020

peteharverson left a comment

Choose a reason for hiding this comment

pheyos left a comment

Choose a reason for hiding this comment

kibanamachine commented Jun 9, 2020

💚 Build Succeeded

History

darnautov commented Jun 4, 2020 •

edited

Loading

walterra left a comment •

edited

Loading