[ML] Transforms: API schemas and integration tests #75164

walterra · 2020-08-17T14:17:46Z

Summary

Part of #63700.

Adds schema definitions to transform API endpoints and adds API integration tests.
The type definitions based on the schema definitions can be used on the client side too.
Adds apidoc documentation. (The scripts to create the docs will be added in a follow up pending a fix for apidocs on master)
As part of creating the schema definitions some code cleanup and consolidation was done (to make certain types available on both server and client side they had to be moved to common).

/api/transform/transforms
/api/transform/transforms/{transformId}
/api/transform/transforms/_stats
/api/transform/transforms/_preview
/api/transform/transforms/{transformId}/_update
/api/transform/start_transforms
/api/transform/stop_transforms

Checklist

Unit or functional tests were updated or added to match the most common scenarios

For maintainers

This was checked for breaking API changes and was labeled appropriately

walterra · 2020-08-21T12:05:33Z

@elasticmachine merge upstream

kibanamachine · 2020-08-31T12:48:29Z

⏳ Build in-progress, with failures

continuous-integration/kibana-ci/pull-request
Commit: f2b0c10b61ed66ea7b3a77ddebc1cd967624d95c
This comment will update when the build is complete

Failed CI Steps

History

💔 Build #71225 failed 6353ef329d6a6e51f172357feb98b488a46b2566
💔 Build #71202 failed 07e0b8d4e8cb4668dc49920b99a86524b87a6506
💔 Build #70924 failed 4b1ca22b005ba3c6aa769af6ab836620f157cf6e
💔 Build #70906 failed b00647513b2365553d1d7955fb8a9bf15ba92320
💔 Build #70902 failed 50534e56c2a1326b8e984f247e74db1655ff25e9

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

…ak the optimizer.

pheyos

API integration tests LGTM 🎉

kibanamachine · 2020-09-14T09:52:52Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: 67b32a8

Build metrics

@kbn/optimizer bundle module count

id	value	diff	baseline
transform	460	+1	459

async chunks size

id	value	diff	baseline
transform	768.7KB	+9.1KB	759.6KB

page load bundle size

id	value	diff	baseline
ml	844.6KB	-244.0B	844.8KB
transform	25.5KB	+230.0B	25.3KB
total		-14.0B

distributable file count

id	value	diff	baseline
default	45545	+20	45525

History

💚 Build #74024 succeeded 00bdc3b
💚 Build #73640 succeeded 221c920
💚 Build #73631 succeeded efcf9b3
💔 Build #73612 failed 7329dea
💔 Build #73595 failed 7725de8

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

qn895 · 2020-09-14T14:03:33Z

Changes LGTM 💯

jgowdyelastic

LGTM

- Adds schema definitions to transform API endpoints and adds API integration tests. - The type definitions based on the schema definitions can be used on the client side too. - Adds apidoc documentation.

* master: (26 commits) updating datatable type (elastic#77320) [ML] Fix custom URLs processing for security app (elastic#76957) [telemetry] add schema guideline + schema_check new check for --path config (elastic#75747) [ML] Transforms: API schemas and integration tests (elastic#75164) [Mappings editor] Add support for wildcard field type (elastic#76574) [Ingest Manager] Fix flyout instruction selection (elastic#77071) [Telemetry Tools] update lodash to 4.17 (elastic#77317) [APM] Service inventory redesign (elastic#76744) Hide management sections based on cluster/index privileges (elastic#67791) [Snapshot Restore] Disable steps when form is invalid (elastic#76540) [Mappings editor] Add support for positive_score_impact to rank_feature (elastic#76824) Update apm.ts (elastic#77310) [OBS] Remove beta badge, change news feed size and add external icon to news feed link (elastic#77164) [Discover] Convert legacy sort to be compatible with multi sort (elastic#76986) [APM] API Snapshot Testing (elastic#77229) [ML] Functional tests - increase wait time for DFA start (elastic#77307) [UiActions][Drilldowns] Fix actions sorting in context menu (elastic#77162) [Drilldowns] Wire up new links to new docs (elastic#77154) Fix APM issue template [Ingest Pipelines] Drop into an empty tree (elastic#76885) ...

* master: (65 commits) [Security Solution][Resolver] Analyzed event styling (elastic#77115) filter invalid SOs from the searc hresults in Task Manager (elastic#76891) [RUM Dashboard] Visitors by region map (elastic#77135) [Security Solution][Endpoint][Admin] Task/endpoint list actions (elastic#76555) [Ingest pipelines] Forms for processors T-U (elastic#76710) updating datatable type (elastic#77320) [ML] Fix custom URLs processing for security app (elastic#76957) [telemetry] add schema guideline + schema_check new check for --path config (elastic#75747) [ML] Transforms: API schemas and integration tests (elastic#75164) [Mappings editor] Add support for wildcard field type (elastic#76574) [Ingest Manager] Fix flyout instruction selection (elastic#77071) [Telemetry Tools] update lodash to 4.17 (elastic#77317) [APM] Service inventory redesign (elastic#76744) Hide management sections based on cluster/index privileges (elastic#67791) [Snapshot Restore] Disable steps when form is invalid (elastic#76540) [Mappings editor] Add support for positive_score_impact to rank_feature (elastic#76824) Update apm.ts (elastic#77310) [OBS] Remove beta badge, change news feed size and add external icon to news feed link (elastic#77164) [Discover] Convert legacy sort to be compatible with multi sort (elastic#76986) [APM] API Snapshot Testing (elastic#77229) ...

- Adds schema definitions to transform API endpoints and adds API integration tests. - The type definitions based on the schema definitions can be used on the client side too. - Adds apidoc documentation.

walterra added :ml v8.0.0 test-api-integration Feature:Transforms ML transforms backport:skip This commit does not require backporting v7.10.0 labels Aug 17, 2020

walterra self-assigned this Aug 17, 2020

walterra added release_note:skip Skip the PR/issue when compiling release notes and removed backport:skip This commit does not require backporting labels Aug 19, 2020

walterra force-pushed the ml-transforms-api-tests branch from 7fe69ff to c53ae25 Compare August 20, 2020 15:39

walterra changed the title ~~[ML] Transforms: API integration tests~~ [ML] Transforms: API schemas integration tests Aug 20, 2020

walterra changed the title ~~[ML] Transforms: API schemas integration tests~~ [ML] Transforms: API schemas and integration tests Aug 20, 2020

walterra force-pushed the ml-transforms-api-tests branch 4 times, most recently from b006475 to 4b1ca22 Compare August 27, 2020 16:05

walterra force-pushed the ml-transforms-api-tests branch from 4b1ca22 to 07e0b8d Compare August 31, 2020 09:38

walterra force-pushed the ml-transforms-api-tests branch 2 times, most recently from 5af74bc to 87647ba Compare September 1, 2020 07:12

walterra added 9 commits September 1, 2020 16:19

[ML] Transform list API integration test.

2a2c3f3

[ML] Get single transform API integration test.

3bdeb4f

[ML] Transform stats API integration test.

3c5eaf2

[ML] Transform preview API integration test.

9f3643a

[ML] Transform update API integration test.

ed353b0

[ML] Fix flakyness.

9a657b1

[ML] WIP: API schemas and API integration tests for starting transforms.

9de6d56

[ML] Delete tmp utility file.

53240e8

[Ml] Fix to import types only of schema based definitions and not bre…

a2014c0

…ak the optimizer.

walterra added 12 commits September 9, 2020 16:11

[ML] Fix type guards.

894b1f8

[ML] Use TRANSFORM_STATE instead of raw strings.

e434c4f

Merge branch 'master' into ml-transforms-api-tests

7725de8

[ML] Fix types.

7329dea

[ML] Extend stats assertions.

b63d86d

[ML] Fix error toasts.

c47f2b0

[ML] Extend transforms assertions.

28ad1f2

[ML] Text tweak.

2320fbc

[ML] Extend stop transforms assertions.

efcf9b3

[ML] Extend transforms assertions.

c6f388f

[ML] Better transforms ids.

cd3703a

[ML] Add comment to clarify destructuring.

221c920

walterra requested review from pheyos, jgowdyelastic and qn895 September 10, 2020 10:31

pheyos approved these changes Sep 10, 2020

View reviewed changes

walterra added 2 commits September 14, 2020 09:46

Merge branch 'master' into ml-transforms-api-tests

00bdc3b

Merge branch 'master' into ml-transforms-api-tests

67b32a8

qn895 approved these changes Sep 14, 2020

View reviewed changes

jgowdyelastic approved these changes Sep 14, 2020

View reviewed changes

walterra merged commit dd18220 into elastic:master Sep 14, 2020

walterra deleted the ml-transforms-api-tests branch September 14, 2020 14:34

walterra mentioned this pull request Sep 14, 2020

[7.x] [ML] Transforms: API schemas and integration tests (#75164) #77357

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ML] Transforms: API schemas and integration tests #75164

[ML] Transforms: API schemas and integration tests #75164

walterra commented Aug 17, 2020 •

edited

Loading

walterra commented Aug 21, 2020

kibanamachine commented Aug 31, 2020

pheyos left a comment

kibanamachine commented Sep 14, 2020

qn895 commented Sep 14, 2020

jgowdyelastic left a comment

[ML] Transforms: API schemas and integration tests #75164

[ML] Transforms: API schemas and integration tests #75164

Conversation

walterra commented Aug 17, 2020 • edited Loading

Summary

Checklist

For maintainers

walterra commented Aug 21, 2020

kibanamachine commented Aug 31, 2020

⏳ Build in-progress, with failures

Failed CI Steps

History

pheyos left a comment

Choose a reason for hiding this comment

kibanamachine commented Sep 14, 2020

💚 Build Succeeded

Build metrics

@kbn/optimizer bundle module count

async chunks size

page load bundle size

distributable file count

History

qn895 commented Sep 14, 2020

jgowdyelastic left a comment

Choose a reason for hiding this comment

walterra commented Aug 17, 2020 •

edited

Loading