[Lens] Supports percentile_ranks aggregation #132430

stratoula · 2022-05-18T13:52:25Z

Summary

Adds support for percentile ranks aggregation in Lens and enables the TSVB to Lens navigation.

Formula usage

percentile_rank(AvgTicketPrice, value=600)

Formula documentation

I took some pieces from the ES documentation as I find it quite explanatory and easy to understand. I am open to feedback!

Checklist

Delete any items that are not applicable to this PR.

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios
This was checked for cross-browser compatibility

stratoula · 2022-05-20T11:15:54Z

src/plugins/vis_types/timeseries/public/trigger_action/index.test.ts

@@ -87,7 +87,7 @@ describe('triggerTSVBtoLensConfiguration', () => {
          ...model.series[0],
          metrics: [
            {
-              type: 'percentile_rank',
+              type: 'std_deviation',


ℹ️ This test is testing the scenario that a metric agg is not supported in Lens. As we support now the percentile_rank I changed it to std_deviation that is not supported

stratoula · 2022-05-20T11:18:39Z

x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/percentile_ranks.tsx

+    undefined,
+    timeShift
+  );
+}


The label is following the aggBased format
Percentile rank (value) of

I could remove the parenthesis but I like it that way, wdyt?

Looks good to me!

stratoula · 2022-05-20T11:19:14Z

x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/percentile_ranks.tsx

+  );
+}
+
+const DEFAULT_PERCENTILE_RANKS_VALUE = 0;


ℹ️ I am defaulting this to 0 as done in TSVB and Agg based.

elasticmachine · 2022-05-25T11:01:53Z

Pinging @elastic/kibana-vis-editors @elastic/kibana-vis-editors-external (Team:VisEditors)

flash1293

Added some comments

x-pack/plugins/lens/public/types.ts

flash1293 · 2022-05-25T15:46:12Z

x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/percentile_ranks.tsx

+          data-test-subj="lns-indexPattern-percentile_ranks-input"
+          compressed
+          value={inputValue ?? ''}
+          onChange={handleInputChange}


This has the same issue as percentiles in that it's not possible to order terms by a decimal point percentile value: elastic/elasticsearch#66677

The query fails if you try to do it on this PR. We can either limit it in the same way (enforce integers) or we can drop out of the terms sorting and fall back to alphabetical. What do you think @stratoula @ghudgins ? I'm not sure tbh - integers make more sense for percentile because the value range is strictly 0-100 which is not the case for percentile rank (it can have any data range so you wouldn't be able to use it for fields which have a domain of 0-1 which would be really annoying).

Awww nice catch. I was always using integer values and never caught it. Hmmm, I don't know either but I feel that as a user I would prefer adding whatever value I want and not be limited only to integers

Yeah, we probably should fall back to alphabetical sorting (which is weird too, no easy win here)

Check this comment #132430 (comment)

x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/percentile_ranks.tsx

flash1293 · 2022-05-25T15:46:51Z

x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/percentile_ranks.tsx

+    undefined,
+    timeShift
+  );
+}


Looks good to me!

stratoula · 2022-05-26T09:40:54Z

@flash1293 about the sorting problem I did the following, tell me what you think about it:

If the user is using a percentile rank with an integer value I allow the sorting by column
If the user is using a percentile rank with a non integer value I default to alphabetical

I thought that with this implementation, only the users with the non-integer values will have this limitation. Wdyt? Otherwise I can disable the column order for all the percentile ranks.

flash1293

LGMT, one minor nit. Great work!

x-pack/plugins/lens/public/types.ts

jloleysens

Changes look good to me! Also tested locally. Great work @stratoula ! I left a couple of nits.

src/plugins/data/common/search/aggs/metrics/single_percentile_rank.test.ts

jloleysens · 2022-05-31T08:39:06Z

src/plugins/data/common/search/aggs/metrics/single_percentile_rank_fn.ts

+export const aggSinglePercentileRank = (): FunctionDefinition => ({
+  name: aggSinglePercentileRankFnName,
+  help: i18n.translate('data.search.aggs.function.metrics.singlePercentileRank.help', {
+    defaultMessage: 'Generates a serialized agg config for a single percentile rank agg',


nit casing consistency with other agg fns:

Suggested change

defaultMessage: 'Generates a serialized agg config for a single percentile rank agg',

defaultMessage: 'Generates a serialized agg config for a Single Percentile Rank agg',

I am happy to do this change @jloleysens but the single percentile follows this format https://github.com/elastic/kibana/blob/main/src/plugins/data/common/search/aggs/metrics/single_percentile_fn.ts#L28 so I think that the single percentile rank should also follow the same format. Also I think that we follow our text guildelines with this format.

kibana-ci · 2022-05-31T10:37:36Z

💚 Build Succeeded

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`data`	513	515	+2
`lens`	870	871	+1
total			+3

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`data`	2863	2871	+8
`lens`	509	512	+3
total			+11

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`lens`	1.2MB	1.2MB	+3.6KB
`visTypeTimeseries`	465.6KB	466.4KB	+836.0B
total			+4.4KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`data`	420.7KB	423.8KB	+3.2KB
`visTypeGauge`	9.9KB	10.0KB	+52.0B
`visTypeHeatmap`	10.4KB	10.5KB	+26.0B
`visTypeMetric`	8.8KB	8.9KB	+26.0B
`visTypeTable`	15.3KB	15.3KB	+26.0B
`visTypeTagcloud`	5.9KB	5.9KB	+26.0B
`visTypeXy`	43.4KB	43.5KB	+104.0B
total			+3.4KB

Unknown metric groups

API count

id	before	after	diff
`data`	3478	3486	+8
`lens`	586	589	+3
total			+11

History

💚 Build #48264 succeeded beccdf9
💚 Build #48056 succeeded 3801f2c
💚 Build #47793 succeeded 5c4edee
💔 Build #47786 failed b53a1da
💔 Build #47763 failed 95f0a45
💚 Build #47755 succeeded 58e7867

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

stratoula added 6 commits May 18, 2022 16:51

[Lens] Supports percentile_ranks aggregation

061f71a

Fix type

ea28b8c

Fix some types and the filter by problem

4ccdccb

Merge branch 'main' into lens-percentile-rank

23b9daa

Enable navigation from TSVB to Lens

24b49bd

Add unit tests, fix terms with filter

466a631

stratoula commented May 20, 2022

View reviewed changes

stratoula added release_note:enhancement Feature:Lens backport:skip This commit does not require backporting v8.3.0 labels May 20, 2022

stratoula marked this pull request as ready for review May 20, 2022 15:29

stratoula requested review from a team as code owners May 20, 2022 15:29

stratoula added 2 commits May 23, 2022 10:11

Merge branch 'main' into lens-percentile-rank

edae37c

Merge branch 'main' into lens-percentile-rank

215026a

stratoula added the Team:Visualizations Visualization editors, elastic-charts and infrastructure label May 25, 2022

flash1293 reviewed May 25, 2022

View reviewed changes

stratoula added 4 commits May 25, 2022 18:18

Merge branch 'main' into lens-percentile-rank

0f1592f

Merge branch 'main' into lens-percentile-rank

58e7867

Change the implementation to use singleRercentileRank agg config

95f0a45

Not allows sorting by column for non integer values

363c14f

stratoula added 2 commits May 26, 2022 13:07

Remove console.log

b53a1da

Fix jest test

5c4edee

stratoula added v8.4.0 and removed v8.3.0 labels May 26, 2022

Merge branch 'main' into lens-percentile-rank

3801f2c

flash1293 approved these changes May 30, 2022

View reviewed changes

x-pack/plugins/lens/public/types.ts Outdated Show resolved Hide resolved

stratoula added 2 commits May 31, 2022 08:29

Merge branch 'main' into lens-percentile-rank

a4514c9

Remove leftover comment

beccdf9

jloleysens approved these changes May 31, 2022

View reviewed changes

Apply PR comment

ae72f81

stratoula merged commit 8686943 into elastic:main May 31, 2022

This was referenced Aug 8, 2023

Sort terms agg by percentiles including decimal point elastic/elasticsearch#66677

Open

[Lens] Allow decimal percentiles like 99.5 #98853

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Lens] Supports percentile_ranks aggregation #132430

[Lens] Supports percentile_ranks aggregation #132430

stratoula commented May 18, 2022 •

edited

Loading

stratoula May 20, 2022

stratoula May 20, 2022 •

edited

Loading

flash1293 May 25, 2022

stratoula May 20, 2022

elasticmachine commented May 25, 2022

flash1293 left a comment

flash1293 May 25, 2022

stratoula May 25, 2022

flash1293 May 25, 2022

stratoula May 26, 2022

flash1293 May 25, 2022

stratoula commented May 26, 2022

flash1293 left a comment

jloleysens left a comment •

edited

Loading

jloleysens May 31, 2022

stratoula May 31, 2022 •

edited

Loading

kibana-ci commented May 31, 2022

API count

	defaultMessage: 'Generates a serialized agg config for a single percentile rank agg',
	defaultMessage: 'Generates a serialized agg config for a Single Percentile Rank agg',

[Lens] Supports percentile_ranks aggregation #132430

[Lens] Supports percentile_ranks aggregation #132430

Conversation

stratoula commented May 18, 2022 • edited Loading

Summary

Checklist

Choose a reason for hiding this comment

stratoula May 20, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

elasticmachine commented May 25, 2022

flash1293 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stratoula commented May 26, 2022

flash1293 left a comment

Choose a reason for hiding this comment

jloleysens left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stratoula May 31, 2022 • edited Loading

Choose a reason for hiding this comment

kibana-ci commented May 31, 2022

💚 Build Succeeded

Metrics [docs]

Module Count

Public APIs missing comments

Async chunks

Page load bundle

API count

History

stratoula commented May 18, 2022 •

edited

Loading

stratoula May 20, 2022 •

edited

Loading

jloleysens left a comment •

edited

Loading

stratoula May 31, 2022 •

edited

Loading