[Lens] Enable new context constants in formula #158452

dej611 · 2023-05-25T12:33:50Z

Summary

Fixes #151827, Fixes #117548

This PR contains constant function features in Lens formula:

direct exposure of the time_range() - as suggested in Timespan user variable #117548)

interval() (as intended in [Lens] Support bucket interval on formulas #151827)

and now() (as suggested in here) functions in the formula input

A visual explanation of what each constant represent:

Documentation:

Some example usage

TSVB => Lens with params._interval support:

Notes:

context values work like static values, with the same limits, therefore it is not possible to build a date histogram with a context value alone (i.e. a formula with only interval() or now()). It works ok without the Date Histogram dimension.
The interval() function will report an error if used without a configured Date Histogram dimension:

The interval() function does not take into account different bucket interval size (i.e. DST changes, leap years, etc...), rather return the same value to all the buckets. This is the same behaviour as in TSVB, but in Lens it can be a problem due to the usage of calendar_interval.
I had to duplicate a couple of function from the helpers to avoid issues with tests. I've tried a different organization of the helpers (between pure vs impure fns) but that took longer than expected, so I would defer this task later in another PR.

General approach with `constant(...)` removed

* a more general approach using `constants(value="...")`

A cloud deployment has been created to test both approaches here. Let me know which one do you prefer cc @elastic/kibana-visualizations

Checklist

Delete any items that are not applicable to this PR.

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios
Any UI touched in this PR is usable by keyboard only (learn more about keyboard accessibility)
Any UI touched in this PR does not create any new axe failures (run axe in browser: FF, Chrome)
If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the docker list
This renders correctly on smaller devices using a responsive layout. (You can test this in your browser)
This was checked for cross-browser compatibility

Risk Matrix

Delete this section if it is not applicable to this PR.

Before closing this PR, invite QA, stakeholders, and other developers to identify risks that should be tested prior to the change/feature release.

When forming the risk matrix, consider some of the following examples and how they may potentially impact the change:

Risk	Probability	Severity	Mitigation/Notes
Multiple Spaces—unexpected behavior in non-default Kibana Space.	Low	High	Integration tests will verify that all features are still supported in non-default Kibana Space and when user switches between spaces.
Multiple nodes—Elasticsearch polling might have race conditions when multiple Kibana nodes are polling for the same tasks.	High	Low	Tasks are idempotent, so executing them multiple times will not result in logical error, but will degrade performance. To test for this case we add plenty of unit tests around this logic and document manual testing procedure.
Code should gracefully handle cases when feature X or plugin Y are disabled.	Medium	High	Unit tests will verify that any feature flag or plugin combination still results in our service operational.
See more potential risk examples

For maintainers

This was checked for breaking API changes and was labeled appropriately

…fix'

…o feature/151827

elasticmachine · 2023-05-31T16:27:47Z

Pinging @elastic/kibana-visualizations @elastic/kibana-visualizations-external (Team:Visualizations)

…o feature/151827

stratoula

I love it!! I have some questions:

Why the timerange() gives me B in the new metric/

Why this returns no data?

I am not sure about the kibana context title here. It makes sense to me but does it make sense for the users/
@gchaps @KOTungseth can you help here?

dej611 · 2023-06-01T13:46:11Z

I love it!! I have some questions:

Why the timerange() gives me B in the new metric/

I see you've taken a ~15 days time range, so in ms number values it's about 1.3 billion ms:

moment.duration(1300000000).humanize()
> '15 days'

moment.duration('15', 'd').asMilliseconds()
> 1296000000 // ~1.3b

Why this returns no data?

That is because it's subject to the same logic as any static value, unfortunately, over a date histogram: same applies if you write 5.
We thought in the past to handle this specific case applying a + count() - count() special formula under the hood, but never got the time to do it - nor think it's a good practice anyway 😅 .

I am not sure about the kibana context title here. It makes sense to me but does it make sense for the users/
@gchaps @KOTungseth can you help here?

Yes, a contribution on their side would be great here 👍

stratoula · 2023-06-02T06:07:59Z

I see you've taken a ~15 days time range, so in ms number values it's about 1.3 billion ms:

Oh it is a billion not bytes 😆

KOTungseth · 2023-06-02T11:12:55Z

...ns/lens/public/datasources/form_based/operations/definitions/formula/editor/formula_help.tsx

+        defaultMessage: 'Kibana context',
+      }),
+      i18n.translate('xpack.lens.formulaDocumentation.constantsSectionDescription', {
+        defaultMessage: 'Functions used to retrieve Kibana context variables.',


The other Formula sections in this window include thoroughly broken down definitions.

How about:
These functions are used to retrieve Kibana context variables, which are... and help you to....

I've revisited the copy here. Maybe we could discuss a bit and have a consistent copy for the entire section with a follow up PR here?

KOTungseth · 2023-06-02T11:17:45Z

...gins/lens/public/datasources/form_based/operations/definitions/formula/context_variables.tsx

+  label: 'Date histogram interval',
+  description: i18n.translate('xpack.lens.indexPattern.interval.documentation.markdown', {
+    defaultMessage: `
+  The current date histogram interval bucket expressed in milliseconds (ms).


How about:

The specified minimum interval for the date histogram, in milliseconds (ms).

KOTungseth · 2023-06-02T11:19:32Z

...gins/lens/public/datasources/form_based/operations/definitions/formula/context_variables.tsx

+  label: 'Current now',
+  description: i18n.translate('xpack.lens.indexPattern.now.documentation.markdown', {
+    defaultMessage: `
+  The current now moment used in Kibana expressed in milliseconds (ms).


I'm unsure what now moment means?

I meant the instant value of now, expressed in ms

KOTungseth · 2023-06-02T11:20:32Z

...gins/lens/public/datasources/form_based/operations/definitions/formula/context_variables.tsx

+  label: 'Time range',
+  description: i18n.translate('xpack.lens.indexPattern.timeRange.documentation.markdown', {
+    defaultMessage: `
+The current time range expressed in milliseconds (ms).


How about:

The specified time range, in milliseconds (ms).

stratoula

This looks great. I am not sure about the now description but I don't have a better alternative tbh.

Other than that (we can always improve on a follow up PR) everything seems to work fine. LGTM!

stratoula

This looks great. I am not sure about the now description but I don't have a better alternative tbh.

Other than that (we can always improve on a follow up PR) everything seems to work fine. LGTM!

kibana-ci · 2023-06-06T11:01:31Z

💛 Build succeeded, but was flaky

Failed CI Steps

FTR Configs #64

Test Failures

[job] [logs] FTR Configs #64 / machine learning - short tests model management trained models for ML power user displays a model with an ingest pipeline and model can be deleted with associated ingest pipelines

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`lens`	884	885	+1

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`lens`	1.2MB	1.2MB	+3.1KB
`visTypeTimeseries`	507.6KB	507.7KB	+82.0B
total			+3.2KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`lens`	34.9KB	34.9KB	+31.0B

Unknown metric groups

ESLint disabled line counts

id	before	after	diff
`enterpriseSearch`	19	21	+2
`securitySolution`	413	417	+4
total			+6

Total ESLint disabled count

id	before	after	diff
`enterpriseSearch`	20	22	+2
`securitySolution`	497	501	+4
total			+6

History

💛 Build #132542 was flaky 18964ef
💚 Build #131655 succeeded 1213bd5
💛 Build #131418 was flaky ab163c3
💚 Build #131394 succeeded 173876b
💔 Build #131304 failed 40dd3d0

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

dej611 added 3 commits May 25, 2023 13:23

✨ Add new constants features in formula

ba70d2f

🐛 Fix for now fn

bb9391a

✨ TSVB to Lens support

6cba35c

dej611 added Team:Visualizations Visualization editors, elastic-charts and infrastructure Feature:Lens ci:cloud-deploy Create or update a Cloud deployment labels May 25, 2023

dej611 and others added 12 commits May 26, 2023 12:12

:bugs: Fix some issues

ba418db

Merge remote-tracking branch 'upstream/main' into feature/151827

1986128

:bugs: Fix interval + other small fixes

9f65b46

🔧 Fix import order

abd79a8

🔧 Minor tweaks

f5c278a

✅ Add unit tests

9e5e135

✅ Fix tests

a4cf88e

[CI] Auto-commit changed files from 'node scripts/lint_ts_projects --…

912a95a

…fix'

✅ Add more tests

a72bdcb

Merge branch 'feature/151827' of https://github.com/dej611/kibana int…

f22399e

…o feature/151827

✅ Remove constant and add tests for functions

700465c

🐛 Duplicate functions for now

1418001

dej611 added v8.9.0 release_note:enhancement labels May 31, 2023

dej611 added 2 commits May 31, 2023 14:59

🐛 Fix importing issues

40dd3d0

📸 Update snapshot tests

173876b

dej611 marked this pull request as ready for review May 31, 2023 16:27

dej611 requested a review from a team as a code owner May 31, 2023 16:27

dej611 and others added 3 commits May 31, 2023 19:02

Merge branch 'main' into feature/151827

ab163c3

Merge remote-tracking branch 'upstream/main' into feature/151827

7ae0789

Merge branch 'feature/151827' of https://github.com/dej611/kibana int…

1213bd5

…o feature/151827

stratoula reviewed Jun 1, 2023

View reviewed changes

KOTungseth reviewed Jun 2, 2023

View reviewed changes

dej611 and others added 3 commits June 5, 2023 14:32

Merge remote-tracking branch 'upstream/main' into feature/151827

297d677

👌 Integrated feedback

18964ef

Merge branch 'main' into feature/151827

a561f0e

stratoula approved these changes Jun 6, 2023

View reviewed changes

dej611 merged commit 3600f97 into elastic:main Jun 6, 2023

kibanamachine added the backport:skip This commit does not require backporting label Jun 6, 2023

dej611 removed the ci:cloud-deploy Create or update a Cloud deployment label Jun 6, 2023

dej611 mentioned this pull request Jun 6, 2023

[Lens] Improve formula context variables documentation #159115

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Lens] Enable new context constants in formula #158452

[Lens] Enable new context constants in formula #158452

dej611 commented May 25, 2023 •

edited by stratoula

Loading

elasticmachine commented May 31, 2023

stratoula left a comment

dej611 commented Jun 1, 2023 •

edited

Loading

stratoula commented Jun 2, 2023

KOTungseth Jun 2, 2023

dej611 Jun 6, 2023

KOTungseth Jun 2, 2023

KOTungseth Jun 2, 2023

dej611 Jun 5, 2023

KOTungseth Jun 2, 2023

stratoula left a comment

stratoula left a comment

kibana-ci commented Jun 6, 2023 •

edited

Loading

ESLint disabled line counts

Total ESLint disabled count

[Lens] Enable new context constants in formula #158452

[Lens] Enable new context constants in formula #158452

Conversation

dej611 commented May 25, 2023 • edited by stratoula Loading

Summary

Checklist

Risk Matrix

For maintainers

elasticmachine commented May 31, 2023

stratoula left a comment

Choose a reason for hiding this comment

dej611 commented Jun 1, 2023 • edited Loading

stratoula commented Jun 2, 2023

KOTungseth Jun 2, 2023

Choose a reason for hiding this comment

dej611 Jun 6, 2023

Choose a reason for hiding this comment

KOTungseth Jun 2, 2023

Choose a reason for hiding this comment

KOTungseth Jun 2, 2023

Choose a reason for hiding this comment

dej611 Jun 5, 2023

Choose a reason for hiding this comment

KOTungseth Jun 2, 2023

Choose a reason for hiding this comment

stratoula left a comment

Choose a reason for hiding this comment

stratoula left a comment

Choose a reason for hiding this comment

kibana-ci commented Jun 6, 2023 • edited Loading

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

Metrics [docs]

Module Count

Async chunks

Page load bundle

ESLint disabled line counts

Total ESLint disabled count

History

dej611 commented May 25, 2023 •

edited by stratoula

Loading

dej611 commented Jun 1, 2023 •

edited

Loading

kibana-ci commented Jun 6, 2023 •

edited

Loading