[Expressions] Variables should be available to sub-expressions

[wylieconlon]

It is currently not possible to set a variable that is used by a sub-expression. This came up recently when looking at how we could move away from the custom lens_multitable implementation.

For example, I would expect this to work:

kibana
| kibana_context
| var_set name="hello" value={esaggs ...}
| lens_xy_expression table={var name="hello"}

There are workarounds we can use for this problem, specifically we could change the example above to:

kibana
| kibana_context
| var_set name="hello" value={esaggs ...}
| lens_xy_expression table_var="hello"

[elasticmachine]

Pinging @elastic/kibana-app-arch (Team:AppArch)

[lukeelmers]

FWIW, at one point there was a proposal to make variables a first-class concept in the expression grammar & treat them like scope in JS, which might look something like this:

kibana
| kibana_context
| $hello={esaggs ...}
| lens_xy_expression table={$hello}

I don't think we ever went too far exploring this as the var and var_set workarounds were sufficient to begin with and didn't require changing the grammar, but perhaps it is worth revisiting that discussion.

[ppisljar]

i think its worth looking into why the example above doesn't work as it should and fixing that.

we can still look into changing grammar but i think that should be done independent from fixing this.
technically changing the grammar could/would work exactly the same. AST parser will most likely convert the special $grammar to specific function call.