[KQL]: Autocomplete performance issues with runtime fields

[stacey-gammon]

Runtime fields will soon be addable to an Elasticsearch mapping (#75469).

When they do, KQL and the query bar will have to explicitly filter out field value suggestions for runtime fields, or there will be performance issues.

In order to do this, we'll have to know which fields returned from the field capabilities API are runtime fields.

LOE: TBD

cc @alexh97 @mattkime @ppisljar @jimczi @AlonaNadler

[elasticmachine]

Pinging @elastic/kibana-app-arch (Team:AppArch)

[AlonaNadler]

I'm conflicted here. I understand there will be a performance issue, we do want to have runtime fields behave like any other field. My question is whether we can have an advance setting for turning on/off the runtime field suggestions (I expect filters will work the same)?

[skearns64]

@stacey-gammon - can you give an example of the query that goes to ES to provide the KQL suggestions? Trying to see the scale of the impact - e.g. are we querying across all docs, a recent set of docs, using sampling, etc?

And how aggressively do we run the requests? e.g. for each keystroke, or do we run only after a stretch of typing is completed?

[stacey-gammon]

They query looks like:

POST /kibana_sample_data_flights/_search
{"size":0,"timeout":"1000ms","terminate_after":100000,"query":{"bool":{"filter":[]}},"aggs":{"suggestions":{"terms":{"field":"Carrier","include":"JetBea.*","execution_hint":"map","shard_size":10}}}}

Looks like we have a 500ms debounce so it's not every keystroke (https://github.com/elastic/kibana/pull/48450/files#diff-b24c79dfa3f4f9234e3cc6791640655dR77)

We have an open issue regarding performances issues with field value suggestions: #46054. Runtime fields would make this worse.

[lukasolson]

I'm conflicted here. I understand there will be a performance issue, we do want to have runtime fields behave like any other field. My question is whether we can have an advance setting for turning on/off the runtime field suggestions (I expect filters will work the same)?

It sounds like, in the current design, we have no way (through the field_caps API which we use to populate index patterns) to know whether a field is a regular or runtime field. Is that a correct assumption? If that's the case, then I'm not sure how we would have an advanced setting to toggle this behavior on/off.

Currently, we only request value suggestions for aggregatable fields of type string. If a runtime field is considered aggregatable and of type string, I'm not sure how to move forward with this issue unless we have some indication that the field is a runtime field.

[stacey-gammon]

@lukasolson - My understanding is that the ES folks will add this information to field_caps if we need it.

cc @jimczi @javanna to confirm.

[javanna]

It's an open discussion. We'd prefer that nobody needs this info in field_caps, because it introduces special treatment for runtime fields which means that they won't be "just like any other field". I think it would be good to discuss options to solve the problem mentioned in this issue.

[stacey-gammon]

APM and Logs UI use custom query bars which use App Arch's auto complete suggestion services. Unless they do extra work to filter runtime fields out, they will end up requesting field value suggestions for runtime fields as well.

Do we have an idea for how bad these performance issues will be? Is the recommendation from ES "definitely do not do this" or, "watch out for this"?

@lukasolson or @lizozom - Is there any logic that cancels the previous request as a new one is sent out, as the user continues typing?

cc @sqren and @weltenwort

[lukasolson]

Yes, we are cancelling the previous requests when a new query is issued (both for KQL value suggestions and for filter editor suggestions).

[stacey-gammon]

@sqren or @weltenwort - are you cancelling outdated queries as well?

[weltenwort]

Not in the infra plugin AFAIK. We're just using the getQuerySuggestions function:

kibana/x-pack/plugins/infra/public/containers/with_kuery_autocompletion.tsx

Lines 78 to 86 in d6c45a2

	const suggestions =
	(await this.props.kibana.services.data.autocomplete.getQuerySuggestions({
	language,
	query: expression,
	selectionStart: cursorPosition,
	selectionEnd: cursorPosition,
	indexPatterns: [indexPattern],
	boolFilter: [],
	})) || [];

But we could cancel if the API allows for it.

It might also be worth considering this in a broader "make it slow" context. Won't searchable snapshots cause a similar performance problem? Can we solve it in a more abstract way by setting a parsimonious timeout?

[ppisljar]

A few time I heard runtime fields should be "just like any other field". So that would mean we shouldn't treat them differently and assume no performance penalty ?

[javanna]

@ppisljar that would be preferable. I was in touch with Stacey yesterday about this, I am going to look at this issue and gather more context so we can discuss it in depth with the team and find a way forward.

[stacey-gammon]

We decided to keep this as a general issue with autocomplete performance which is being tracked here: #46054

We shouldn't do anything special with runtime fields. We'll live with the fact that they will be slow for suggestions and try to solve that via a new ES endpoint that would help all slow suggestions, not just runtime fields.

Closing in favor of #46054