Add an API for field value suggestions?

[jpountz]

Kibana currently powers field value suggestions with aggregations, by enforcing strict timeouts in order to make sure to get suggestions in a timely fashion. Unfortunately, despite great care to make these suggestions cheap, this has been a source of performance/load issues in the past. Upcoming features like searchable snapshots and schema on read are going to make this worse.

With the introduction of constant_keyword and possibly enum, we now have the opportunity to return field suggestions on fields that are mapped with these types without performing any disk I/O. Should we expose this via a new API?

[elasticmachine]

Pinging @elastic/es-search (:Search/Search)

[romseygeek]

We can also get relatively cheap prefix suggestions directly from the terms dictionaries of keyword fields. I'd suggest a new API that looks something like this:

GET [index-pattern]/_field_values/[field]?size=10
GET [index-pattern]/_field_values/[field]/[prefix]?size=10

For constant_keyword fields we can return immediately from mappings; for keyword we consult the terms dictionary and return the first n matching values. Any other field types return an error, so you can't accidentally hit a source-only field and get a very slow response. We can filter out frozen indices, or maybe return values if we know that the relevant parts of the index have been cached on a node.

In order to keep things fast, we would not support:

• adding filters to only return values that are definitely included in a result set
• sorting by anything other than lexicographical value (eg no sorting by frequency or similar)

[markharwood]

for keyword we consult the terms dictionary and return the first n matching values.

I wonder to what extent aesthetics might play a part here and users want New York not new york because the field was normalized. It's a lot of extra work to get that cleaner experience.