No need to use Field Stats API in the dashboard and visualise tabs (and possibly Discover too)

[colings86]

Now that requests are rewritten in elasticsearch on each shard. If the range query and the values for the field on the shard do not overlap the query is rewritten to a MatchNoDocsQuery. This means that that part of the query is effectively not run if the shard does not contain terms in the range of the range query. Also because the request will now be cached in the request cache, subsequent searches will also be rewritten and retrieve the results for the request cache making searching non-matching indexes very fast.

Because of this neither the visualise or dashboard tabs need to make calls to the field stats API before they execute the search and can instead just run the search on `indexPattern*. So the field stats API can be removed from these tabs.

The discover tab still may require the field stats API for the segmented fetch it performs to know which order to request data from the indexes, but it would be worth testing whether the request cache and rewrite improvements in ES 5.0 negate the need consult field stats and if the discover tab is fast enough without using field stats now.

[spalger]

I love any chance I get to remove code @epixa wrote

[epixa]

Interesting... this is exciting.

We should make sure to test this change to Kibana on different setups before shipping it, but this could be a very nice change.

[epixa]

One thing to note here that was brought up recently in a discussion about the cross-cluster client: there is currently a soft-limit of a max of 1000 shards at a time for queries. That will probably need to be addressed in Elasticsearch by doing a pre-query check phase to reduce shard numbers or by implementing stepwise-reduction of results. @clintongormley is there an elasticsearch issue for that?

[clintongormley]

@epixa not yet - i'll update here when there is one

[clintongormley]

@epixa First step towards incremental reduction of query responses elastic/elasticsearch#23253

[epixa]

I created #11011 to track the effort to support cross-cluster search, and that will end up addressing this issue by deprecating this old field_stats based behavior for new index patterns.

[jbudz]

Closed by #12814