[TSVB] Change term sorting to match Kibana Core

[simianhacker]

This PR changes the way TSVB sorts terms aggs to use the entire time range instead of the last bucket. For example, let say you're looking at the last 15 minutes, if you create a series where the metric is "average of system.cpu.user.pct" and set the interval to auto, TSVB currently will calculate the bucket size to be 10 seconds and then create a filter aggregation with a range that encapsulates the last 10 seconds of the time period along with the metric above and sort on the result. With this PR that filter aggregation (with the range) is removed and instead will sort on the average of system.cpu.user.pct for the entire time range (basically averaging all the buckets together).

This will make the results for basic aggregations in TSVB and Kibana Core match. This has been a source of confusion for a few users.

Thoughts? @alexfrancoeur @tbragin @rashidkpc

[simianhacker]

@monicasarbu This concerns you too :D

[alexfrancoeur]

@simianhacker woah, big change! While I agree that it is a huge source of confusion for users and am happy we're addressing it, I also believe that having the ability to show the most recent value is one of the stronger features of TSVB. Especially for operational monitoring use case. Will we still allow the option to report on the last full or partial bucket for a series? Or are we completely removing it? Also, how does this affect things like overall aggregations? Are they still useful? Maybe we can sync offline here, I'd just like to understand the impact a bit more.

[simianhacker]

@alexfrancoeur This only affects the order of the terms returned when doing a group by, everything else remains unchanged.

[shaharmor]

This not only fixes the order, it also fixes the issue of not showing the data for all the time range for some terms.

Without the fix:

With the fix:

This is a blessed fix!

[mattapperson]

LGTM

[simianhacker]

@alexfrancoeur Any objections? I'm ready to merge this.

[alexfrancoeur]

@simianhacker LGTM as well, no objections. I think we'll need to make sure #14798 is completed soon to reiterate what it means to be experimental. This will likely affect visualizations in production. @monicasarbu any objections on the beats side for modules?

[shaharmor]

Hey, any update on this?

[simianhacker]

Back ported to 6.x with 0b3de29