-
Notifications
You must be signed in to change notification settings - Fork 24.9k
Commit
Adds support to the compute engine for filtering which positions are processed by grouping aggs. This should allow syntax like ``` | STATS success = COUNT(*) WHERE 200 <= response_code AND response_code < 300, redirect = COUNT(*) WHERE 300 <= response_code AND response_code < 400, client_err = COUNT(*) WHERE 400 <= response_code AND response_code < 500, server_err = COUNT(*) WHERE 500 <= response_code AND response_code < 600, total_count = COUNT(*) BY hostname ``` We could translate the WHERE expression into an `ExpressionEvaluator` and run it, then plug it into the filtering support added in this PR. The actual filtering is done by creating a `FilteredGroupingAggregatorFunction` which runs wraps a regular `GroupingAggregatorFunction` first executing the filter against the incoming `Page` and then `null`ing any positions in the group that don't match. Then passing the resulting groups into the real aggregator. When the real grouping aggregator implementation sees `null` value for groups it skips collecting that position. We had to make two changes to every agg for this to work: 1. Add a method to force local group tracking mode on any aggregator. Previously this was only required if the agg encountered `null` values, but when we're filtering aggs we can no longer trust the `seen` parameter we get when building the result. This local group tracking mode let's us track what we've actually seen locally. 2. Add `Releasable` to the `AddInput` thing we use to handle chunked pages in grouping aggs. This is required because the results of the filter must be closed on completion. Both of these are fairly trivial changes, but require touching every aggregation.
- Loading branch information
There are no files selected for viewing
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.