Optimize lone single bucket date_histogram
#71180
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This restores SQL's test for fetching `half_floats` after we backported the precision change in that fetch (elastic#70653)
This optimizes the `date_histogram` agg when there is a single bucket and no sub-aggregations. We expect this to happen from time to time when the buckets are larger than a day because folks often use "daily" indices. This was already fairly fast, but using the metadata makes it 10x faster. Something like 98ms becomes 7.5ms. Nice if you can get it! Like elastic#69377 this optimization will disable itself if you have document level security enabled or are querying a rollup index. Also like elastic#69377 it won't do anything if there is a top level query.
|
elasticmachine
added
the
Team:Analytics
|
Pinging @elastic/es-analytics-geo (Team:Analytics) |
|
run elasticsearch-ci/1 |
not-napoleon
approved these changes
May 5, 2021
| @@ -52,6 +55,15 @@ | |||
| if (query instanceof TermQuery) { | |||
| return new TermQueryToFilterAdapter(searcher, key, (TermQuery) query); | |||
| } | |||
| if (query instanceof ConstantScoreQuery) { | |||
There was a problem hiding this comment.
It seems like an obvious question "Why don't we check for a wrapped TermsQuery or MatchAllDocsQuery?" Would be good to have a comment to answer that.
There was a problem hiding this comment.
Because I've not seen it come up. But, looking at it with fresh eyes now, I think the safest thing is to always unwrap.
| @@ -386,4 +398,50 @@ void collectDebugInfo(BiConsumer<String, Object> add) { | |||
| add.accept("results_from_metadata", resultsFromMetadata); | |||
| } | |||
| } | |||
|
|
|||
| private static class DocValuesFieldExistsAdapter extends QueryToFilterAdapter<DocValuesFieldExistsQuery> { | |||
There was a problem hiding this comment.
At some point, keeping all the implementations as static inner classes is going to get unwieldy. Do you think we should refactor QueryToFilterAdapeter into its own package and make these static inners top level package private classes?
There was a problem hiding this comment.
Yeah.... I'll see about breaking them out in a mechanical follow up PR.
nik9000
added a commit
to nik9000/elasticsearch
that referenced
this pull request
May 12, 2021
This optimizes the `date_histogram` agg when there is a single bucket and no sub-aggregations. We expect this to happen from time to time when the buckets are larger than a day because folks often use "daily" indices. This was already fairly fast, but using the metadata makes it 10x faster. Something like 98ms becomes 7.5ms. Nice if you can get it! Like elastic#69377 this optimization will disable itself if you have document level security enabled or are querying a rollup index. Also like elastic#69377 it won't do anything if there is a top level query.
nik9000
added a commit
that referenced
this pull request
May 12, 2021
…2989) This optimizes the `date_histogram` agg when there is a single bucket and no sub-aggregations. We expect this to happen from time to time when the buckets are larger than a day because folks often use "daily" indices. This was already fairly fast, but using the metadata makes it 10x faster. Something like 98ms becomes 7.5ms. Nice if you can get it! Like #69377 this optimization will disable itself if you have document level security enabled or are querying a rollup index. Also like #69377 it won't do anything if there is a top level query.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This optimizes the
date_histogramagg when there is a single bucketand no sub-aggregations. We expect this to happen from time to time when
the buckets are larger than a day because folks often use "daily"
indices.
This was already fairly fast, but using the metadata makes it 10x
faster. Something like 98ms becomes 7.5ms. Nice if you can get it!
Like #69377 this optimization will disable itself if you have document
level security enabled or are querying a rollup index. Also like #69377
it won't do anything if there is a top level query.