Change how TopHitsAggregator collects top docs #26753
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
martijnvg
added
:Analytics/Aggregations
martijnvg
force-pushed
the
top_hits_fix_initializing_collectors
branch
from
d0ae19a to
176c76f
Compare
Both TopDocsCollector and LeafCollector were being kept around at the aggregator level. In case the nested aggregator would do a post collection then this could cause pushing down docids to top hits child aggregators that already moved the next LeafCollector (causing assertions to trip and incorrect results). By keeping track of the LeafCollector in a seperate map at the leaf bucket level this problem can simply not happen any more as the place holding LeafCollector is no longer shared. Also LeafCollector instances for TopDocsCollectors are no longer pre-created as the beginning a new segment is evaluated. There is no guarantee that TopHitsAggregator encounters a document for a particular bucket and there has to be logic to create LeafCollector instances which have not been seen before. Closes elastic#26738
martijnvg
force-pushed
the
top_hits_fix_initializing_collectors
branch
from
176c76f to
a056c5d
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Both
TopDocsCollectorandLeafCollectorwere being kept around at the aggregator level. In case thenestedaggregator would do a post collection then this could cause pushing down docids to top hits child aggregators that already moved the next LeafCollector (causing assertions to trip and incorrect results).By keeping track of the
LeafCollectorin a seperate map at the leaf bucket level this problem can simply not happen any more as the place holdingLeafCollectoris no longer shared.Also
LeafCollectorinstances forTopDocsCollectorsare no longer pre-created as the beginning a new segment is evaluated. There is no guarantee thatTopHitsAggregatorencounters a document for a particular bucket and there has to be logic to createLeafCollectorinstances which have not been seen before.This PR fixes an unreleased bug and therefor this PR has been labelled as non-issue.
PR for #26738