Skip to content

Commit

Permalink
LUCENE-9555 Ensure scorerIterator is fresh for opt
Browse files Browse the repository at this point in the history
Some collectors provide iterators that can efficiently skip
non-competitive docs. When using DefaultBulkScorer#score function
we create a conjunction of scorerIterator and collectorIterator.
As collectorIterator always starts from a docID = -1,
and for creation of conjunction iterator we need all of its
sub-iterators to be on the same doc, the creation of conjuction
iterator will fail if scorerIterator has already been advanced
to some other document.

This patch ensures that we create conjunction between scorerIterator
and collectorIterator only if scorerIterator has not been advanced yet.

Relates to apache#1725
Relates to apache#1937
  • Loading branch information
mayya-sharipova committed Oct 2, 2020
1 parent 2aa51fe commit 4d572cf
Show file tree
Hide file tree
Showing 2 changed files with 13 additions and 4 deletions.
3 changes: 3 additions & 0 deletions lucene/CHANGES.txt
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,9 @@ Improvements

Bug fixes

* LUCENE-9555: For sort optimization, create conjunction between scorerIterator
and collectorIterator only if scorerIterator has not been advanced yet. (Mayya Sharipova)

* LUCENE-8663: NRTCachingDirectory.slowFileExists may open a file while
it's inaccessible. (Dawid Weiss)

Expand Down
14 changes: 10 additions & 4 deletions lucene/core/src/java/org/apache/lucene/search/Weight.java
Original file line number Diff line number Diff line change
Expand Up @@ -203,10 +203,16 @@ public long cost() {
public int score(LeafCollector collector, Bits acceptDocs, int min, int max) throws IOException {
collector.setScorer(scorer);
DocIdSetIterator scorerIterator = twoPhase == null ? iterator : twoPhase.approximation();
DocIdSetIterator collectorIterator = collector.competitiveIterator();
// if possible filter scorerIterator to keep only competitive docs as defined by collector
DocIdSetIterator filteredIterator = collectorIterator == null ? scorerIterator :
ConjunctionDISI.intersectIterators(Arrays.asList(scorerIterator, collectorIterator));

DocIdSetIterator filteredIterator = scorerIterator;
if (scorerIterator.docID() == -1) {
DocIdSetIterator collectorIterator = collector.competitiveIterator();
if (collectorIterator != null) {
// filter scorerIterator to keep only competitive docs as defined by collector
filteredIterator = ConjunctionDISI.intersectIterators(Arrays.asList(scorerIterator, collectorIterator));
}
}

if (filteredIterator.docID() == -1 && min == 0 && max == DocIdSetIterator.NO_MORE_DOCS) {
scoreAll(collector, filteredIterator, twoPhase, acceptDocs);
return DocIdSetIterator.NO_MORE_DOCS;
Expand Down

0 comments on commit 4d572cf

Please sign in to comment.