Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

IndexSortSortedNumericDocValuesRangeQuery can fail on sparse values. #61766

Closed
jtibshirani opened this issue Sep 1, 2020 · 2 comments
Closed

IndexSortSortedNumericDocValuesRangeQuery can fail on sparse values. #61766

jtibshirani opened this issue Sep 1, 2020 · 2 comments
Labels
blocker >bug :Search/Search Search-related issues that do not fall into other categories Team:Search Meta label for search team

Comments

@jtibshirani
Copy link
Contributor

jtibshirani commented Sep 1, 2020 •
edited
Loading

In #56657 we integrated a new Lucene query called IndexSortSortedNumericDocValuesRangeQuery that can speed up certain range queries. In the following scenario, this query can fail:

  • The index is sorted, and not all documents have a value for the index sort field.
  • We run a bool query containing these must clauses: a range query on the index sort field, plus another restrictive clause.

The root cause is a bug in IndexSortSortedNumericDocValuesRangeQuery where we don't always obey the DocIdSetIterator contract. In particular, even when advance returns NO_MORE_DOCS, the docID() method could return an old doc ID.

Since the bug is a violation of a lower-level Lucene contract, there may be other situations where the query fails other than the one above.
@jtibshirani jtibshirani added >bug :Search/Search Search-related issues that do not fall into other categories labels Sep 1, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-search (:Search/Search)

@elasticmachine elasticmachine added the Team:Search Meta label for search team label Sep 1, 2020
@jimczi jimczi mentioned this issue Sep 1, 2020
Merged
jimczi added a commit that referenced this issue Sep 1, 2020
@jimczi
Remove usage of IndexSortSortedNumericDocValuesRangeQuery (#61772)
13b9536 
This change reverts the optimization added in #56657.
We found a bug in `IndexSortSortedNumericDocValuesRangeQuery` that can fail the entire shard search request so this commit removes the optimization and restore the old behavior (range query on points) in this release branch.

Relates #61766
@jtibshirani
Copy link
Contributor Author

This was fixed in Lucene in https://issues.apache.org/jira/browse/LUCENE-9501, and incorporated in Elasticsearch in #62334.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
blocker >bug :Search/Search Search-related issues that do not fall into other categories Team:Search Meta label for search team
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jtibshirani @elasticmachine