Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Highlighters should reject ignored keyword field early #43800

Closed
jimczi opened this issue Jul 1, 2019 · 2 comments · Fixed by #53408
Closed

Highlighters should reject ignored keyword field early #43800

jimczi opened this issue Jul 1, 2019 · 2 comments · Fixed by #53408
Assignees
Labels
>enhancement :Search Relevance/Highlighting How a query matched a document Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch

Comments

@jimczi
Copy link
Contributor

jimczi commented Jul 1, 2019

Highlighters are able to handle text and keyword field by reanalyzing their content at search time. However they don't check if the field was ignored at search time due to the ignore_above option. For instance a keyword field that is greater than the ignore_above option would skip indexing but if it is set in an highlighter option, the content will be extracted from _source and reanalyzed (ignoring ignore_above option). We should check the ignore_above option at highlighting time in order to avoid hitting the highlight.max_analyzed_offset on a field that was ignored at index time.

@jimczi jimczi added >enhancement :Search Relevance/Highlighting How a query matched a document labels Jul 1, 2019
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-search

@jimczi jimczi added the help wanted adoptme label Dec 13, 2019
@mayya-sharipova
Copy link
Contributor

Related proposal: #52155 (comment) to introduce ignore_above parameter in a highlighter.
That proposal doesn't replace the current issue.

@mayya-sharipova mayya-sharipova removed the help wanted adoptme label Mar 9, 2020
@mayya-sharipova mayya-sharipova self-assigned this Mar 9, 2020
mayya-sharipova added a commit to mayya-sharipova/elasticsearch that referenced this issue Mar 11, 2020
Keyword field values with length more than ignore_above are not
indexed. But highlighters still were retrieving these values
from _source and were trying to highlight them. This sometimes lead to
errors if a field length exceeded  max_analyzed_offset. But also this
is a wrong behaviour to attempt to highlight something that was not
ignored during indexing.

This PR checks if a keyword value was ignored because of its length,
and if yes, skips highlighting it.

Closes elastic#43800
mayya-sharipova added a commit that referenced this issue Mar 16, 2020
Keyword field values with length more than ignore_above are not
indexed. But highlighters still were retrieving these values
from _source and were trying to highlight them. This sometimes lead to
errors if a field length exceeded  max_analyzed_offset. But also this
is a wrong behaviour to attempt to highlight something that was not
ignored during indexing.

This PR checks if a keyword value was ignored because of its length,
and if yes, skips highlighting it.

Closes #43800
mayya-sharipova added a commit to mayya-sharipova/elasticsearch that referenced this issue Mar 16, 2020
Keyword field values with length more than ignore_above are not
indexed. But highlighters still were retrieving these values
from _source and were trying to highlight them. This sometimes lead to
errors if a field length exceeded  max_analyzed_offset. But also this
is an overall wrong behaviour to attempt to highlight something that was
ignored during indexing.

This PR checks if a keyword value was ignored because of its length,
and if yes, skips highlighting it.

Closes elastic#43800
mayya-sharipova added a commit that referenced this issue Mar 16, 2020
Keyword field values with length more than ignore_above are not
indexed. But highlighters still were retrieving these values
from _source and were trying to highlight them. This sometimes lead to
errors if a field length exceeded  max_analyzed_offset. But also this
is an overall wrong behaviour to attempt to highlight something that was
ignored during indexing.

This PR checks if a keyword value was ignored because of its length,
and if yes, skips highlighting it.

Backport: #53408
Closes #43800
@javanna javanna added the Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch label Jul 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>enhancement :Search Relevance/Highlighting How a query matched a document Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants