Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Searcher acquired on transport thread #53985

Closed
DaveCTurner opened this issue Mar 23, 2020 · 2 comments · Fixed by #54044
Closed

Searcher acquired on transport thread #53985

DaveCTurner opened this issue Mar 23, 2020 · 2 comments · Fixed by #54044
Assignees
@jimczi
Labels
blocker >bug :Distributed Indexing/Engine Anything around managing Lucene and the Translog in an open shard. v7.7.0

Comments

@DaveCTurner
Copy link
Contributor

Today SearchService#rewriteShardRequest runs on a transport thread (Names.SAME) and calls Engine#acquireSearcher. If the engine is a frozen engine then this may involve opening a directory reader, which does a fair amount of IO and therefore should not happen on a transport thread.
@DaveCTurner DaveCTurner added >bug :Distributed Indexing/Engine Anything around managing Lucene and the Translog in an open shard. labels Mar 23, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-distributed (:Distributed/Engine)

@ywelsch
Copy link
Contributor

ywelsch commented Mar 23, 2020

I think this is caused by the changes in #51708 where we now always rewrite the request, even in later phases. FrozenEngine has special handling for the case where the acquired searcher has source type "can_match", but the searcher acquired here is used for more than just rewriting.

@jimczi How should we fix this?

@jimczi jimczi mentioned this issue Mar 24, 2020
Merged
@DaveCTurner DaveCTurner mentioned this issue Mar 24, 2020
Open
jimczi added a commit to jimczi/elasticsearch that referenced this issue Mar 24, 2020
@jimczi
Avoid I/O operations when rewriting shard search request
cba588b 
This commit ensures that we rewrite the shard request with a short-lived can_match searcher.
This is required for frozen indices since the high level rewrite is now performed on a network thread
where we don't want to perform I/O.

Closes elastic#53985
jimczi added a commit that referenced this issue Mar 24, 2020
@jimczi
Avoid I/O operations when rewriting shard search request (#54044)
f301f49 
This commit ensures that we rewrite the shard request with a short-lived can_match searcher.
This is required for frozen indices since the high level rewrite is now performed on a network thread where we don't want to perform I/O.

Closes #53985
jimczi added a commit that referenced this issue Mar 25, 2020
@jimczi
Avoid I/O operations when rewriting shard search request (#54044) (#5…
3b4751b 
…4139)

This commit ensures that we rewrite the shard request with a short-lived can_match searcher.
This is required for frozen indices since the high level rewrite is now performed on a network thread where we don't want to perform I/O.

Closes #53985
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jimczi jimczi

Labels
blocker >bug :Distributed Indexing/Engine Anything around managing Lucene and the Translog in an open shard. v7.7.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Avoid I/O operations when rewriting shard search request jimczi/elasticsearch
4 participants
@ywelsch @DaveCTurner @elasticmachine @jimczi