Introduce search context - point in time view of indices

[dnhatn]

Open a new search context:

POST /twitter/_search_context?keep_alive=5m

Execute a search with a specific search context

POST /_search
{
    "size": 100,
    "query": {
        "match" : {
            "title" : "elasticsearch"
        }
    },
    "search_context": {
       "id":  "46ToAwEHdHdpdHRlchZSa1dRZDFELVNLbTZkQVJudUtGMFFnAAAWNVAwS09JWTdTRUdibWE1ZC1id0tjQRRURnhPWjNFQmI3bmVOeVZReS1tRAAAAAAAAAAB",
       "keep_alive": "5m" 
    }
}

Close a search context when it's no longer needed

DELETE /_search_context
{
    "id" : "46ToAwEHdHdpdHRlchZSa1dRZDFELVNLbTZkQVJudUtGMFFnAAAWNVAwS09JWTdTRUdibWE1ZC1id0tjQRRURnhPWjNFQmI3bmVOeVZReS1tRAAAAAAAAAAB"
}

Notable works in this change:

• Move the search state to the coordinating node: Move states of search to coordinating node #52741
• Allow searches with a specific reader context: Allow searches with specific reader contexts #53989
• Adds the ability to acquire readers in IndexShard: Adds the ability to acquire readers in IndexShard #54966

Relates #46523
Relates #26472

Co-authored-by: Jim Ferenczi [email protected]

[dnhatn]

@mayya-sharipova I think we have addressed your comments. Would you please take another look?

[mayya-sharipova]

@dnhatn @jimczi Thanks for addressing my comments.

I was trying to test recent changes with alias filters, and I am getting the following error:

"can't read named writeable from StreamInput"

java.lang.UnsupportedOperationException: can't read named writeable from StreamInput
        at org.elasticsearch.common.io.stream.StreamInput.readNamedWriteable(StreamInput.java:1121) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
        at org.elasticsearch.common.io.stream.StreamInput.readOptionalNamedWriteable(StreamInput.java:1146) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
        at org.elasticsearch.search.internal.AliasFilter.<init>(AliasFilter.java:51) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
        at org.elasticsearch.common.io.stream.StreamInput.readMap(StreamInput.java:641) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
        at org.elasticsearch.action.search.SearchContextId.decode(SearchContextId.java:113) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
        at org.elasticsearch.action.search.TransportSearchAction.lambda$executeRequest$2(TransportSearchAction.java:221) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]

I think that error happens because ByteBufferStreamInput that is used in SearchContextId::decode is not NamedWriteableAwareStreamInput and can't read AliasFilter::new.

[dnhatn]

"can't read named writeable from StreamInput"

Thanks Mayya. 4a38f0d should address the issue.

[mayya-sharipova]

@dnhatn Thanks for the latest changes, open context search on alias filter and multiple indexes works well for me! I have also checked the latest code changes, and everything LGTM!

[jpountz]

This PR is very large so I focused on the high level and it makes sense to me. I wonder what is the motivation for marking it beta, this looks straightforward enough to me to not feel the need to mark it beta?

[jpountz]

Another thing is that the docs make it obvious that it works with _search but they don't make it obvious that it works with _async_search too.

[dnhatn]

Jim and I discussed this PR offline. We decided to take a different approach. Thanks, everyone!