[FEATURE] Support for 'hybrid' queries in java client

[ksingh17i]

Is your feature request related to a problem?
Yes, OpenSearch SearchSourceBuilder does not allow hybrid search queries (we are using both query and knn search fields together) throwing NamedObjectNotFoundException with the error message unknown field [hybrid]. We can run the DSL query on the console, but the query builder is unable to parse the same with the java client.

What solution would you like?
We would like to have hybrid query support with the Java client. Probably a HybridPlugin such as KNNPlugin to support the hybrid search queries.

What alternatives have you considered?
We are highly relying on the core library for text and semantic search. We are currently blocked because of missing hybrid support. Implementation of alternatives will be a ground-up work for us and will add additional complexity.

Do you have any additional context?
Query used for testing

{
  "query": {
    "hybrid": {
      "queries": [
        {
          "term": {
            "field_name_1": "Foo bar"
          }
        },
        {
          "knn": {
            "vector_field_name": {
              "vector":  [...],
              "k": 2
            }
          }
        }
      ]
    }
  },
  "size": 10
}

Exception:

org.opensearch.core.common.ParsingException: unknown query [hybrid] 
at org.opensearch.index.query.AbstractQueryBuilder.parseInnerQueryBuilder(AbstractQueryBuilder.java:338) ~[opensearch-2.11.1.jar:2.11.1] 
at org.opensearch.search.builder.SearchSourceBuilder.parseXContent(SearchSourceBuilder.java:1200) ~[opensearch-2.11.1.jar:2.11.1] 
at org.opensearch.search.builder.SearchSourceBuilder.parseXContent(SearchSourceBuilder.java:1130) ~[opensearch-2.11.1.jar:2.11.1] 
at com.a.b.c.Helper.parseRequest(Helper.java:61) [classes/:?] 
.
.
.
Caused by: org.opensearch.core.xcontent.NamedObjectNotFoundException: [3:15] unknown field [hybrid] 
at org.opensearch.core.xcontent.NamedXContentRegistry.parseNamedObject(NamedXContentRegistry.java:164) ~[opensearch-core-2.11.1.jar:2.11.1] 
at org.opensearch.core.xcontent.AbstractXContentParser.namedObject(AbstractXContentParser.java:429) ~[opensearch-core-2.11.1.jar:2.11.1] 
at org.opensearch.index.query.AbstractQueryBuilder.parseInnerQueryBuilder(AbstractQueryBuilder.java:330) ~[opensearch-2.11.1.jar:2.11.1]

[navneet1v]

Moved issue to neural repo as hybrid query clause is provided by neural plugin

[ksingh17i]

Update: we are currently using opensearch-rest-high-level-client version 2.11.0 and opensearch-knn plugin version 2.11.0.0

[navneet1v]

@ksingh17i the opensearch-rest-high-level-client is marked for deprecation and will be removed in 3.0 version of Opensearch. Hence the query clause will not be added in opensearch-rest-high-level-client.

Ref: https://opensearch.org/docs/latest/clients/java-rest-high-level/

The only place where we can add hybrid query clause now is https://github.com/opensearch-project/opensearch-java .

[ksingh17i]

Any idea by when can we expect hybrid query support in https://github.com/opensearch-project/opensearch-java

[ksingh17i]

@navneet1v any update on this?

[vibrantvarun]

@ksingh17i we have merged the PR for the support of hybrid query in java client.

[navneet1v]

@ksingh17i the release issue on the Opensearch java client is also created: opensearch-project/opensearch-java#868 More details on the release will be available there.

[navneet1v]

@ksingh17i the java client has been released with the hybrid query. As this issue is resolved: opensearch-project/opensearch-java#868

Artifacts can be found here: https://repo1.maven.org/maven2/org/opensearch/client/opensearch-java/2.9.0/

Resolving the issue.