[Feature] Add efficient k-NN filtering support to Neural query

[SeyedAlirezaFatemi]

Is your feature request related to a problem?

Currently, there is no way to use efficient k-NN filtering with a Neural query (documentation)

What solution would you like?

Adding a filter field to the neural query which is pretty similar to the syntax of efficient k-NN filtering:

"neural": {
  "<vector_field>": {
    "query_text": "<query_text>",
    "query_image": "<image_binary>",
    "model_id": "<model_id>",
    "k": 100,

    "filter": {
      ...
    }

  }
}

[navneet1v]

@SeyedAlirezaFatemi the feature is already supported with neural query clause and works exactly you have defined in the description.

The documentation is not updated to say that it is supported,

cc: @jmazanec15 @vamshin

[SeyedAlirezaFatemi]

I think the Java Client for OpenSearch also doesn't support it.

[jmazanec15]

For filtering support, we do mention it here: https://opensearch.org/docs/latest/search-plugins/semantic-search/#step-4-search-the-index-using-neural-search. That being said, its not easy to find. We should have a section on the query itself and the parameters of the query.

[navneet1v]

I think the Java Client for OpenSearch also doesn't support it.

@SeyedAlirezaFatemi for which java client? high level java rest client or opensearch-java(low level client)?

[SeyedAlirezaFatemi]

@navneet1v opensearch-java. NeuralQuery

[navneet1v]

@SeyedAlirezaFatemi thanks for providing the details.

@VachaShah, @dblock: can we move this issue in opensearch-java repo. I don't have permission to transfer the issue.

cc: @vamshin , @jmazanec15

[rahulbhatia2702]

@navneet1v is prefiltering also available for neural sparse search ?

[navneet1v]

@navneet1v is prefiltering also available for neural sparse search ?

as per my understanding the ans is no. But @zane-neo can you please provide more details.

[heemin32]

Closing the issue. Please reopen the issue in java client repo.
For sparse vector search, it works in a manner similar to text search. Therefore, the concept of pre-filtering does not apply.