Optimize the check when to do exact search vs ANN for filtered Search

[navneet1v]

Description

As part of implementing the Filters for Native Engine(Faiss), currently we are checking if filterIds < k we switch to exact search. The ask here is to improve this check to come up with a better check when we should do exact search vs ANN Search.

PR: #928

Code Ref: https://github.com/opensearch-project/k-NN/pull/928/files/b2bd95dbf92115a43f24773c39af42cade6083f6#diff-b0eb527e944c912b803db37ed9dd3dfa7689864b77d7146a0d45d7b73138d7b8R114

Solution

As per my understanding we should look at the ratio of total documents in the segment, filterids and K. We can add ef_search parameter here also.

Expectation

Come up with a way to improve this logic.

[navneet1v]

Closing this issue, as I have published more detailed feature request here: #1049