[FEATURE] Is possible to add new KNNs using microservices? That broadens the KNNs that can be used as backends

[ksrinivs64]

Is your feature request related to a problem?
I'd like to be able to call out to a KNN written in Python not C (PyNNDescent). It has support for computing Hamming distances over integer vectors, something that most other KNNs do not support. However, trying to POST from the library following code for NMS and FAISS seems to run into security problems even when URL permissions are given to the plugin to POST. The POST never happens, nothing happens in the logs, and the system behaves like it has no results.

What solution would you like?
A way to call out to a vector database using HTTP.

What alternatives have you considered?
We tried launching a Python interpreter through Jep but that failed due to threading issues.

Do you have any additional context?
Add any other context or screenshots about the feature request here.

[navneet1v]

Hi @ksrinivs64,
Thanks for creating the feature request. We have been thinking about separating the core K-NN engines via pluggable interfaces. Using those interfaces users can integrate their custom code to call any library or endpoint as per their use case.

But right now we don't have a github issue around this. But I will try to create one and link the issue here. But having said that, it will take some time to develop that feature, as this will be a massive undertaking.

cc: @vamshin , @jmazanec15 , @heemin32

[ksrinivs64]

Thanks that would be helpful.

…

On Mon, Dec 4, 2023, 9:37 PM Navneet Verma ***@***.***> wrote: Hi @ksrinivs64 <https://github.com/ksrinivs64>, Thanks for creating the feature request. We have been thinking about separating the core K-NN engines via pluggable interfaces. Using those interfaces users can integrate their custom code to call any library or endpoint as per their use case. But right now we don't have a github issue around this. But I will try to create one and link the issue here. But having said that, it will take some time to develop that feature, as this will be a massive undertaking. — Reply to this email directly, view it on GitHub <#1328 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACNS6QXP6ROG2G5KZ3TABHLYH2CHDAVCNFSM6AAAAAA76GNK72VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMZZHEYDAMRVGQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>