Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error using assign method with IndexBinary in FAISS #3343

Closed
landEpita opened this issue Apr 4, 2024 · 1 comment
Closed

Error using assign method with IndexBinary in FAISS #3343

landEpita opened this issue Apr 4, 2024 · 1 comment

Comments

@landEpita
Copy link

Description:

I am encountering an issue when attempting to use the assign method on an IndexBinary object in FAISS. The objective is to assign each input vector to the label of its nearest neighbor within the index. However, when I call the assign method, I receive a TypeError indicating a mismatch in the number or type of arguments.

Steps to Reproduce:

  1. Import FAISS and other necessary libraries.
  2. Generate random floating-point vectors and quantize them using sentence_transformers.quantization.quantize_embeddings.
  3. Create an IndexBinary object and set its parameters.
  4. Attempt to train the index with quantized vectors.
  5. Initialize an empty numpy array for nearest neighbor labels.
  6. Call the assign method with the quantized vectors and the labels array.

Expected Behavior:

The assign method should correctly assign each input vector to the label of its nearest neighbor within the index without any errors.

Actual Behavior:

When calling the assign method, the following TypeError is thrown:

TypeError: Wrong number or type of arguments for overloaded function 'IndexBinary_assign'.
Possible C/C++ prototypes are:
    faiss::IndexBinary::assign(faiss::idx_t,uint8_t const *,faiss::idx_t *,faiss::idx_t) const
    faiss::IndexBinary::assign(faiss::idx_t,uint8_t const *,faiss::idx_t *) const

Environment:

  • FAISS version: 1.8.0
  • Python version: 3.11.6

Additional Context:

I am using FAISS for vector similarity search in a binary space. The vectors are first quantized to a binary format before being used with the IndexBinary object. Below is a snippet of the code that leads to the error:

import faiss
from sentence_transformers.quantization import quantize_embeddings
import numpy as np

d = 1024                          
nb = 100000  
k=1

xb = np.random.random((nb, d)).astype('float32')
vectors = quantize_embeddings(xb, "ubinary")
count = len(vectors)

index = faiss.index_binary_factory(d, "BIVF64")
index.nprobe = 4
index.train(vectors)

list_nos = np.empty((count, k), dtype=np.int64)

index.assign(n=count, x=vectors, labels=list_nos, k=k)

I have ensured that the quantized vectors are of type uint8 as required. Any insights into what might be causing this error or how to resolve it would be greatly appreciated.

@mdouze
Copy link
Contributor

mdouze commented Apr 5, 2024

Right, assign is not wrapped properly here
https://github.com/facebookresearch/faiss/blob/main/faiss/python/class_wrappers.py#L835
marking this as an enhancement.
In the meantime you can just use index.search(x, 1) to get the assignment.

asadoughi pushed a commit that referenced this issue Apr 24, 2024
Summary: Fixes #3343

Differential Revision: D56526842
asadoughi pushed a commit that referenced this issue Apr 24, 2024
Summary: Fixes #3343

Reviewed By: kuarora, junjieqi

Differential Revision: D56526842
asadoughi pushed a commit that referenced this issue Apr 24, 2024
Summary: Fixes #3343

Reviewed By: kuarora, junjieqi

Differential Revision: D56526842
abhinavdangeti pushed a commit to blevesearch/faiss that referenced this issue Jul 12, 2024
Summary: Fixes facebookresearch#3343

Reviewed By: kuarora, junjieqi

Differential Revision: D56526842

fbshipit-source-id: b7c4377495db4e68283cf4ce2b7c8fae008cd404
aalekhpatel07 pushed a commit to aalekhpatel07/faiss that referenced this issue Oct 17, 2024
Summary: Fixes facebookresearch#3343

Reviewed By: kuarora, junjieqi

Differential Revision: D56526842

fbshipit-source-id: b7c4377495db4e68283cf4ce2b7c8fae008cd404
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants