Skip to content

Commit

Permalink
[BUG] Convert OpenCLIP embeddings to numpy arrays (#3051)
Browse files Browse the repository at this point in the history
## Description of changes
OpenCLIP embeddings are failing type validation, as they return a
`Tensor`. Converting to NuPy arrays to fix.

## Test plan
- [ ] Tests pass locally with `pytest` for python, `yarn test` for js,
`cargo test` for rust

## Documentation Changes
To discuss
  • Loading branch information
itaismith authored Nov 4, 2024
1 parent 264e782 commit 154587c
Showing 1 changed file with 2 additions and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -59,13 +59,13 @@ def _encode_image(self, image: Image) -> Embedding:
self._preprocess(pil_image).unsqueeze(0)
)
image_features /= image_features.norm(dim=-1, keepdim=True)
return cast(Embedding, image_features.squeeze())
return cast(Embedding, image_features.squeeze().cpu().numpy())

def _encode_text(self, text: Document) -> Embedding:
with self._torch.no_grad():
text_features = self._model.encode_text(self._tokenizer(text))
text_features /= text_features.norm(dim=-1, keepdim=True)
return cast(Embedding, text_features.squeeze())
return cast(Embedding, text_features.squeeze().cpu().numpy())

def __call__(self, input: Union[Documents, Images]) -> Embeddings:
embeddings: Embeddings = []
Expand Down

0 comments on commit 154587c

Please sign in to comment.