Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Openai embedding fix to support jina-embeddings-v2 #4642

Merged
merged 11 commits into from
Nov 18, 2023
Merged

Openai embedding fix to support jina-embeddings-v2 #4642

Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
df8288c
update default max_new_tokens to 4096
wizd Nov 11, 2023
454fcf3
Merge pull request #4579 from oobabooga/dev
oobabooga Nov 13, 2023
2337aeb
Merge pull request #4606 from oobabooga/dev
oobabooga Nov 15, 2023
8a2af87
Merge pull request #4608 from oobabooga/dev
oobabooga Nov 15, 2023
0ee8d2b
Merge pull request #4627 from oobabooga/dev
oobabooga Nov 17, 2023
f889302
Merge pull request #4628 from oobabooga/dev
oobabooga Nov 17, 2023
3146124
Merge pull request #4632 from oobabooga/dev
oobabooga Nov 17, 2023
204d967
Merge branch 'oobabooga:main' into openai_embedding_fix
wizd Nov 17, 2023
828f6a7
add support to load embeding model from local path to device (like cuda)
wizd Nov 18, 2023
d3fb920
check model name before loading
wizd Nov 18, 2023
b3b5202
Small changes
oobabooga Nov 18, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion extensions/openai/embeddings.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import os

import numpy as np
from transformers import AutoModel

from extensions.openai.errors import ServiceUnavailableError
from extensions.openai.utils import debug_msg, float_list_to_base64
Expand Down Expand Up @@ -41,7 +42,12 @@ def load_embedding_model(model: str):
global embeddings_device, embeddings_model
try:
print(f"Try embedding model: {model} on {embeddings_device}")
embeddings_model = SentenceTransformer(model, device=embeddings_device)
if 'jina-embeddings' in model:
embeddings_model = AutoModel.from_pretrained(model, trust_remote_code=True) # trust_remote_code is needed to use the encode method
embeddings_model = embeddings_model.to(embeddings_device)
else:
embeddings_model = SentenceTransformer(model, device=embeddings_device)

print(f"Loaded embedding model: {model}")
except Exception as e:
embeddings_model = None
Expand Down