Add Jina Embedding model #52
Conversation
| self._model = None | ||
|
|
||
| def load(self): | ||
| self._model = AutoModel.from_pretrained( |
There was a problem hiding this comment.
bonus, you can use hf_cache in the config with these settings so the cold start is fire out of the box.
| environment_variables: {} | ||
| external_package_dirs: [] | ||
| model_metadata: | ||
| example_model_input: {"text": ["I want to eat pasta", "I want to eat pizza"]} |
There was a problem hiding this comment.
maybe add the max length to the sample so it's obvious that users can edit it.
| environment_variables: {} | ||
| external_package_dirs: [] | ||
| hf_cache: | ||
| - repo_id: jinaai/jina-embeddings-v2-base-en |
There was a problem hiding this comment.
you probably wanna pin the revision here just like you're doing in the load.
There was a problem hiding this comment.
Ah, makes sense, do you know how to do that? Would it just be revision: abcd1234?
There was a problem hiding this comment.
|
@bolasim I was having trouble getting the The model is a quarter of a gig, so the download actually only takes about a second (log line below): The whole model.load() is 8.5 seconds which is not great but not terrible. And the model is running faster now with the new revision, I'm getting 40-45 seconds for my shakespeare dataset. |
|
Whatever you think is best! |
Add a Truss for Jina embedding model, intending to add to model library today + publish a blog post on this