Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support embedding models #458

Closed
SensorLock opened this issue Jul 13, 2023 · 2 comments
Closed

Support embedding models #458

SensorLock opened this issue Jul 13, 2023 · 2 comments

Comments

@SensorLock
Copy link

It would be nice if VLLM could serve Transformer-based embedding models (ex. BERT) as well.

Having one host server that supports generative and embedding LLM APIs makes deployment of applications involving vector indexing easier (document retrieval and memory insertion into prompts).

This may be related to #187 for BERT-derived models, since they are encoder-only.
@zhuohan123
Copy link
Member

Hi! Thanks for the suggestion. We actually had this discussion before and decided to postpone this feature. Please refer to #310 for more info.

@WoosukKwon WoosukKwon mentioned this issue Jan 31, 2024
Closed
30 tasks
@meng2468
Copy link

Potential to open it up again? Was included in the Q1 Roadmap but not completed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@meng2468 @zhuohan123 @SensorLock