Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

T5 model support #404

Closed
havietisov opened this issue Jul 8, 2023 · 4 comments
Closed

T5 model support #404

havietisov opened this issue Jul 8, 2023 · 4 comments
Labels
duplicate This issue or pull request already exists new model Requests to new models

Comments

@havietisov
Copy link

Any plans on adding T5 models?
FLAN-T5 are particularly interesting
@WoosukKwon WoosukKwon added the new model Requests to new models label Jul 8, 2023
@zhuohan123 zhuohan123 mentioned this issue Jul 18, 2023
Closed
76 tasks
@shermansiu
Copy link

Yes. See #187. It's also on the dev roadmap (#244).

@nlpkiddo-2001
Copy link

Any updates reg this?

@shermansiu
Copy link

In #187 (comment), js8544 mentioned that his team would be working on it in November, but no updates have come since then.

@zhuohan123 zhuohan123 mentioned this issue Jan 31, 2024
Closed
30 tasks
@hmellor hmellor added the duplicate This issue or pull request already exists label Mar 6, 2024
@hmellor
Copy link
Collaborator

hmellor commented Mar 6, 2024

Closing as duplicate of #187

@hmellor hmellor closed this as completed Mar 6, 2024
jikunshang pushed a commit to jikunshang/vllm that referenced this issue Oct 24, 2024
@kzawora-intel
Add forward_hpu to RotaryEmbedding, remove custom module (vllm-projec…
8c43ff1 
…t#404)

This PR removes the usage of custom HPU RotaryEmbedding modules, and
adds a forward_hpu method to existing RotaryEmbedding, for reusing
multiple derived implementations without the need of adding them to HPU
extension.
Mark_steps should not be needed within the test, but for whatever
reason, if they are not there, PT bridge crashes. To be investigated
later on. It does not affect actual model execution in any way I could
test/observe.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
duplicate This issue or pull request already exists new model Requests to new models
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@havietisov @shermansiu @hmellor @WoosukKwon @nlpkiddo-2001