Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Free up VRAM if not in use #1090

Closed
senscarlos opened this issue Dec 20, 2023 · 1 comment
Closed

Free up VRAM if not in use #1090

senscarlos opened this issue Dec 20, 2023 · 1 comment
Labels
duplicate This issue or pull request already exists

Comments

@senscarlos
Copy link

Would be great to have the option to unload the model(s) from VRAM if tabby is not being used for some time.

To allow other things to run instead. For example during part of the day I use Tabby and during another time ollama for a chat interface. There's no place (or reason) to have them both loaded at the same time.
@senscarlos senscarlos added the enhancement New feature or request label Dec 20, 2023
@wsxiaoys
Copy link
Member

Related discussion and a solution in #624

@wsxiaoys wsxiaoys added duplicate This issue or pull request already exists and removed enhancement New feature or request labels Dec 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
duplicate This issue or pull request already exists
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@wsxiaoys @senscarlos