New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(server): use mem_get_info to get kv cache size #664

Merged

OlivierDehaene merged 1 commit into main from feat/free_memory

Jul 20, 2023

Member

OlivierDehaene commented Jul 20, 2023 •

edited

Loading

Close #649
Close #651
Close #653
Close #636


          fix(server): use mem_get_info to get kv cache size

3db5d4a

OlivierDehaene mentioned this pull request

feat: add cuda memory fraction #659

Merged

OlivierDehaene merged commit bf94df3 into main

5 checks passed

OlivierDehaene deleted the feat/free_memory branch

July 20, 2023 15:23

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet