-
Notifications
You must be signed in to change notification settings - Fork 116
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update compression config for openlm-research/open_llama_3b_v2 #860
Update compression config for openlm-research/open_llama_3b_v2 #860
Conversation
@KodiaqQ please take a look |
@nikita-savelyevv, you've provided numbers in the description with the ASYM mode, while configuration contains |
@KodiaqQ Thanks for noticing. There were some errors in the collected metrics. I've updated the metrics in the PR description and changed the mode to asymmetric. |
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
* Remove compression with all_layers=True for openlm-research/open_llama_3b_v2 * Fix sym parameter * Add AWQ
Copy changes from huggingface/optimum-intel#860
Copy changes from huggingface/optimum-intel#860
Reduce PPL for compressed
openlm-research/open_llama_3b_v2
by removingall_layers=True
and adding AWQ.Before submitting