Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

量化后的模型4bit，模型输出是乱码 #23

Closed
GUORUIWANG opened this issue Apr 1, 2023 · 3 comments
Closed

量化后的模型4bit，模型输出是乱码 #23

GUORUIWANG opened this issue Apr 1, 2023 · 3 comments

Comments

@GUORUIWANG
Copy link

用的llama_quant.py，未用数据集评估，量化后模型输出是乱码，请问啥原因呀
@Chuge0335
Copy link
Collaborator

Chuge0335 commented Apr 3, 2023 •
edited
Loading

用的llama_quant.py，未用数据集评估，量化后模型输出是乱码，请问啥原因呀

方便提供更多输出细节吗？

@GUORUIWANG
Copy link
Author

GUORUIWANG commented Apr 6, 2023 •
edited
Loading

用的llama_quant.py，未用数据集评估，量化后模型输出是乱码，请问啥原因呀

方便提供更多输出细节吗？

输入不一样，输出都是一样的乱码
image

@Chuge0335
Copy link
Collaborator

量化工具已经更新，效果请查看:https://github.com/Facico/Chinese-Vicuna/blob/master/tools/readme_zh.md

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@GUORUIWANG @Chuge0335