Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inference Qwen1.5-7B-Chat failed #10514

Closed
Fred-cell opened this issue Mar 23, 2024 · 2 comments
Closed

Inference Qwen1.5-7B-Chat failed #10514

Fred-cell opened this issue Mar 23, 2024 · 2 comments

Comments

@Fred-cell
Copy link

Bigdl-llm version:2.5.0b20240322, inference Qwen1.5-7B-Chat failed based generate.py
Traceback (most recent call last):
File "/home/intel/LLM/BigDL/python/llm/example/GPU/HF-Transformers-AutoModels/Model/qwen1.5/generate.py", line 45, in
model = AutoModelForCausalLM.from_pretrained(model_path,
File "/home/intel/anaconda3/envs/bigdl-llm/lib/python3.10/site-packages/bigdl/llm/transformers/model.py", line 320, in from_pretrained
model = cls.load_convert(q_k, optimize_model, *args, **kwargs)
File "/home/intel/anaconda3/envs/bigdl-llm/lib/python3.10/site-packages/bigdl/llm/transformers/model.py", line 434, in load_convert
model = cls.HF_Model.from_pretrained(*args, **kwargs)
File "/home/intel/anaconda3/envs/bigdl-llm/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 461, in from_pretrained
config, kwargs = AutoConfig.from_pretrained(
File "/home/intel/anaconda3/envs/bigdl-llm/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py", line 998, in from_pretrained
config_class = CONFIG_MAPPING[config_dict["model_type"]]
File "/home/intel/anaconda3/envs/bigdl-llm/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py", line 710, in getitem
raise KeyError(key)
KeyError: 'qwen2'

@qiuxin2012
Copy link
Contributor

@hkvision
Copy link
Contributor

hkvision commented Apr 2, 2024

Should work, closing this issue.

@hkvision hkvision closed this as completed Apr 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants