Demo is broken

[Lyken17]

It seems the huggingface's demo is broken now

After cloning the local and running some simple tests, the issues should be related to the attention / tokenizer (I guess?)

File ~/anaconda3/envs/pth/lib/python3.9/site-packages/transformers/generation/utils.py:737, in GenerationMixin._update_model_kwargs_for_generation(self, outputs, model_kwargs, is_encoder_decoder, standardize_cache_format)
    735     if "attention_mask" in model_kwargs:
    736         attention_mask = model_kwargs["attention_mask"]
--> 737         model_kwargs["attention_mask"] = torch.cat(
    738             [attention_mask, attention_mask.new_ones((attention_mask.shape[0], 1))], dim=-1
    739         )
    740 else:
    741     # update decoder attention mask
    742     if "decoder_attention_mask" in model_kwargs:

RuntimeError: Tensors must have same number of dimensions: got 4 and 2

[Lyken17]

Problem located. THU-GLM has changed pos and attention mask in https://huggingface.co/THUDM/chatglm-6b/commit/373fd6b9d484841b490856a5570d6c450f20c22c

Thus, swiching from latest impl would solve the issue

# from modeling_chatglm import ChatGLMForConditionalGeneration
# from transformers import AutoTokenizer, GenerationConfig
# model = ChatGLMForConditionalGeneration.from_pretrained("THUDM/chatglm-6b").float()
# tokenizer = AutoTokenizer.from_pretrained("THUDM/chatglm-6b", trust_remote_code=True)

from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("THUDM/chatglm-6b", trust_remote_code=True)
model = AutoModel.from_pretrained("THUDM/chatglm-6b", trust_remote_code=True).float()

[ljsabc]

The upstream is keeping up with breaking changes, while I wrongly setup my huggingface space without pinning the upstream.
I'm thinking if it's better that we make everything from an Automodel such that everyone of us can eat our own dog food.

Will keep this issue updated.

[ljsabc]

Have updated the repo but not fully assured everything is working now. As the model weights are not changed at all I am like 90% confident. Will keep this issue open for a while as I am testing with the new environment.