Error happens when using SiglipVisionModel.forward with interpolate_pos_encoding=True

[kaitolucifer]

System Info

transformers==4.46.0.dev0

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

If the size of pixel_values is 384 x 384, below will throw an error.

vision_tower =  SiglipVisionModel.from_pretrained("google/siglip-so400m-patch14-384")
image_outputs = vision_tower(pixel_values, interpolate_pos_encoding=True)

Error:

embeddings = embeddings + self.interpolate_pos_encoding(embeddings, height, width)
                         ~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
TypeError: unsupported operand type(s) for +: 'Tensor' and 'Embedding'

Expected behavior

Shouldn't raise an error even though pos_encoding doesn't need to be interpolated when interpolate_pos_encoding=True

[ArthurZucker]

Hey! I think this is also related to #33965

[ArthurZucker]

Thanks for opening a PR seems right indeed! 🤗