Llama-3.2-11B-Vision-Instruct for Multimodal RAG Search

[pin-lpt]

On the model card, it states that the model can be used for Image-Text Retrieval.
"Image-Text Retrieval: Image-text retrieval is like a matchmaker for images and their descriptions. Similar to a search engine but one that understands both pictures and words."
I tried extracting last hidden state after loading it from MllamaVisionModel and MllamaTextModel with AutoProcessor, which produced shape of [1, 1, 4, 1601, 7680] and [1,5,4096] respectively.
Anyone tried to use Llama3.2 for similiar purposes? It seems vague from the documentation whether it can be used for this purpose and how.