[Feature] Adding support for Mixtral and Gemma models

[martinakaduc]

This pull request intends to add support for Mixtral and Gemma as LLM backbone.

[muhark]

@martinakaduc I see some of my comments in here--did you figure out the necessary changes to the preprocess_gemma function? :)
FWIW yours looks right, I realize that I may have been running an old version of the chat template that didn't include the BOS token when I was developing.

[martinakaduc]

preprocess_gemma

@muhark Yes, I have referenced your code. I also figured out the problem with preprocess_gemma where <end_of_turn>\n requires 2 tokens.

I have successfully trained my llava_gemma for Vietnamese! 😄 (https://huggingface.co/ura-hcmut/GemSUraV-7B)

[ShawnAn-WHU]

@martinakaduc Sorry to bother. I see some warnings when pretrain Mixtral 8*7B, just like #1417. Did you face the same problem? Thanks in advance!

[Jayantverma2]

why did you use
"additional_special_tokens": [
"<start_of_turn>",
"<end_of_turn>"
],
instead of
"bos_token": "",
"eos_token": "",