Bradley-Terry model removes lm head while saving

[Arnav0400]

Hello and thanks for your work!

While running bradley-terry-rm/llama3_rm.py the final saved model does not have a lm head as the script is using a AutoModelForSequenceClassification model and not CausalLM. Because of this the lm head is initialized from scratch while loading the saved model. Is this correct or do you manually add the lm head weights?

[WeiXiongUST]

Maybe you can test the saved model by the useful_code/eval_reward_bench_bt.py to see whether the model is saved correctly.