Unusual Model Utility Gap: Gradient Difference vs Ascent (Llama2)

[jeongjin0]

Hi, I noticed that for Llama2(forget10), gradient difference shows much lower model utility (~0.27) than gradient ascent (~0.63) in the leaderboard. This seems unusual since gradient difference is designed to maintain performance on the retain set while unlearning.

Interestingly, in Phi model results, gradient difference shows higher utility than ascent as expected. Could you help explain this significant performance gap in Llama2 implementation?

Thanks!