Skip to content

Latest commit

 

History

History
14 lines (6 loc) · 406 Bytes

File metadata and controls

14 lines (6 loc) · 406 Bytes

TRL_chat_mode_FT_LLAMA3

Support Deepspeed to reduce RAM

image

image

how to run:

accelerate launch --config_file=dp_z2.yaml --gradient_accumulation_steps 4 trl_ft_chatmode.py