You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I'm interested in your work but I have run into problems reproducing your results. With the given code and training scripts (scripts/v1_5/pretrain.sh and scripts/v1_5/finetune.sh), I can only get a GQA accuracy of 63.08, which is far lower than 63.8 reported in the paper. Am I missing some important parts?
And by the way, the eval scripts cannot run successfully. I'm not sure where is the model path DenseConnector-v1.5-7B, so I use the checkpoints_stage2/DenseConnector-v1.5-7b-FineTuning path. Is there a problem of this operation?
The text was updated successfully, but these errors were encountered:
I'm interested in your work but I have run into problems reproducing your results. With the given code and training scripts (
scripts/v1_5/pretrain.shandscripts/v1_5/finetune.sh), I can only get a GQA accuracy of 63.08, which is far lower than 63.8 reported in the paper. Am I missing some important parts?And by the way, the eval scripts cannot run successfully. I'm not sure where is the model path
DenseConnector-v1.5-7B, so I use thecheckpoints_stage2/DenseConnector-v1.5-7b-FineTuningpath. Is there a problem of this operation?The text was updated successfully, but these errors were encountered: