From 1fbe5794dfec83da8350af4a9beb99c483a078aa Mon Sep 17 00:00:00 2001
From: stceum <50257864+stceum@users.noreply.github.com>
Date: Sat, 27 Jan 2024 15:37:33 +0800
Subject: [PATCH] Not a bug, just missing a space in README.md

---
 .../training/step2_reward_model_finetuning/README.md            | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/applications/DeepSpeed-Chat/training/step2_reward_model_finetuning/README.md b/applications/DeepSpeed-Chat/training/step2_reward_model_finetuning/README.md
index ede072a79..3c62b9f82 100644
--- a/applications/DeepSpeed-Chat/training/step2_reward_model_finetuning/README.md
+++ b/applications/DeepSpeed-Chat/training/step2_reward_model_finetuning/README.md
@@ -6,7 +6,7 @@ Finetuning the Reward Model (RM) is more or less similar to Step-1 Supervised  F
 
 For SFT finetuning, the data is the concatenation of a query and an answer. However, for RM finetuning, each batch of data consists of two query-answer pairs, i.e., the same query with a high-score answer and a low-score answer. This also leads to the second difference as describe below.
 
-👉**The training objective difference**
+👉 **The training objective difference**
 
 For RW, the training objective is the pairwise ranking score, i.e., for the two query-answer pairs, RM is supposed to give a higher score to the better answer. There are multiple ways to achieve this. In our implementation, we use either the end token of the sequence or the first padding token as the aggregated score and compare them. Others may also use the average score for the entire answer as an alternative.