[Bugfix] fix detokenizer shallow copy (vllm-project#5919)

Signed-off-by: Erkin Sagiroglu <[email protected]>
MErkinSag · Oct 26, 2024 · e853685 · e853685
1 parent 6b9d4ac
commit e853685
Showing 1 changed file with 1 addition and 1 deletion.
diff --git a/vllm/transformers_utils/detokenizer.py b/vllm/transformers_utils/detokenizer.py
@@ -90,7 +90,7 @@ def decode_prompt_logprobs_inplace(self, seq_group: SequenceGroup,
             prefix_offset = next_iter_prefix_offset
             read_offset = next_iter_read_offset
             if prev_tokens is None:
-                prev_tokens = next_iter_tokens
+                prev_tokens = next_iter_tokens.copy()
             else:
                 prev_tokens.extend(next_iter_tokens)