Skip to content

Commit

Permalink
fix: token counter variable should be int not float
Browse files Browse the repository at this point in the history
Signed-off-by: Travis Johnson <[email protected]>
  • Loading branch information
tjohnson31415 committed Nov 18, 2024
1 parent 4f686d1 commit 781b874
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion vllm/engine/llm_engine.py
Original file line number Diff line number Diff line change
Expand Up @@ -1716,7 +1716,7 @@ def _get_stats(self,
# not counted (to avoid double counting)
actual_num_batched_tokens = scheduler_outputs.num_batched_tokens # type: ignore

num_generation_tokens_from_prefill_groups = 0.
num_generation_tokens_from_prefill_groups = 0
# NOTE: if scheduler_outputs.num_prefill_groups > 0 and
# the len of scheduler_outputs.scheduled_seq_groups is !=
# scheduler_outputs.num_prefill_groups, this means that
Expand Down

0 comments on commit 781b874

Please sign in to comment.