Skip to content

Commit

Permalink
[Bugfix] Benchmark serving script used global parameter 'args' in fun…
Browse files Browse the repository at this point in the history
…ction 'sample_random_requests' (#6428)
  • Loading branch information
lxline authored Jul 15, 2024
1 parent a754dc2 commit ccb20db
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion benchmarks/benchmark_serving.py
Original file line number Diff line number Diff line change
Expand Up @@ -203,7 +203,7 @@ def sample_random_requests(
)
offsets = np.random.randint(0, tokenizer.vocab_size, size=num_prompts)
input_requests = []
for i in range(args.num_prompts):
for i in range(num_prompts):
prompt = tokenizer.decode([(offsets[i] + i + j) % tokenizer.vocab_size
for j in range(input_lens[i])])
input_requests.append(
Expand Down

0 comments on commit ccb20db

Please sign in to comment.