Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Defensively copy
sampling_params
(vllm-project#2881)
If the SamplingParams object passed to LLMEngine.add_request() is mutated after it returns, it could affect the async sampling process for that request. Suggested by @Yard1 vllm-project#2514 (comment)
- Loading branch information