Skip to content
This repository has been archived by the owner on Oct 11, 2024. It is now read-only.

Commit

Permalink
Pass tokenizer_revision when getting tokenizer in openai serving (v…
Browse files Browse the repository at this point in the history 
…llm-project#4214)
  • Loading branch information
@chiragjn @robertgshaw2-neuralmagic
chiragjn authored and robertgshaw2-neuralmagic committed Apr 21, 2024
1 parent 6433ad7 commit 0f098a0
Showing 1 changed file with 1 addition and 0 deletions.
1 change: 1 addition & 0 deletions vllm/entrypoints/openai/serving_engine.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,7 @@ async def _post_init(self):
self.tokenizer = get_tokenizer(
engine_model_config.tokenizer,
tokenizer_mode=engine_model_config.tokenizer_mode,
tokenizer_revision=engine_model_config.tokenizer_revision,
trust_remote_code=engine_model_config.trust_remote_code,
truncation_side="left")

Expand Down

0 comments on commit 0f098a0

Please sign in to comment.