Skip to content

Commit

Permalink
[Frontend] Avoid creating guided decoding LogitsProcessor unnecessari…
Browse files Browse the repository at this point in the history
…ly (vllm-project#9521)

Signed-off-by: Amit Garg <[email protected]>
  • Loading branch information
njhill authored and garg-amit committed Oct 28, 2024
1 parent c5a9c80 commit 17f1a79
Showing 1 changed file with 5 additions and 2 deletions.
7 changes: 5 additions & 2 deletions vllm/sampling_params.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,14 +49,17 @@ class GuidedDecodingParams:

@staticmethod
def from_optional(
json: Optional[Union[Dict, BaseModel, str]],
json: Optional[Union[Dict, BaseModel, str]] = None,
regex: Optional[str] = None,
choice: Optional[List[str]] = None,
grammar: Optional[str] = None,
json_object: Optional[bool] = None,
backend: Optional[str] = None,
whitespace_pattern: Optional[str] = None,
) -> "GuidedDecodingParams":
) -> Optional["GuidedDecodingParams"]:
if all(arg is None
for arg in (json, regex, choice, grammar, json_object)):
return None
# Extract json schemas from pydantic models
if isinstance(json, (BaseModel, type(BaseModel))):
json = json.model_json_schema()
Expand Down

0 comments on commit 17f1a79

Please sign in to comment.