Skip to content

Commit

Permalink
community[patch]: Fix tool_calls parsing when streaming from DeepIn…
Browse files Browse the repository at this point in the history
…fra (#26813)

- **Description:** This PR fixes the response parsing logic for
`ChatDeepInfra`, more specifially `_convert_delta_to_message_chunk()`,
which is invoked when streaming via `ChatDeepInfra`.
- **Issue:** Streaming from DeepInfra via `ChatDeepInfra` is currently
broken because the response parsing logic doesn't handle that
`tool_calls` can be `None`. (There is no GitHub issue for this problem
yet.)
- **Dependencies:** –
- **Twitter handle:** –

Keeping this here as a reminder:
> If no one reviews your PR within a few days, please @-mention one of
baskaryan, efriis, eyurtsev, ccurme, vbarda, hwchase17.
  • Loading branch information
thunze authored Sep 24, 2024
1 parent 997d95c commit 2b83c7c
Showing 1 changed file with 2 additions and 3 deletions.
5 changes: 2 additions & 3 deletions libs/community/langchain_community/chat_models/deepinfra.py
Original file line number Diff line number Diff line change
Expand Up @@ -144,13 +144,12 @@ def _convert_delta_to_message_chunk(
) -> BaseMessageChunk:
role = _dict.get("role")
content = _dict.get("content") or ""
tool_calls = _dict.get("tool_calls") or []

if role == "user" or default_class == HumanMessageChunk:
return HumanMessageChunk(content=content)
elif role == "assistant" or default_class == AIMessageChunk:
tool_calls = [
_parse_tool_calling(tool_call) for tool_call in _dict.get("tool_calls", [])
]
tool_calls = [_parse_tool_calling(tool_call) for tool_call in tool_calls]
return AIMessageChunk(content=content, tool_calls=tool_calls)
elif role == "system" or default_class == SystemMessageChunk:
return SystemMessageChunk(content=content)
Expand Down

0 comments on commit 2b83c7c

Please sign in to comment.