Empty args/inputs when turning streaming on and setting tool choice to any #658
Comments
|
@kwnath thanks for the report and the detailed example notebook. Can you share the python version you're using and the dependency versions you're using? e.g. with I can't reproduce the issue you're seeing, with this script from anthropic import Anthropic
client = Anthropic()
stream = client.messages.create(
model="claude-3-5-sonnet-20240620",
max_tokens=2048,
temperature=0.0,
tools=[
{
"name": "get_weather",
"description": "Get the current weather in a given location",
"input_schema": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA",
}
},
"required": ["location"],
},
}
],
tool_choice={"type": "any"},
stream=True,
messages=[{"role": "user", "content": "What's the weather like in San Francisco?"}],
)
for event in stream:
print(event)I get back multiple events |
|
Ah completely forgot to add this! Apologies! |
|
Thanks! I still can't reproduce with the dependencies you listed, I'm getting output like this: Have you tested using the SDK outside of a notebook? I wonder if that's causing issues? |
|
Hmm interesting this is only happening on the notebook. I ran this in a python shell and I get multiple chunks. |
|
Interesting, thanks. Could you share more details on how you have your notebook setup? |
|
Thanks @RobertCraigie I'll have a deeper look into how we're setting up our notebook but for now I'm happy to close this issue and update when I do find something in our setup! |
|
Thanks for confirming! Did you manage to determine what was causing issues in your setup? |
|
@kwnath did you manage to figure anything out? |
|
Hey Robert, I haven't yet, I had run a few tests around this and am still experiencing a few function calls without args. Will be investigating a bit more deeply today. |
|
@RobertCraigie I've done a bit more investigation just toggling on/off things in our app. I finally pin pointed it to a specific package we initialise. Sentry seems to be the culprit. I'll make a reproducible example in a bit but it seems like they've had an issue before: https://github.com/getsentry/sentry-python/releases/tag/2.10.0 Which links to this issue I'm currently using 2.13 which should have fixed it but I think there is something else at play. Edit: |
|
Okay here is how you can reproduce: https://gist.github.com/kwnath/b5e432e7492c0ba9dec7740344d8891e First try without initialising sentry, you'll notice the end output is there are tool args. Next initialise sentry and rerun the first example you'll notice there are no args. |
|
This is also problematic on the latest version of sentry: |
|
I've figured this out! I'm going to write up a small report a bit later! Making the fix here: getsentry/sentry-python#3615 I don't think there is much we can actually do here other than be wary about api changes in the future and use our own integrations. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Description
When calling the Anthropic client with
streaming=Trueandtool_choice={"type": "any"/"tool" }the output returns a tool call but with empty args.This is problematic for a few other reasons beyond no args being returned. For example, quite a few packages rely on the
anthropic-sdk, one of which islangchain-anthropic(ref).Expected response
I would expect that the output includes the inputs/args required for the tool call when streaming.
Reproduction steps
I've added a notebook to highlight some things:
https://gist.github.com/kwnath/f42737c023767d5effdcca20cb5bd0a6
The text was updated successfully, but these errors were encountered: