-
Notifications
You must be signed in to change notification settings - Fork 2.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Agent streaming - stream final output and steps #3398
Comments
🤖 Hi @david1542, it's nice to see you again! Hope you're doing well. Based on the context provided, it seems like you're trying to stream both the intermediate steps of your agent's actions and the final output in real time. For the first part of your question, the const stream = await chain.stream({
foo: "bears",
});
// Stream a diff as JSON patch operations
for await (const chunk of stream) {
console.log(chunk);
} You can find more details in the examples/src/prompts/json_structured_output_parser_streaming.ts file in the LangChainJS repository. For the second part of your question, you can use the const stream2 = await chain2.stream({
foo: "beets",
});
// Stream the entire aggregated JSON object
for await (const chunk of stream2) {
console.log(chunk);
} In both cases, the Regarding the I hope this helps! If you have any other questions or need further clarification, feel free to ask. Sources
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant. If you want to continue the conversation, start your reply with @dosu-bot. |
Yes, it seems like OpenAI tool calling agent streaming is broken using LCEL right now. Using |
Hey, apologies for missing this! Will have a look as soon as I can (hopefully within the next few days)! |
Thanks so much! |
I notice this as well that agent streaming does not seems to work. But streamLog works ok
|
Hey @david1542 sorry for taking so long on this, unfortunately you can't use the |
Closing for now, we are working on a new and easier to use API for streaming these internal agent steps. |
Hi everyone. I have 2 questions related to agents and streaming:
I'm trying to listen to my agent's actions in real time and notify the user about them in my UI. I noticed a new PR with agent streaming capabilities has been merged to main in last couple of days, that allows consumers to receive agent's actions in real time.
This is my agent's code:
And this is how I use it:
However, when I look at the logs, the only time I see the console.log statement is when the agent finishes and provides the final answer. The intermediate steps (tool callings) are not given to me. This is the only log statement I see:
I was wondering how can I achieve this kind of functionaliy. I know I can attach a callback manager to the
.call
and listen tohandleAgentAction
but I was wondering if there is a more neat way of doing that.Is there a way to stream the final output in chunks? Sometimes, the final output is long, and I'd like to show it to the user chunk by chunk, like using the native OpenAI SDK. I noticed there is a solution in the Python version. Is there a solution in the JS version as well?
The text was updated successfully, but these errors were encountered: