Support for Human Input Mode in AutoGen Studio #3484
Conversation
Update *WorkflowManager* classes: - Add async `a_send_message_function` parameter to mirror `send_message_function` param. - Add async `a_process_message` coroutine to mirror the synchronous `process_message` function. - Add async `a_run` coroutine to mirror the `run` function - Add async `_a_run_workflow` coroutine to mirror the synchronous `_run_workflow` function. Update *ExtendedConversableAgent* and *ExtendedGroupChatManager* classes: - Override the async `a_receive` coroutines Update *AutoGenChatManager*: - Add async `a_send` and `a_chat` coroutines to mirror their sync counterparts. - Accept the `WebSocketManager` instance as a parameter, so it can do Async comms directly. Update *app.py* - Provide the `WebSocketManager` instance to the *AutoGenChatManager* constructor - Await the manager's `a_chat` coroutine, rather than calling the synchronous `chat` function.
Updates to *ExtendedConversableAgent* and *ExtendedGroupChatManager* classes - override the `get_human_input` function and async `a_get_human_input` coroutine Updates to *WorkflowManager* classes: - add parameters `a_human_input_function` and `a_human_input_timeout` and pass along on to the ExtendedConversableAgent and ExtendedGroupChatManager - fix for invalid configuration passed from UI when human input mode is not NEVER and no model is attached Updates to *AutoGenChatManager* class: - add parameter `human_input_timeout` and pass it along to *WorkflowManager* classes - add async `a_prompt_for_input` coroutine that relies on `websocket_manager.get_input` coroutine (which snuck into last commit) Updates to *App.py* - global var HUMAN_INPUT_TIMEOUT_SECONDS = 180, we can replace this with a configurable value in the future
Enable human interaction in AutoGenStudio - Solution 2
@gagb |
There was a problem hiding this comment.
Thank you @victordibia and @SailorJoe6 -- fantastic PR and sorry for the delay. I was away for the weekend.
@victordibia's suggestion to manually checkout the icon.png fixed the issue I was getting with yarn build. Is there a more permanent fix for this -- it will affect anyone who uses Codespaces for development?
I was able to run the UI to create a simple ChatGPT where the user proxy is in human_input_mode=ALWAYS and it worked.
There are some small design decisions that I found confusing, in particular:
- The request for user input is posted as another message to the chat history in the UI .
- The spinning wheel that indicates agents are working continues to spin the exact same way while waiting for user input.
I think a different visual identity for both of these could reduce confusion. Choice 2 in particular made me think multiple times that I was waiting for system response. For choice 1, maybe we can change the color (if possible)?
|
I agree with the feedback @gagb , perhaps we can open and track in a separate repo. |
Sounds great! |
|
Thanks for considering my PR! Excellent feedback. Re: comment 1) I agree. My first attempt relied only on the Info toast-message, but I found I overlooked it too many times. Adding the prompt as the "default message" in the input textarea was probably enough, and it's super easy to just ignore it rather than post it as another message. In the end, I'm not a very good UI designer, and much better at implementing the ideas of better designers ;-). Anyway, I'll be following the conversation. AutoGenStudio is just awesome! Excellent work on this! |
|
@jackgerrits, can we please enable adding @SailorJoe6 as a reviewer to this project? :) |
Add support for human input mode implemented by @SailorJoe6 in PR #3476
The AutoGenStudio UI doesn't currently support user input modes 'ALWAYS', or 'TERMINATE' which are vital when allowing agents to generate and run code, so that the user can verify the code before it gets execute in their environment. This is likely due to difficulties encountered with Python's asyncio event loop getting blocked by synchronous calls to the AutoGen chat functions. This PR fixes those issues and adds support for all human input modes:
This code comes in multiple commits so you can follow the logical progression through:
Why are these changes needed?
Related issue number
Resolves #1358
Resolves #1664
Part of roadmap #737
Checks