Add editable URL to browser tab #3078
Conversation
Add editable URL input field to Browser component * Add an input field in `frontend/src/components/Browser.tsx` to allow editing the URL. * Add `handleUrlChange` and `handleUrlBlur` handlers to update the URL state. * Implement `updateBrowserTabUrl` function in `frontend/src/services/browseService.ts` to send data using websocket for updating the browser tab URL. * Modify `frontend/src/state/browserSlice.ts` to include a new action to update the URL state and call `updateBrowserTabUrl` function. --- For more details, open the [Copilot Workspace session](https://copilot-workspace.githubnext.com/OpenDevin/OpenDevin?shareId=b39056db-d2ea-402e-aabd-c77505aa2575).
There was a problem hiding this comment.
What is the motivation behind this though? The browser supposedly to be controlled by the agent - and that tab is used to display the agent's action to the user. It is probably not really important for human to able to change the URL - we can't interact with the displayed screenshot anyway?
|
Like LLM Model: groq/llama3-8b-8192 Why it creates
If I ask
|
|
sounds good! Similar to interactive terminal, would the agent also knows that the user changed the URL of the browser? (e.g., you push an |
Yes.
Log:
LLM Msg:
|
| @@ -121,6 +123,8 @@ def action_to_str(self, action: Action) -> str: | |||
| return f'{action.thought}\n<execute_browse>\n{action.inputs["task"]}\n</execute_browse>' | |||
| elif isinstance(action, MessageAction): | |||
| return action.content | |||
| elif isinstance(action, BrowseURLAction): | |||
| return f'Open {action.url} in browser' | |||
There was a problem hiding this comment.
This is only added to CodeAct, but I guess most complex browsing actions are actually performed by browsing agents I'm not sure if there's an easy way to do that for browsing agents as well.
Also, the message added here is just plain string (and for agent, it will recognize it as its past action), and got the BrowserOutputObservation as output for such plain string action. This will most likely confuses the agent - as it may try to issue action like "Open {action.url} in browser" and only find it will not be able to get observations.
Making the URL editable will add a lot of things to consider on the agent side, so I'd advise we be careful with it & come up with a more general solution. If we can't do that in a general way, it will be cleaner for us not to implement it rather than open ourselves to tech debt in the future.
There was a problem hiding this comment.
After reworded Opening {action.url} in browser manually
There was a problem hiding this comment.
I think both the action and the observation here are sent to the LLM with role 'user'?
There was a problem hiding this comment.
@xingyaoww @SmartManoj
can you guide me how to use browse ??
i tried to click but it is not allow me to edit URL .
|
Like this, it will solve #3034 |
|
We're doing some cleanup of PRs, and are going to close this one for now. |
Solves Discord thread
Add functionality to make the browser tab URL editable.
onChangehandler to update the URL state.onKeyDownhandler to trigger the URL update.updateBrowserTabUrlfromfrontend/src/services/browseService.ts.For more details, open the Copilot Workspace session.