Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

State: Add local_iteration attribute #2990

Merged
merged 3 commits into from
Jul 18, 2024
Merged

Conversation

li-boxuan
Copy link
Collaborator

We have iteration that tracks OpenDevin progress. When OpenDevin exceeds config.max_iterations, it pauses execution.

The introduction of agent delegation mechanism essentially makes OpenDevin a multi-agentic system. Without this PR, every agent instance keeps track of their own "local" iterations. What this means is with max_iterations = 10, OpenDevin could speak to LLM for 10 * 10 = 100 times in the worst case (root agent delegates 10 times). This defeats the purpose of config.max_iterations.

This PR fixes this problem by introducing local_iteration that had the same behavior as prior iteration. The attribute iteration now becomes a global/shared value per task.

Do we really need configs for both max iterations per task and max iterations per subtask? I don't think so. Thus, this PR doesn't introduce any new config.

Closes #2121

@tobitege
Copy link
Collaborator

For usability purposes: how does the UI behave one the limit is reached?
What options does the user have then?

@li-boxuan
Copy link
Collaborator Author

For usability purposes: how does the UI behave one the limit is reached?
What options does the user have then?

It's as bad as the status quo. No functional change. The user needs to click on continue to proceed one more step.

I know you tried and it didn't work well for Claude, presumably because Claude was too conservative and cautious.

@tobitege
Copy link
Collaborator

I know you tried and it didn't work well for Claude, presumably because Claude was too conservative and cautious.

Yeh, we need to come up with a PR to make this a more graceful "ending" of a chat session.
Like give options like resetting the counter, or start a new session with a summary of the current one or something similar.
If the use isn't actually done with what they wanted to achieve, it's a bit frustrating hanging there with one-step iterations.

Copy link
Collaborator

@tobitege tobitege left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haven't run it myself, but lgtm.
Thanks!

@li-boxuan
Copy link
Collaborator Author

Yeh, we need to come up with a PR to make this a more graceful "ending" of a chat session.
Like give options like resetting the counter, or start a new session with a summary of the current one or something similar.

I am in favor of this idea. I don't want to take it though.

@neubig
Copy link
Contributor

neubig commented Jul 18, 2024

Tests are failing due to #2992

@xingyaoww xingyaoww enabled auto-merge (squash) July 18, 2024 14:04
@xingyaoww xingyaoww merged commit 9d41314 into main Jul 18, 2024
2 checks passed
@xingyaoww xingyaoww deleted the state/iteration-counter branch July 18, 2024 14:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

New config: GLOBAL_MAX_ITERATIONS
4 participants