Skip to content

Commit

Permalink
Address conflicts; Merge commit '0322693ec2183cbb340d570c475d67bd44f5…
Browse files Browse the repository at this point in the history
…7f19' into print-with-color
  • Loading branch information
xingyaoww committed Apr 2, 2024
2 parents d3ac882 + 0322693 commit 97bab51
Show file tree
Hide file tree
Showing 160 changed files with 2,828 additions and 41,124 deletions.
45 changes: 45 additions & 0 deletions .github/workflows/ghcr.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
name: Build and publish multi-arch container images

on:
push:
branches: [ main ]
workflow_dispatch:
inputs:
reason:
description: 'Why manual trigger?'
required: false
default: ''

jobs:
ghcr_build_and_push:
runs-on: ubuntu-latest
if: github.event_name == 'push'

steps:
- name: checkout
uses: actions/checkout@v4

- name: Set up QEMU
uses: docker/setup-qemu-action@v3

- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@v3

- name: Log-in to ghcr.io
run: echo "${{ secrets.GITHUB_TOKEN }}" | docker login ghcr.io -u ${{ github.actor }} --password-stdin

- name: Build and push multi-arch container images
run: |
# set env for fork repo
DOCKER_BUILD_ORG=$(echo "${{ github.repository }}" | tr '[A-Z]' '[a-z]' | cut -d '/' -f 1)
# Find directories containing Dockerfile but not containing .dockerfileignore
while IFS= read -r dockerfile_dir; do
# Check if .dockerfileignore exists in the directory
if [ ! -f "$dockerfile_dir/.dockerfileignore" ]; then
# Change directory and execute 'make all'
pushd "$dockerfile_dir" > /dev/null
make all DOCKER_BUILD_ORG=$DOCKER_BUILD_ORG
popd > /dev/null
fi
done < <(find . -type f -name Dockerfile -exec dirname {} \; | sort -u)
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -191,3 +191,4 @@ yarn-error.log*
# agent
.envrc
/workspace
/debug
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ For example, a PR title could be:

You may also check out previous PRs in the [PR list](https://github.com/OpenDevin/OpenDevin/pulls).

As described in [here](https://github.com/OpenDevin/OpenDevin/labels), we create several labels. Every PR should better tag with corresponding labels.
As described in [here](https://github.com/OpenDevin/OpenDevin/labels), we create several labels. Every PR should be tagged with the corresponding labels.

### 2. Pull Request description

Expand Down
Binary file removed OpenDevinLogo.jpg
Binary file not shown.
46 changes: 43 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<p align="center">
<img alt="OpenDevin Logo" src="./OpenDevinLogo.jpg" width="100" />
<img alt="OpenDevin Logo" src="./logo.png" width="150" />
</p>

# OpenDevin: Code Less, Make More
Expand All @@ -14,14 +14,24 @@ Welcome to OpenDevin, an open-source project aiming to replicate [Devin](https:/

## Work in Progress

OpenDevin is still a work in progress. But you can run the current app to see things working end-to-end.
OpenDevin is still a work in progress. But you can run the alpha version to see things working end-to-end.

### Requirements
* [Docker](https://docs.docker.com/engine/install/)
* [Python](https://www.python.org/downloads/) >= 3.10
* [NPM](https://docs.npmjs.com/downloading-and-installing-node-js-and-npm)
* [NodeJS](https://docs.npmjs.com/downloading-and-installing-node-js-and-npm) >= 14.8

### Installation
First, make sure Docker is running:
```bash
docker ps # this should exit successfully
```
Then pull our latest image [here](https://github.com/opendevin/OpenDevin/pkgs/container/sandbox)
```bash
docker pull ghcr.io/opendevin/sandbox:v0.1
```

Then start the backend:
```bash
export OPENAI_API_KEY="..."
export WORKSPACE_DIR="/path/to/your/project"
Expand All @@ -37,6 +47,34 @@ npm run start -- --port 3001

You'll see OpenDevin running at localhost:3001

### Picking a Model
We use LiteLLM, so you can run OpenDevin with any foundation model, including OpenAI, Claude, and Gemini.
LiteLLM has a [full list of providers](https://docs.litellm.ai/docs/providers).

To change the model, set the `LLM_MODEL` and `LLM_API_KEY` environment variables.

For example, to run Claude:
```bash
export LLM_API_KEY="your-api-key"
export LLM_MODEL="claude-3-opus-20240229"
```

You can also set the base URL for local/custom models:
```bash
export LLM_BASE_URL="https://localhost:3000"
```

And you can customize which embeddings are used for the vector database storage:
```bash
export LLM_EMBEDDING_MODEL="llama2" # can be "llama2", "openai", "azureopenai", or "local"
```

### Running on the Command Line
You can run OpenDevin from your command line:
```bash
PYTHONPATH=`pwd` python opendevin/main.py -d ./workspace/ -i 100 -t "Write a bash script that prints 'hello world'"
```

## 🤔 What is [Devin](https://www.cognition-labs.com/introducing-devin)?

Devin represents a cutting-edge autonomous agent designed to navigate the complexities of software engineering. It leverages a combination of tools such as a shell, code editor, and web browser, showcasing the untapped potential of LLMs in software development. Our goal is to explore and expand upon Devin's capabilities, identifying both its strengths and areas for improvement, to guide the progress of open code models.
Expand Down Expand Up @@ -88,3 +126,5 @@ We use Slack to discuss. Feel free to fill in the [form](https://forms.gle/758d5
Stay updated on OpenDevin's progress, share your ideas, and collaborate with fellow enthusiasts and experts. Together, we can make significant strides towards simplifying software engineering tasks and creating more efficient, powerful tools for developers everywhere.

🐚 **Code less, make more with OpenDevin.**

[![Star History Chart](https://api.star-history.com/svg?repos=OpenDevin/OpenDevin&type=Date)](https://star-history.com/#OpenDevin/OpenDevin&Date)
55 changes: 19 additions & 36 deletions agenthub/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,48 +6,31 @@ For example, `agenthub/langchain_agent`, `agenthub/metagpt_agent`, `agenthub/cod
Contributors from different backgrounds and interests can choose to contribute to any (or all!) of these directions.

## Constructing an Agent

The abstraction for an agent can be found [here](../opendevin/agent.py).

On a high-level, at each step, an agent takes in a [State](../opendevin/state.py) object and outputs an [Action](../opendevin/action).

Your agent must implement the following methods:

### `step`
```
def step(self, cmd_mgr: CommandManager) -> Event:
def step(self, state: "State") -> "Action"
```
`step` moves the agent forward one step towards its goal. This probably means
sending a prompt to the LLM, then parsing the response into an action `Event`.

Each Event has an `action` and a dict of `args`. Supported Events include:
* `read` - reads the contents of a file. Arguments:
* `path` - the path of the file to read
* `write` - writes the contents to a file. Arguments:
* `path` - the path of the file to write
* `contents` - the contents to write to the file
* `run` - runs a command. Arguments:
* `command` - the command to run
* `background` - if true, run the command in the background, so that other commands can be run concurrently. Useful for e.g. starting a server. You won't be able to see the logs. You don't need to end the command with `&`, just set this to true.
* `kill` - kills a background command
* `id` - the ID of the background command to kill
* `browse` - opens a web page. Arguments:
* `url` - the URL to open
* `recall` - recalls a past memory. Arguments:
* `query` - the query to search for
* `think` - make a plan, set a goal, or record your thoughts. Arguments:
* `thought` - the thought to record
* `finish` - if you're absolutely certain that you've completed your task and have tested your work, use the finish action to stop working.

For Events like `read` and `run`, a follow-up event will be added via `add_event` with the output.

### `add_event`
```
def add_event(self, event: Event) -> None:
```
`add_event` adds an event to the agent's history. This could be a user message,
an action taken by the agent, log output, file contents, or anything else.

You'll probably want to keep a history of events, and use them in your prompts
so that the agent knows what it did recently. You may also want to keep events
in a vector database so the agent can refer back to them.

The output of `step` will automatically be passed to this method.
sending a prompt to the LLM, then parsing the response into an `Action`.

We now have [two main categories of actions](../opendevin/action/base.py):
- `ExecutableAction`: will produces a corresponding `Observation` (source [here](../opendevin/observation.py)) for the agent to take the next `Action`.
- `NotExecutableAction`: will produces a `NullObservation` by the [controller](../opendevin/controller/__init__.py), which could means telling the agent to ignore this action.

For `ExecutableAction`, we currently have:
- `CmdRunAction` and `CmdKillAction` for bash command (see source [here](../opendevin/action/bash.py)).
- `FileReadAction` and `FileWriteAction` for file operations (see source [here](../opendevin/action/fileop.py)).
- `BrowseURLAction` to open a web page (see source [here](../opendevin/action/browse.py)).
- `AgentThinkAction`, `AgentFinishAction`: these are non-executable actions for agent to update its status to the user. For example, agent could use `AgentThink` to explain its though process to the user (see source [here](../opendevin/action/agent.py)).
- `AgentEchoAction`: the agent can produce some messages as its own Observation in the next `.step`, this will produces a `AgentMessageObservation` (see source [here](../opendevin/action/agent.py)).
- `AgentRecallAction`: recalls a past memory (see source [here](../opendevin/action/agent.py)).

### `search_memory`
```
Expand Down
2 changes: 1 addition & 1 deletion agenthub/codeact_agent/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

This folder implements the [CodeAct idea](https://arxiv.org/abs/2402.13463) that relies on LLM to autonomously perform actions in a Bash shell. It requires more from the LLM itself: LLM needs to be capable enough to do all the stuff autonomously, instead of stuck in an infinite loop.

A minimalistic exmaple can be found at [research/codeact/examples/run_flask_server_with_bash.py](./examples/run_flask_server_with_bash.py):
A minimalistic example can be found at [research/codeact/examples/run_flask_server_with_bash.py](./examples/run_flask_server_with_bash.py):

```bash
mkdir workspace
Expand Down
126 changes: 1 addition & 125 deletions agenthub/codeact_agent/__init__.py
Original file line number Diff line number Diff line change
@@ -1,128 +1,4 @@
import os
import re
from typing import List, Mapping

from opendevin.agent import Agent
from opendevin.state import State
from opendevin.action import (
Action,
CmdRunAction,
AgentEchoAction,
AgentFinishAction,
)
from opendevin.observation import (
CmdOutputObservation,
AgentMessageObservation,
)

from opendevin.llm.llm import LLM

assert (
"OPENAI_API_KEY" in os.environ
), "Please set the OPENAI_API_KEY environment variable."



SYSTEM_MESSAGE = """You are a helpful assistant. You will be provided access (as root) to a bash shell to complete user-provided tasks.
You will be able to execute commands in the bash shell, interact with the file system, install packages, and receive the output of your commands.
DO NOT provide code in ```triple backticks```. Instead, you should execute bash command on behalf of the user by wrapping them with <execute> and </execute>.
For example:
You can list the files in the current directory by executing the following command:
<execute>ls</execute>
You can also install packages using pip:
<execute> pip install numpy </execute>
You can also write a block of code to a file:
<execute>
echo "import math
print(math.pi)" > math.py
</execute>
When you are done, execute "exit" to close the shell and end the conversation.
"""

INVALID_INPUT_MESSAGE = (
"I don't understand your input. \n"
"If you want to execute command, please use <execute> YOUR_COMMAND_HERE </execute>.\n"
"If you already completed the task, please exit the shell by generating: <execute> exit </execute>."
)


def parse_response(response) -> str:
action = response.choices[0].message.content
if "<execute>" in action and "</execute>" not in action:
action += "</execute>"
return action


class CodeActAgent(Agent):
def __init__(
self,
llm: LLM,
) -> None:
"""
Initializes a new instance of the CodeActAgent class.
Parameters:
- instruction (str): The instruction for the agent to execute.
- max_steps (int): The maximum number of steps to run the agent.
"""
super().__init__(llm)
self.messages: List[Mapping[str, str]] = []
self.instruction: str = ""

def step(self, state: State) -> Action:
if len(self.messages) == 0:
assert self.instruction, "Expecting instruction to be set"
self.messages = [
{"role": "system", "content": SYSTEM_MESSAGE},
{"role": "user", "content": self.instruction},
]
updated_info = state.updated_info
if updated_info:
for prev_action, obs in updated_info:
assert isinstance(prev_action, (CmdRunAction, AgentEchoAction)), "Expecting CmdRunAction or AgentEchoAction for Action"
if isinstance(obs, AgentMessageObservation): # warning message from itself
self.messages.append({"role": "user", "content": obs.content})
elif isinstance(obs, CmdOutputObservation):
content = "OBSERVATION:\n" + obs.content
content += f"\n[Command {obs.command_id} finished with exit code {obs.exit_code}]]"
self.messages.append({"role": "user", "content": content})
else:
raise NotImplementedError(f"Unknown observation type: {obs.__class__}")
response = self.llm.completion(
messages=self.messages,
stop=["</execute>"],
temperature=0.0,
seed=42,
)
action_str: str = parse_response(response)
self.messages.append({"role": "assistant", "content": action_str})

command = re.search(r"<execute>(.*)</execute>", action_str, re.DOTALL)
if command is not None:
# a command was found
command_group = command.group(1)
if command_group.strip() == "exit":
return AgentFinishAction()
return CmdRunAction(command = command_group)
# # execute the code
# # TODO: does exit_code get loaded into Message?
# exit_code, observation = self.env.execute(command_group)
# self._history.append(Message(Role.ASSISTANT, observation))
else:
# we could provide a error message for the model to continue similar to
# https://github.com/xingyaoww/mint-bench/blob/main/mint/envs/general_env.py#L18-L23
# observation = INVALID_INPUT_MESSAGE
# self._history.append(Message(Role.ASSISTANT, observation))
return AgentEchoAction(content=INVALID_INPUT_MESSAGE) # warning message to itself


def search_memory(self, query: str) -> List[str]:
raise NotImplementedError("Implement this abstract method")

from .codeact_agent import CodeActAgent

Agent.register("CodeActAgent", CodeActAgent)
Loading

0 comments on commit 97bab51

Please sign in to comment.