refactor: testing/experimenting with single agent without workflow

[tomaszfelczyk]

For testing/researching purposes it would be nice to be able to test single agent behaviour based on single task (state). e.g.:

const state = basicTask(`Find me...`)
agent.run(state)

1. question of whether the run should take something more than just state. (ps.: by the way state is a very generic name, every time I read the code I replaced it with task for myself).
2. Following the famous principle "Functions should do one thing", question if run function should inference model AND reduce state or reducing state should be done outside of it?

[grabbou]

Regarding your questions, there's been some changes on main, that are not released to npm yet, so depending on whether you checked GitHub or worked with local version, the answer to this might be different.

Right now, run takes 4 arguments.

• Provider - the LLM provider to use for calls. This is workflow-default, or custom per-agent provided.
• State - the state of workflow at a current level. This is your work, potential child states (if you delegated something), and messages from your conversation.
• Context - all messages from the parent and their parent, up to the root. This is "work done so far"
• Workflow - the workflow, in case you want some additional metadata (e.g. check available agents for delegation)

With that in mind, I am not sure how to read your first question, so I will check for a follow-up clarification!

Regarding second question, the idea behind agents reducing state was to make it easier / automatic to perform delegations/handoffs from within the agent. It was historically outside of run, but it made things more complex.

Now, if you want to "delegate", simply add new child state. When that agent is done, your run method will run again, and their messages will be appended to the state.messages, allowing you to continue. If you want to "hand-off" to a different agent, simply return entirely different state.

Regarding your original question, I agree there's no way right now to run single agent, something that I will inevitably need in Cali, where I am performing a bunch of requests on a single agent - so yes, I agree this is good idea.

I guess easiest way to do it today, albeit cumbersome, would be to write the following:

teamwork(workflow, childState({ agent, messages: [user("What is weather in SF?"] })

I think we should create a new top-level entry point similar to teamwork, that works on a single task / agent.

[tomaszfelczyk]

Sticking to state reducing.
Looking at old Redux architecture which is maybe not the best comparison (but will be good to visualize concept), agent run() could return object like action in Redux (maybe with naming close to model decision) that will be reduced in iterate run function. This will also help to achieve #125 easily.

[tomaszfelczyk]

Thinking in way to simplify things:

• Workflow as can be eliminated by refactor: more flexibility in workflow composition  #132 as agent for delegation can be provided as tools. Now workflow in agent.run is like circular dependency - you need to know what and how workflow works to use agent.
• Context can be provided in State as first message outside of agent run()
• Provider previous implementation without it was IMHO better because now agent run can be corrupted with wrong provider. I can define provider in agent() and then provide different one when call agent.run()

[grabbou]

Provider previous implementation without it was IMHO better because now agent run can be corrupted with wrong provider. I can define provider in agent() and then provide different one when call agent.run()

agent.run is called by the framework and framework ensures its either the provider set on the agent, or workflow-wide one (which is the default). Using closure (as perviously) was not a good idea, otherwise you would have to overwrite provider in a lot of different places, which is not a good DX.

Looking at old Redux architecture which is maybe not the best comparison (but will be good to visualize concept), agent run() could return object like action in Redux (maybe with naming close to model decision) that will be reduced in iterate run function.

This is very interesting, indeed. I will think about it! I can imagine agent.run being an action as well, and then, we can run this somewhere else. There's definitely a lot of similarities with Redux architecture, something that we should perhaps take a look into!

Context can be provided in State as first message outside of agent run()

Right now, Context is just an array of Messages, so without turning the shape of Context into multi-dimensional array, it would be hard to detect where the actual request is. When calling LLM, we want to treat the actual request/work that happened at a current level different from messages that are coming from the parent (in the future, we may convert Context to a better abstraction)

Workflow as can be eliminated by refactor: more flexibility in workflow composition #132 as agent for delegation can be provided as tools

I like that direction, but passing tools to agent with delegation will be tricky due to comment I mentioned in #132. At the moment, agent is just a function, everything else, including agent() function is just a helper to avoid creating objects without type checking. We would still need to pass that to run, otherwise it's not a pure function anymore.

[tomaszfelczyk]

This is very interesting, indeed. I will think about it! I can imagine agent.run being an action as well, and then, we can run this somewhere else. There's definitely a lot of similarities with Redux architecture, something that we should perhaps take a look into!

I tried an example implementation with abstraction changed a little bit and Redux (literally) and it looks clean and quite promising!

Two conclusions, from architectural perspective:

• In server-less environments, there should be now workflow, only iterator with reducer and some kind of middleware inside (all of them could be pure functions). So these elements can be part of a framework that can be used in many different workflow engines.
• Whole state management should be part of the workflow and workflow need to produce some side effects (workflow =/= pure function). Workflow should be package using the above "pure functions" to easily bootstrap apps in long-running environments.

[grabbou]

I would be keen to look into details, although it is hard to avoid this pattern reminding me a bit about Redux Sagas in some places 😅