Add language model base and work out interactions with planning manager #3850
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 1 Ignored Deployment
|
|
Some thoughts on how this will work: # This may happen "client" application side before we create the Agent, I'm thinking
planner = Planner(...)
language_model = LanguageModel(...)
initial_user_input = ... # Application logic for this
objective_prompt = planner.construct_objective_prompt_from_user_input(initial_user_input)
model_response = language_model.determine_agent_objective(objective_prompt)
agent_objective = model_response.content # This should be parsed into a standard format already
# This may happen on the "server" side (across the message broker interface)
# ...Update API budget, etc. ...
agent = Agent(
agent_objective,
planner,
language_model,
...,
)
agent.run()class Agent:
def __init__(self, agent_objective, planner, language_model, ...):
self.objective = agent_objective
self.planner = planner
self.language_model = language_model
...
def run():
while True:
# Logic to construct the context (checking budget, looking for memories, getting user input)
planning_prompt_context = PlanningPromptContext(...)
planning_prompt = self.planner.construct_planning_prompt_from_context(planning_prompt_context)
action_plan_response = self.language_model.plan_next_axtion(planning_prompt)
action_plan = action_plan_response.content
# ...Update API budget, check for user confirmation, update memory, etc.
# Logic to construct the context ...
self_feedback_prompt_context = SelfFeedbackPromptContext(...)
self_feedback_prompt = self.planner.get_self_feedback_prompt(self_feedback_prompt_context)
self_feedback_response = self.language_model.get_self_feedback(self_feedback_prompt)
self_feedback = self_feedback_response.content
# ...Update API budget, resolve action plan with self feedback, update memory, do next action, etc. |
|
People interested in learning more about ongoing talks about the planning manager, refer to: #3593 |
| @abc.abstractmethod | ||
| def __init__( | ||
| self, | ||
| configuration: Configuration, |
There was a problem hiding this comment.
Just wondering, why are we importing the entire configuration here?
There was a problem hiding this comment.
I'm starting permissive because we're going to have to port the existing logic into this system and we're going to find edge cases in that port that we can't isolate without significant work. We don't even have the most basic form of encapsulation yet. We can tighten contracts later once we have a little structure to lean on, and our priority in this work is getting that structure in quickly.
|
|
||
| @abc.abstractmethod | ||
| def get_model_info(self, model_name: str) -> ModelInfo: | ||
| """Get information about a specific model.""" |
There was a problem hiding this comment.
Where do we add new models?
There was a problem hiding this comment.
That's a good question and one I'll show in concrete implementation. Essentially the core system will define a couple of models. The plugin system may register new ones (eventually). Part of the language model definition will include concrete instances of LanguageModelInfo and EmbeddingModelInfo objects.
|
|
||
| if typing.TYPE_CHECKING: |
There was a problem hiding this comment.
Why are we doing this? Is there any recursive import dependencies?
There was a problem hiding this comment.
There is an import dependency structure, it is currently unclear and in flux and will include cyclic dependencies, I'm proactively making sure the other developers contributing systems don't hit those cyclic dependencies an can work independently. We will resolve imports in the more standard way when all the work is joined.
| @abc.abstractmethod | ||
| def list_models(self) -> Dict[str, ModelInfo]: | ||
| """List all available models.""" | ||
| ... |
There was a problem hiding this comment.
No need for ellipsis if docstrings. We can get rid of all ellipsis in favor of docstrings to make it more readable.
There was a problem hiding this comment.
This is a nitpick and one I don't totally agree with. It also has no bearing on the PR content.
This PR adds the base abstraction for the language model and adjusts some of the interface of the Planning model to create their communication contract.