Add fallback models support #532
Draft
+151
−63
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Fixes pydantic#516 Add support for fallback models in the `Agent` class. * **Agent Class Changes:** - Add `fallback_models` attribute to store a list of fallback models. - Modify `run`, `run_sync`, and `run_stream` methods to iterate through `fallback_models` if the primary model fails. - Update `_get_model` method to handle fallback models. * **Model and AgentModel Classes Changes:** - Add `fallback_models` attribute to `Model` and `AgentModel` classes. * **Tests:** - Add tests in `tests/test_agent.py` to verify the functionality of fallback models.
|
Sorry, as described on #516, this isn't the way we want to implement this feature. Instead we'd like to introduce a new model that references multiple other models, e.g.: @dataclass
class FallbackModel(Model):
models: list[Model]
...This should mean minimal changes to other parts of the system. The hardest part is going to be raising an exception from models which we can detect relates to downtime - e.g. a HTTP response with code |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #516
Add support for fallback models in the
Agentclass.Agent Class Changes:
fallback_modelsattribute to store a list of fallback models.run,run_sync, andrun_streammethods to iterate throughfallback_modelsif the primary model fails._get_modelmethod to handle fallback models.Model and AgentModel Classes Changes:
fallback_modelsattribute toModelandAgentModelclasses.Tests:
tests/test_agent.pyto verify the functionality of fallback models.