more resilient anthropic (model settings, mdown prompt structure, json correction) #34
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Introduced a dictionary (LLM_MODEL_MAP) to map LLM types and model name prefixes to their respective classes. - Updated the model prefix extraction logic to correctly handle the structure of model names (e.g., extracting 'anthropic' from 'anthropic.claude-3-sonnet-20240229-v1:0'). - Improved error handling to provide more specific error messages when LLM types or model prefixes are not implemented. - Added type annotations and imports for better clarity and maintainability. - Enhanced logging to provide clear information on the LLM provider and model being used. These changes make the function more resilient, modular, and easier to extend with new LLM models.
…ibility - Updated Prompter to allow dynamic configuration of model settings through the decorator. - Added `model_settings` parameter to the `@Prompter` decorator. - Modified `_Pr` class to pass `model_settings` to the LLM provider. - Refactored BedRockAnthropic to support configurable model settings. - Introduced `model_settings` parameter in the constructor to allow dynamic configuration of settings like `max_tokens`, `temperature`, `top_p`, `top_k`, `stop_sequences`, and `anthropic_version`. - Ensured `stop_sequences` and `anthropic_version` are always included in the request body. - Improved error handling and logging for better traceability. These changes enhance the flexibility and configurability of the prompter system, allowing for dynamic adjustment of LLM settings to suit different scenarios and requirements.
… updates, model prompt structure using mdown and better instructions, and using fix_busted_json to resolve cases with llm responding with extra text rather than just json object (requires a dependancy lib: fix_busted_json)
helmanofer
approved these changes
May 26, 2024
helmanofer
approved these changes
May 26, 2024
helmanofer
added a commit
that referenced
this pull request
May 26, 2024
…n correction) (#34) * Refactored get_llm function for improved maintainability and flexibility - Introduced a dictionary (LLM_MODEL_MAP) to map LLM types and model name prefixes to their respective classes. - Updated the model prefix extraction logic to correctly handle the structure of model names (e.g., extracting 'anthropic' from 'anthropic.claude-3-sonnet-20240229-v1:0'). - Improved error handling to provide more specific error messages when LLM types or model prefixes are not implemented. - Added type annotations and imports for better clarity and maintainability. - Enhanced logging to provide clear information on the LLM provider and model being used. These changes make the function more resilient, modular, and easier to extend with new LLM models. * Refactored Prompter and BedRockAnthropic for configurability and flexibility - Updated Prompter to allow dynamic configuration of model settings through the decorator. - Added `model_settings` parameter to the `@Prompter` decorator. - Modified `_Pr` class to pass `model_settings` to the LLM provider. - Refactored BedRockAnthropic to support configurable model settings. - Introduced `model_settings` parameter in the constructor to allow dynamic configuration of settings like `max_tokens`, `temperature`, `top_p`, `top_k`, `stop_sequences`, and `anthropic_version`. - Ensured `stop_sequences` and `anthropic_version` are always included in the request body. - Improved error handling and logging for better traceability. These changes enhance the flexibility and configurability of the prompter system, allowing for dynamic adjustment of LLM settings to suit different scenarios and requirements. * Upgrading anthropic bedrock model more resillient: boto client config updates, model prompt structure using mdown and better instructions, and using fix_busted_json to resolve cases with llm responding with extra text rather than just json object (requires a dependancy lib: fix_busted_json) --------- Co-authored-by: Ofer Helman <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
making anthropic bedrock model more resilient: boto client config updates, model prompt structure using mdown and better instructions, and using fix_busted_json to resolve cases with llm responding with extra text rather than just json object (requires a dependency lib: fix_busted_json)