Refactor ChatBedrock to improve support for providers

[3coins]

Problem

Bedrock supports several providers (AI21, Anthropic, Amazon, Mistral, Meta etc.), each of which has a slightly different request/response structure. The current ChatBedrock class currently accommodates these providers by incorporating several conditional blocks which have become unsustainable as more providers, models and features have been introduced in Bedrock. This has also made it difficult and error prone to debug and add bug fixes over time.

Solution

Refactor the ChatBedrock class to individual provider classes, for example ChatBedrockAnthropic, and provide a mechanism in ChatBedrock to switch to these, so the users of don't have to create these classes separately.

[Forma-Lin]

Hello, I'm a student from University of Toronto. Me and my team from class are looking for ways we can contribute to langchain. Is this issue intended to solve the different promts model providers accept? How's the expected result different with ChatBedrockConverse class? It looks like also a solution to this issue. Thank you so much for your time reading this.

[cab938]

I'm not @3coins but I'm happy to share my frustrations @Forma-Lin . For instance, I use llama models on bedrock, but right now the langchain_aws module doesn't include support for many (any?) of the features within the llama 3.1+ models with respect to tool calling or vision. It would be ideal imo to have the ability to use various models with bedrock and get model-specific support functions.

[Forma-Lin]

Hi @cab938, thank you so much for the answer. I totally get it now. Do you consider this class refactoring as a reasonable task for four fourth-year university students to complete in a month?

[cab938]

Hi @Forma-Lin yes, I do, but I'll note that I'm not a langchain-aws committer so I can't approve any patches or the like. I don't know what the actual process is for creating langchain-aws PRs and getting them approved.

The way I would break this problem down is to think about decoupling the underlying model based on the model ID passed into bedrock, and then creating some top level classes for major models. Sometimes there is no significant difference between two models (e.g. the 70b llama 3.1 and the 405b) with respect to their vocabulary, and sometimes there is (e.g. llama 3.2 90b and llama 3.1 70b). I think you could probably build out a great version just thinking about the different llama models which exist.

1 week planning architecture, 2 weeks writing code and tests, last week actually generating PRs and ensuring docs are good. Across four senior undergraduates in CS this seems doable to me.

[Forma-Lin]

Hi @cab938 Thank you for the advice! I'm trying to contact @3coins to take this issue.

[3coins]

@Forma-Lin
Thanks for offering help with this. The immediate place to help might be look at the tasks in v0.2.7 milestone. I am discussing long term viability of maintaining the invoke API in ChatBedrock with the LangChain team, and will have more concrete tasks by end of week. In the mean time, it will be helpful to add tests for ChatBedrockConverse and fix the gaps. For example we are missing integration tests for meta models, there are some existing gaps with mistral and anthropic etc.

[3coins]

@cab938
As an open source project, any community member can contribute by either opening a PR or helping review an existing PR, opening issues to report problems or discuss solutions. All of these don't require any privileged access. This repo like LangChain follows a fork and pull-request process, as documented in the contribution guide.

If you have any other questions that is stopping you from contributing, don't hesitate to ask. I can also help assign specific issues or PRs or tag you on them, if you are interested in providing review/feedback.