Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix standard unit tests marked xfail for ChatBedrock and ChatBedrockConverse #236

Open
3coins opened this issue Oct 10, 2024 · 0 comments
Open
Labels
bedrock bug Something isn't working todo
Milestone

Comments

@3coins
Copy link
Collaborator

3coins commented Oct 10, 2024

3/63 unit tests are marked xfail for ChatBedrock and ChatBedrockConverse.

Test Log

tests/unit_tests/chat_models/test_bedrock_converse.py::TestBedrockStandard::test_init_streaming FAILED
tests/unit_tests/test_standard.py::TestBedrockStandard::test_standard_params FAILED
tests/unit_tests/test_standard.py::TestBedrockAsConverseStandard::test_standard_params FAILED

==================================================================== FAILURES =====================================================================
_____________________________________________________ TestBedrockStandard.test_init_streaming _____________________________________________________

self = <tests.unit_tests.chat_models.test_bedrock_converse.TestBedrockStandard object at 0x1246dadd0>

    @pytest.mark.xfail(reason="Doesn't support streaming init param.")
    def test_init_streaming(self) -> None:
>       super().test_init_streaming()

tests/unit_tests/chat_models/test_bedrock_converse.py:74: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/Users/pijain/Library/Caches/pypoetry/virtualenvs/langchain-aws-eH7P7gjZ-py3.10/lib/python3.10/site-packages/langchain_standard_tests/unit_tests/chat_models.py:176: in test_init_streaming
    model = self.chat_model_class(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = ChatBedrockConverse(), args = ()
kwargs = {'max_tokens': 100, 'model': 'anthropic.claude-3-sonnet-20240229-v1:0', 'region_name': 'us-west-1', 'stop': [], ...}

    def __init__(self, *args: Any, **kwargs: Any) -> None:
        """"""
>       super().__init__(*args, **kwargs)
E       pydantic_core._pydantic_core.ValidationError: 1 validation error for ChatBedrockConverse
E       streaming
E         Extra inputs are not permitted [type=extra_forbidden, input_value=True, input_type=bool]
E           For further information visit https://errors.pydantic.dev/2.9/v/extra_forbidden

/Users/pijain/Library/Caches/pypoetry/virtualenvs/langchain-aws-eH7P7gjZ-py3.10/lib/python3.10/site-packages/langchain_core/load/serializable.py:110: ValidationError
____________________________________________________ TestBedrockStandard.test_standard_params _____________________________________________________

self = <tests.unit_tests.test_standard.TestBedrockStandard object at 0x1246d6ad0>
model = ChatBedrock(client=<botocore.client.BedrockRuntime object at 0x12485e9b0>, region_name='us-east-1', model_id='anthropic.claude-3-sonnet-20240229-v1:0')

    @pytest.mark.xfail(reason="Not implemented.")
    def test_standard_params(self, model: BaseChatModel) -> None:
>       super().test_standard_params(model)

tests/unit_tests/test_standard.py:30: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/Users/pijain/Library/Caches/pypoetry/virtualenvs/langchain-aws-eH7P7gjZ-py3.10/lib/python3.10/site-packages/langchain_standard_tests/unit_tests/chat_models.py:235: in test_standard_params
    model = self.chat_model_class(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = ChatBedrock(), args = ()
kwargs = {'max_tokens': 10, 'model_id': 'anthropic.claude-3-sonnet-20240229-v1:0', 'region_name': 'us-east-1', 'stop': ['test']}

    def __init__(self, *args: Any, **kwargs: Any) -> None:
        """"""
>       super().__init__(*args, **kwargs)
E       pydantic_core._pydantic_core.ValidationError: 1 validation error for ChatBedrock
E       stop
E         Extra inputs are not permitted [type=extra_forbidden, input_value=['test'], input_type=list]
E           For further information visit https://errors.pydantic.dev/2.9/v/extra_forbidden

/Users/pijain/Library/Caches/pypoetry/virtualenvs/langchain-aws-eH7P7gjZ-py3.10/lib/python3.10/site-packages/langchain_core/load/serializable.py:110: ValidationError
_______________________________________________ TestBedrockAsConverseStandard.test_standard_params ________________________________________________

self = <tests.unit_tests.test_standard.TestBedrockAsConverseStandard object at 0x1246d5c90>
model = ChatBedrock(client=<botocore.client.BedrockRuntime object at 0x124e76aa0>, region_name='us-east-1', model_id='anthropic.claude-3-sonnet-20240229-v1:0', model_kwargs={'stop': []}, temperature=0, max_tokens=100, beta_use_converse_api=True)

    @pytest.mark.xfail(reason="Not implemented.")
    def test_standard_params(self, model: BaseChatModel) -> None:
>       super().test_standard_params(model)

tests/unit_tests/test_standard.py:58: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/Users/pijain/Library/Caches/pypoetry/virtualenvs/langchain-aws-eH7P7gjZ-py3.10/lib/python3.10/site-packages/langchain_standard_tests/unit_tests/chat_models.py:235: in test_standard_params
    model = self.chat_model_class(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = ChatBedrock(), args = ()
kwargs = {'beta_use_converse_api': True, 'max_tokens': 10, 'model_id': 'anthropic.claude-3-sonnet-20240229-v1:0', 'region_name': 'us-east-1', ...}

    def __init__(self, *args: Any, **kwargs: Any) -> None:
        """"""
>       super().__init__(*args, **kwargs)
E       pydantic_core._pydantic_core.ValidationError: 1 validation error for ChatBedrock
E       stop
E         Extra inputs are not permitted [type=extra_forbidden, input_value=['test'], input_type=list]
E           For further information visit https://errors.pydantic.dev/2.9/v/extra_forbidden

/Users/pijain/Library/Caches/pypoetry/virtualenvs/langchain-aws-eH7P7gjZ-py3.10/lib/python3.10/site-packages/langchain_core/load/serializable.py:110: ValidationError
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bedrock bug Something isn't working todo
Projects
None yet
Development

No branches or pull requests

1 participant