[Bug]: Model throws "400 Request contains an invalid argument" error wnen multiple Tools are registered

[aminzergophr]

File Name

gemini/function-calling/intro_function_calling.ipynb

What happened?

Hi!

There is an issue with Model response when passing multiple Tools into it.
It throws google.api_core.exceptions.InvalidArgument with message 400 Request contains an invalid argument error.

There is an example file to reproduce this issue in the bottom.

1. If a single Tool is used - no exception is thrown, the response is as expected.
2. If 2 Tools are used - the exception above is thrown.

Can you please check why the exception is thrown in the second scenario when 2 tools are used?

Thanks!

---

• Windows
• Python 3.11.0
• vertexai package version 1.71.1

---

import json
import os
from environ import environ
import vertexai
from google.oauth2 import service_account
from vertexai.preview.generative_models import FunctionDeclaration, GenerativeModel, Tool

env = environ.Env()
env.read_env('./.env')

vertexai.init(
    project=os.environ.get("GOOGLE_CLOUD_PROJECT"),
    location=os.environ.get("GOOGLE_CLOUD_PROJECT_LOCATION"),
    credentials=service_account.Credentials.from_service_account_info(
        json.loads(os.environ.get("AI_BOT_SERVICE_ACCOUNT_KEY_FILE", None)),
    ),
)

get_product_sku_func = FunctionDeclaration(
    name="get_product_sku",
    description="Get the available inventory for a Google products, e.g: Pixel phones, Pixel Watches, Google Home etc",
    parameters={
        "type": "object",
        "properties": {
            "product_name": {"type": "string", "description": "Product name"}
        },
    },
)

get_store_location_func = FunctionDeclaration(
    name="get_store_location",
    description="Get the location of the closest store",
    parameters={
        "type": "object",
        "properties": {"location": {"type": "string", "description": "Location"}},
    },
)

retail_tool = Tool(
    function_declarations=[
        get_product_sku_func,
        get_store_location_func,
    ],
)

get_product_sku_tool = Tool(
    function_declarations=[
        get_product_sku_func,
    ],
)

get_store_location_tool = Tool(
    function_declarations=[
        get_store_location_func,
    ],
)

model = GenerativeModel(
    model_name="gemini-1.5-flash-002",

    # Uncomment the next line to make it work as expected
    # tools=[retail_tool],

    # Uncomment the next line to make it throw an exception
    tools=[
        get_product_sku_tool,
        get_store_location_tool,
    ],
)

response = model.generate_content(
    "Do you have the Pixel 8 Pro 128GB in stock?",
)

print('function to be called:')
print(response.candidates[0].function_calls)

Relevant log output

Traceback (most recent call last):
  File "D:\Projects\Gophr\ai-bot\test_tools.py", line 71, in <module>
    response = model.generate_content(
               ^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python311\Lib\site-packages\vertexai\generative_models\_generative_models.py", line 654, in generate_content
    return self._generate_content(
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python311\Lib\site-packages\vertexai\generative_models\_generative_models.py", line 779, in _generate_content
    gapic_response = self._prediction_client.generate_content(request=request)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python311\Lib\site-packages\google\cloud\aiplatform_v1beta1\services\prediction_service\client.py", line 2319, in generate_content
    response = rpc(
               ^^^^
  File "C:\Python311\Lib\site-packages\google\api_core\gapic_v1\method.py", line 131, in __call__
    return wrapped_func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python311\Lib\site-packages\google\api_core\grpc_helpers.py", line 78, in error_remapped_callable
    raise exceptions.from_grpc_error(exc) from exc
google.api_core.exceptions.InvalidArgument: 400 Request contains an invalid argument.

Code of Conduct

• I agree to follow this project's Code of Conduct

[aminzergophr]

As per this existing Issue it's not possible at the moment to pass 2 or more Tools into Model. If that's still true - please feel free to close this ticket.

[holtskinner]

@aminzergophr

As per this existing Issue it's not possible at the moment to pass 2 or more Tools into Model. If that's still true - please feel free to close this ticket.

That is the case for Gemini 1.0 & 1.5. However, it is possible to use multiple tools with Gemini 2.0 Flash. You can see how to do this in https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_2_0_flash.ipynb

[aminzergophr]

Got it, thanks for reply 👍