[Bug]: Grounding in Gemini throws an error "invalid argument"

[timjabez]

File Name

gemini/grounding/intro-grounding-gemini.ipynb

What happened?

While trying out Grounding with Generative AI, the command response = model.generate_content(PROMPT, tools=[tool]) throws the error :

_InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
    status = StatusCode.INVALID_ARGUMENT
    details = "Request contains an invalid argument."
    debug_error_string = "UNKNOWN:Error received from peer ipv6:%5B2404:6800:4007:82b::200a%5D:443 {grpc_message:"Request contains an invalid argument.", grpc_status:3, created_time:"2024-04-11T16:48:11.261411+05:30"}"
>

Whereas response = model.generate_content(PROMPT) works just fine. I see that there's some issue in using the tools parameter.

Relevant log output

---------------------------------------------------------------------------
_InactiveRpcError                         Traceback (most recent call last)
File ~/.local/lib/python3.11/site-packages/google/api_core/grpc_helpers.py:76, in _wrap_unary_errors.<locals>.error_remapped_callable(*args, **kwargs)
     75 try:
---> 76     return callable_(*args, **kwargs)
     77 except grpc.RpcError as exc:

File ~/.local/lib/python3.11/site-packages/grpc/_channel.py:1176, in _UnaryUnaryMultiCallable.__call__(self, request, timeout, metadata, credentials, wait_for_ready, compression)
   1170 (
   1171     state,
   1172     call,
   1173 ) = self._blocking(
   1174     request, timeout, metadata, credentials, wait_for_ready, compression
   1175 )
-> 1176 return _end_unary_response_blocking(state, call, False, None)

File ~/.local/lib/python3.11/site-packages/grpc/_channel.py:1005, in _end_unary_response_blocking(state, call, with_call, deadline)
   1004 else:
-> 1005     raise _InactiveRpcError(state)

_InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
	status = StatusCode.INVALID_ARGUMENT
	details = "Request contains an invalid argument."
	debug_error_string = "UNKNOWN:Error received from peer ipv6:%5B2404:6800:4007:80a::200a%5D:443 {created_time:"2024-04-12T13:01:34.945989+05:30", grpc_status:3, grpc_message:"Request contains an invalid argument."}"
>

The above exception was the direct cause of the following exception:

InvalidArgument                           Traceback (most recent call last)
Cell In[29], line 4
      1 tool = Tool.from_google_search_retrieval(grounding.GoogleSearchRetrieval())
      2 tools=[tool]
----> 4 response = model.generate_content(PROMPT, tools=tools)

File ~/.local/lib/python3.11/site-packages/vertexai/generative_models/_generative_models.py:405, in _GenerativeModel.generate_content(self, contents, generation_config, safety_settings, tools, tool_config, stream)
    397     return self._generate_content_streaming(
    398         contents=contents,
    399         generation_config=generation_config,
   (...)
    402         tool_config=tool_config,
    403     )
    404 else:
--> 405     return self._generate_content(
    406         contents=contents,
    407         generation_config=generation_config,
    408         safety_settings=safety_settings,
    409         tools=tools,
    410         tool_config=tool_config,
    411     )

File ~/.local/lib/python3.11/site-packages/vertexai/generative_models/_generative_models.py:494, in _GenerativeModel._generate_content(self, contents, generation_config, safety_settings, tools, tool_config)
    469 """Generates content.
    470 
    471 Args:
   (...)
    485     A single GenerationResponse object
    486 """
    487 request = self._prepare_request(
    488     contents=contents,
    489     generation_config=generation_config,
   (...)
    492     tool_config=tool_config,
    493 )
--> 494 gapic_response = self._prediction_client.generate_content(request=request)
    495 return self._parse_response(gapic_response)

File ~/.local/lib/python3.11/site-packages/google/cloud/aiplatform_v1beta1/services/prediction_service/client.py:2102, in PredictionServiceClient.generate_content(self, request, model, contents, retry, timeout, metadata)
   2099 self._validate_universe_domain()
   2101 # Send the request.
-> 2102 response = rpc(
   2103     request,
   2104     retry=retry,
   2105     timeout=timeout,
   2106     metadata=metadata,
   2107 )
   2109 # Done; return the response.
   2110 return response

File ~/.local/lib/python3.11/site-packages/google/api_core/gapic_v1/method.py:131, in _GapicCallable.__call__(self, timeout, retry, compression, *args, **kwargs)
    128 if self._compression is not None:
    129     kwargs["compression"] = compression
--> 131 return wrapped_func(*args, **kwargs)

File ~/.local/lib/python3.11/site-packages/google/api_core/grpc_helpers.py:78, in _wrap_unary_errors.<locals>.error_remapped_callable(*args, **kwargs)
     76     return callable_(*args, **kwargs)
     77 except grpc.RpcError as exc:
---> 78     raise exceptions.from_grpc_error(exc) from exc

InvalidArgument: 400 Request contains an invalid argument. [detail: "[ORIGINAL ERROR] generic::invalid_argument: The GenerateContentRequest proto is invalid:\n  * tools[0].tool_type: required one_of \'tool_type\' must have one initialized field"
]

Code of Conduct

• I agree to follow this project's Code of Conduct

[holtskinner]

It's possible that you need to accept the additional Terms of Service for Grounding with Google Search in Gemini. You can find this in Vertex AI Studio:

And you can access the form directly here:

https://docs.google.com/forms/d/e/1FAIpQLSfj7SJNAqXOe7vB9Re-WtwJedTWjHkBKZ6OHaLoGoXRG93ymg/viewform

I'll update the notebook to include this step.

[ShyamKadari]

I accepted the Terms of Services using the form, but I am still facing an error. I am using the sample provided by Google.

Can I get get any help?

model = GenerativeModel("gemini-1.5-pro-preview-0409")
my_prompt = (
"What are the price, available colors, and storage size options of a Pixel Tablet?"
)
response = model.generate_content(my_prompt)
print(response.candidates[0].text)

tool = Tool.from_google_search_retrieval(grounding.GoogleSearchRetrieval())
response = model.generate_content(my_prompt,tools=[tool])

The Exception

Traceback (most recent call last):
File "C:\Users\shyam\GenAI\untitled1\GenAIGrounding.py", line 24, in
response = model.generate_content(my_prompt,tools=[tool])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\shyam\AppData\Roaming\Python\Python312\site-packages\vertexai\generative_models_generative_models.py", line 405, in generate_content
return self._generate_content(
^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\shyam\AppData\Roaming\Python\Python312\site-packages\vertexai\generative_models_generative_models.py", line 494, in _generate_content
gapic_response = self._prediction_client.generate_content(request=request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\shyam\AppData\Roaming\Python\Python312\site-packages\google\cloud\aiplatform_v1beta1\services\prediction_service\client.py", line 2102, in generate_content
response = rpc(
^^^^
File "C:\Program Files\Python312\Lib\site-packages\google\api_core\gapic_v1\method.py", line 131, in call
return wrapped_func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Python312\Lib\site-packages\google\api_core\grpc_helpers.py", line 78, in error_remapped_callable
raise exceptions.from_grpc_error(exc) from exc
google.api_core.exceptions.InvalidArgument: 400 Request contains an invalid argument.

Process finished with exit code 1

[holtskinner]

@ShyamKadari did you get a confirmation that you were allowlisted for the feature?

[ShyamKadari]

No. I did not get a confirmation email. Shyam

…

On Fri, Apr 26, 2024, 12:05 PM Holt Skinner ***@***.***> wrote: @ShyamKadari <https://github.com/ShyamKadari> did you get a confirmation that you were allowlisted for the feature? — Reply to this email directly, view it on GitHub <#536 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BICC2YXXINDOQYFZHIDYQDDY7KCNPAVCNFSM6AAAAABGDUL73KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANZZG43DKMZQG4> . You are receiving this because you were mentioned.Message ID: ***@***.***>