[Bug]: Grounding Gemini with Data Store. Error 400 Request contains an invalid argument.

[ElliotK-IB]

File Name

gemini/grounding/intro-grounding-gemini.ipynb

What happened?

I want to ground Gemini in a Vertex AI Data Store (not Google Search). I've seen previous issues similar to this one, but not quite the same/the solutions did not solve my problem.

What doesn't work:

• Grounding Gemini in Data Store

What works:

• Grounding Gemini in Google Search
• Querying Vertex AI Search grounded in Data Store (in both Agent Builder and Workbench)

I already connected the Data Store to a Search App in Agent Builder, Enterprise enabled.

I didn't request to be added to the allowlist because the form says it's for grounding in Google Search - not my use case (Data Store).

I used the following official sources for reference:

• Getting Started with Grounding with Gemini in Vertex AI
• Ground responses for Gemini models
• Ground Gemini to a Vertex AI Search data store
• Building Search Applications with Vertex AI Search

I also checked out the following issues:

• Problem when connecting Gemini to Vertex Search Data Store (currently known as Agent Builder) #569
• [Bug]: Grounding/Tools seems down #631
• [Bug]: Grounding in Gemini throws an error "invalid argument" #536

Copying the code and only changing the project ID, Data Store ID, and region did not work for me.

More details

• Python 3.10.15
• google-cloud-aiplatform 1.70.0
• Data Store: 3 PDFs (<100 pages total)

Relevant log output

---------------------------------------------------------------------------
_InactiveRpcError                         Traceback (most recent call last)
File /opt/conda/lib/python3.10/site-packages/google/api_core/grpc_helpers.py:76, in _wrap_unary_errors.<locals>.error_remapped_callable(*args, **kwargs)
     75 try:
---> 76     return callable_(*args, **kwargs)
     77 except grpc.RpcError as exc:

File /opt/conda/lib/python3.10/site-packages/grpc/_channel.py:1181, in _UnaryUnaryMultiCallable.__call__(self, request, timeout, metadata, credentials, wait_for_ready, compression)
   1175 (
   1176     state,
   1177     call,
   1178 ) = self._blocking(
   1179     request, timeout, metadata, credentials, wait_for_ready, compression
   1180 )
-> 1181 return _end_unary_response_blocking(state, call, False, None)

File /opt/conda/lib/python3.10/site-packages/grpc/_channel.py:1006, in _end_unary_response_blocking(state, call, with_call, deadline)
   1005 else:
-> 1006     raise _InactiveRpcError(state)

_InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
	status = StatusCode.INVALID_ARGUMENT
	details = "Request contains an invalid argument."
	debug_error_string = "UNKNOWN:Error received from peer ipv4:209.85.200.95:443 {created_time:"2024-10-17T17:54:34.673019348+00:00", grpc_status:3, grpc_message:"Request contains an invalid argument."}"
>

The above exception was the direct cause of the following exception:

InvalidArgument                           Traceback (most recent call last)
Cell In[9], line 11
      1 tool = Tool.from_retrieval(
      2     preview_grounding.Retrieval(
      3         preview_grounding.VertexAISearch(
   (...)
      8     )
      9 )
---> 11 response = model.generate_content(PROMPT, tools=[tool])
     12 print_grounding_response(response)

File ~/.local/lib/python3.10/site-packages/vertexai/generative_models/_generative_models.py:596, in _GenerativeModel.generate_content(self, contents, generation_config, safety_settings, tools, tool_config, stream)
    588     return self._generate_content_streaming(
    589         contents=contents,
    590         generation_config=generation_config,
   (...)
    593         tool_config=tool_config,
    594     )
    595 else:
--> 596     return self._generate_content(
    597         contents=contents,
    598         generation_config=generation_config,
    599         safety_settings=safety_settings,
    600         tools=tools,
    601         tool_config=tool_config,
    602     )

File ~/.local/lib/python3.10/site-packages/vertexai/generative_models/_generative_models.py:711, in _GenerativeModel._generate_content(self, contents, generation_config, safety_settings, tools, tool_config)
    686 """Generates content.
    687 
    688 Args:
   (...)
    702     A single GenerationResponse object
    703 """
    704 request = self._prepare_request(
    705     contents=contents,
    706     generation_config=generation_config,
   (...)
    709     tool_config=tool_config,
    710 )
--> 711 gapic_response = self._prediction_client.generate_content(request=request)
    712 return self._parse_response(gapic_response)

File ~/.local/lib/python3.10/site-packages/google/cloud/aiplatform_v1/services/prediction_service/client.py:2120, in PredictionServiceClient.generate_content(self, request, model, contents, retry, timeout, metadata)
   2117 self._validate_universe_domain()
   2119 # Send the request.
-> 2120 response = rpc(
   2121     request,
   2122     retry=retry,
   2123     timeout=timeout,
   2124     metadata=metadata,
   2125 )
   2127 # Done; return the response.
   2128 return response

File /opt/conda/lib/python3.10/site-packages/google/api_core/gapic_v1/method.py:131, in _GapicCallable.__call__(self, timeout, retry, compression, *args, **kwargs)
    128 if self._compression is not None:
    129     kwargs["compression"] = compression
--> 131 return wrapped_func(*args, **kwargs)

File /opt/conda/lib/python3.10/site-packages/google/api_core/grpc_helpers.py:78, in _wrap_unary_errors.<locals>.error_remapped_callable(*args, **kwargs)
     76     return callable_(*args, **kwargs)
     77 except grpc.RpcError as exc:
---> 78     raise exceptions.from_grpc_error(exc) from exc

InvalidArgument: 400 Request contains an invalid argument.

Code of Conduct

• I agree to follow this project's Code of Conduct

[koverholt]

Thanks for the thorough bug report w/ logs, links, and related issues! ❤️

Sorry you are running into this. In Vertex AI Search, you now need to have a data store attached to a search app to avoid this 400 error. I added warnings / notes in the notebook, so take a look at the note near the end of the notebook where that retrieval happens, and you'll see a couple of links there on how to add a search app that's linked to your data store:

Once you add and attach a search app to that data store, that should resolve the 400 error that you're running into.

[ElliotK-IB]

Hi @koverholt , thanks for your input. I still get this error when I try to ground Gemini in the data store. I've attached a Search app in Agent Builder to the data store and the search app is functional. I can query the data store from Workbench as well.

I already connected the Data Store to a Search App in Agent Builder, Enterprise enabled.

[holtskinner]

@ElliotK-IB Be sure that your location for Vertex AI Search is the location of the data store (should be one of global, us or eu) and not the same region as the GenerativeModel put in vertexai.init()

Also be sure that your data store is in the same project as where you're calling the Gemini model (set in vertexai.init())

[ElliotK-IB]

Project: us-central1
Workbench: us-central1-a
Vertex AI Search: global
Data Store: global

REGION (used in vertexai.init(project=PROJECT_ID, location=REGION)): us-central1
DATA_STORE_REGION (used in Retrieval): global

As I mentioned in my initial bug report, I have no problem querying Vertex AI Search without Gemini.

My data store is within the same project as well.

[ElliotK-IB]

I got things working by creating a new datastore with virtually the same process, but would like to provide additional info if I can find the original source of the problem. Will update if I find anything that may be insightful.

[koverholt]

Thanks for letting us know! And we appreciate your thoroughness in checking the config settings, etc.

I wonder if any data store or search app settings are different between the apps, such as "Advanced LLM features" (which is distinct from "Enterprise edition features"), or the type of data when importing (unstructured vs. JSONL vs CSV, or the chunking). I wonder if there is a combination that works when querying the data store / app directly, but doesn't work with the retrieval tool call since it might not support the same set of options (related to the data store and/or search app config) as the former approach.

[ElliotK-IB]

Unfortunately, I couldn't find any differences between the search apps/data stores. In the future, if more configuration details are surfaced for straightforward inspection, I believe it'd be easier to troubleshoot. Thanks for your help! @koverholt @holtskinner

[koverholt]

Thanks so much for your detailed debugging! And sorry we were not able to identify a clear root cause, but I am glad you were able to get it to a working state with the new app / data store! In the event you come up with a minimal reproducible code example in the future, please feel free to open a bug report on the public Vertex AI issue tracker, and we can help escalate it to the right product and engineering folks. Thanks!