docs: review google vertex integration

[wochinge]

Proposed Changes

• fix Docstrings - integrations.google_vertex haystack#7136
• fix API Docs - integrations.google_vertex haystack#7189

[wochinge]

copy pasted all this stuff from our docs (e.g. https://docs.haystack.deepset.ai/v2.0/docs/vertexaitextgenerator)

[wochinge]

---

title: Google Vertex
excerpt: Google Vertex integration for Haystack
category: placeholder-integrations-api
slug: integrations-google-vertex
parentDoc:
order: 100
hidden: false

Module haystack_integrations.components.generators.google_vertex.gemini

VertexAIGeminiGenerator

@component
class VertexAIGeminiGenerator()

VertexAIGeminiGenerator enables text generation using Google Gemini models.

VertexAIGeminiGenerator supports both gemini-pro and gemini-pro-vision models.
Prompting with images requires gemini-pro-vision. Function calling, instead, requires gemini-pro.

Usage example:

from haystack_integrations.components.generators.google_vertex import VertexAIGeminiGenerator


gemini = VertexAIGeminiGenerator(project_id=project_id)
result = gemini.run(parts = ["What is the most interesting thing you know?"])
for answer in result["answers"]:
    print(answer)

>>> 1. **The Origin of Life:** How and where did life begin? The answers to this ...
>>> 2. **The Unseen Universe:** The vast majority of the universe is ...
>>> 3. **Quantum Entanglement:** This eerie phenomenon in quantum mechanics allows ...
>>> 4. **Time Dilation:** Einstein's theory of relativity revealed that time can ...
>>> 5. **The Fermi Paradox:** Despite the vastness of the universe and the ...
>>> 6. **Biological Evolution:** The idea that life evolves over time through natural ...
>>> 7. **Neuroplasticity:** The brain's ability to adapt and change throughout life, ...
>>> 8. **The Goldilocks Zone:** The concept of the habitable zone, or the Goldilocks zone, ...
>>> 9. **String Theory:** This theoretical framework in physics aims to unify all ...
>>> 10. **Consciousness:** The nature of human consciousness and how it arises ...

VertexAIGeminiGenerator.__init__

def __init__(*,
             model: str = "gemini-pro-vision",
             project_id: str,
             location: Optional[str] = None,
             generation_config: Optional[Union[GenerationConfig,
                                               Dict[str, Any]]] = None,
             safety_settings: Optional[Dict[HarmCategory,
                                            HarmBlockThreshold]] = None,
             tools: Optional[List[Tool]] = None)

Multi-modal generator using Gemini model via Google Vertex AI.

Authenticates using Google Cloud Application Default Credentials (ADCs).
For more information see the official Google documentation.

Arguments:

• project_id: ID of the GCP project to use.
• model: Name of the model to use.
• location: The default location to use when making API calls, if not set uses us-central-1.
• generation_config: The generation config to use.
  Can either be a GenerationConfig
  object or a dictionary of parameters.
  Accepted fields are:
  • temperature
  • top_p
  • top_k
  • candidate_count
  • max_output_tokens
  • stop_sequences
• safety_settings: The safety settings to use. See the documentation
  for HarmBlockThreshold
  and HarmCategory
  for more details.
• tools: List of tools to use when generating content. See the documentation for
  Tool
  the list of supported arguments.

VertexAIGeminiGenerator.to_dict

def to_dict() -> Dict[str, Any]

Serializes the component to a dictionary.

Returns:

Dictionary with serialized data.

VertexAIGeminiGenerator.from_dict

@classmethod
def from_dict(cls, data: Dict[str, Any]) -> "VertexAIGeminiGenerator"

Deserializes the component from a dictionary.

Arguments:

• data: Dictionary to deserialize from.

Returns:

Deserialized component.

VertexAIGeminiGenerator.run

@component.output_types(answers=List[Union[str, Dict[str, str]]])
def run(parts: Variadic[Union[str, ByteStream, Part]])

Generates content using the Gemini model.

Arguments:

• parts: Prompt for the model.

Returns:

A dictionary with the following keys:

• answers: A list of generated content.

Module haystack_integrations.components.generators.google_vertex.captioner

VertexAIImageCaptioner

@component
class VertexAIImageCaptioner()

VertexAIImageCaptioner enables text generation using Google Vertex AI imagetext generative model.

Authenticates using Google Cloud Application Default Credentials (ADCs).
For more information see the official Google documentation.

Usage example:

import requests

from haystack.dataclasses.byte_stream import ByteStream
from haystack_integrations.components.generators.google_vertex import VertexAIImageCaptioner

captioner = VertexAIImageCaptioner(project_id=project_id)

image = ByteStream(data=requests.get("https://raw.githubusercontent.com/silvanocerza/robots/main/robot1.jpg").content)
result = captioner.run(image=image)

for caption in result["captions"]:
    print(caption)

>>> two gold robots are standing next to each other in the desert

VertexAIImageCaptioner.__init__

def __init__(*,
             model: str = "imagetext",
             project_id: str,
             location: Optional[str] = None,
             **kwargs)

Generate image captions using a Google Vertex AI model.

Authenticates using Google Cloud Application Default Credentials (ADCs).
For more information see the official Google documentation.

Arguments:

• project_id: ID of the GCP project to use.
• model: Name of the model to use.
• location: The default location to use when making API calls, if not set uses us-central-1.
  Defaults to None.
• kwargs: Additional keyword arguments to pass to the model.
  For a list of supported arguments see the ImageTextModel.get_captions() documentation.

VertexAIImageCaptioner.to_dict

def to_dict() -> Dict[str, Any]

Serializes the component to a dictionary.

Returns:

Dictionary with serialized data.

VertexAIImageCaptioner.from_dict

@classmethod
def from_dict(cls, data: Dict[str, Any]) -> "VertexAIImageCaptioner"

Deserializes the component from a dictionary.

Arguments:

• data: Dictionary to deserialize from.

Returns:

Deserialized component.

VertexAIImageCaptioner.run

@component.output_types(captions=List[str])
def run(image: ByteStream)

Prompts the model to generate captions for the given image.

Arguments:

• image: The image to generate captions for.

Returns:

A dictionary with the following keys:

• captions: A list of captions generated by the model.

Module haystack_integrations.components.generators.google_vertex.code_generator

VertexAICodeGenerator

@component
class VertexAICodeGenerator()

This component enables code generation using Google Vertex AI generative model.

VertexAICodeGenerator supports code-bison, code-bison-32k, and code-gecko.

Usage example:

    from haystack_integrations.components.generators.google_vertex import VertexAICodeGenerator

    generator = VertexAICodeGenerator(project_id=project_id)

    result = generator.run(prefix="def to_json(data):")

    for answer in result["answers"]:
        print(answer)

    >>> ```python
    >>> import json
    >>>
    >>> def to_json(data):
    >>>   """Converts a Python object to a JSON string.
    >>>
    >>>   Args:
    >>>     data: The Python object to convert.
    >>>
    >>>   Returns:
    >>>     A JSON string representing the Python object.
    >>>   """
    >>>
    >>>   return json.dumps(data)
    >>> ```

VertexAICodeGenerator.__init__

def __init__(*,
             model: str = "code-bison",
             project_id: str,
             location: Optional[str] = None,
             **kwargs)

Generate code using a Google Vertex AI model.

Authenticates using Google Cloud Application Default Credentials (ADCs).
For more information see the official Google documentation.

Arguments:

• project_id: ID of the GCP project to use.
• model: Name of the model to use.
• location: The default location to use when making API calls, if not set uses us-central-1.
• kwargs: Additional keyword arguments to pass to the model.
  For a list of supported arguments see the TextGenerationModel.predict() documentation.

VertexAICodeGenerator.to_dict

def to_dict() -> Dict[str, Any]

Serializes the component to a dictionary.

Returns:

Dictionary with serialized data.

VertexAICodeGenerator.from_dict

@classmethod
def from_dict(cls, data: Dict[str, Any]) -> "VertexAICodeGenerator"

Deserializes the component from a dictionary.

Arguments:

• data: Dictionary to deserialize from.

Returns:

Deserialized component.

VertexAICodeGenerator.run

@component.output_types(answers=List[str])
def run(prefix: str, suffix: Optional[str] = None)

Generate code using a Google Vertex AI model.

Arguments:

• prefix: Code before the current point.
• suffix: Code after the current point.

Returns:

A dictionary with the following keys:

• answers: A list of generated code snippets.

Module haystack_integrations.components.generators.google_vertex.image_generator

VertexAIImageGenerator

@component
class VertexAIImageGenerator()

This component enables image generation using Google Vertex AI generative model.

Authenticates using Google Cloud Application Default Credentials (ADCs).
For more information see the official Google documentation.

Usage example:

from pathlib import Path

from haystack_integrations.components.generators.google_vertex import VertexAIImageGenerator

generator = VertexAIImageGenerator(project_id=project_id)
result = generator.run(prompt="Generate an image of a cute cat")
result["images"][0].to_file(Path("my_image.png"))

VertexAIImageGenerator.__init__

def __init__(*,
             model: str = "imagegeneration",
             project_id: str,
             location: Optional[str] = None,
             **kwargs)

Generates images using a Google Vertex AI model.

Authenticates using Google Cloud Application Default Credentials (ADCs).
For more information see the official Google documentation.

Arguments:

• project_id: ID of the GCP project to use.
• model: Name of the model to use.
• location: The default location to use when making API calls, if not set uses us-central-1.
• kwargs: Additional keyword arguments to pass to the model.
  For a list of supported arguments see the ImageGenerationModel.generate_images() documentation.

VertexAIImageGenerator.to_dict

def to_dict() -> Dict[str, Any]

Serializes the component to a dictionary.

Returns:

Dictionary with serialized data.

VertexAIImageGenerator.from_dict

@classmethod
def from_dict(cls, data: Dict[str, Any]) -> "VertexAIImageGenerator"

Deserializes the component from a dictionary.

Arguments:

• data: Dictionary to deserialize from.

Returns:

Deserialized component.

VertexAIImageGenerator.run

@component.output_types(images=List[ByteStream])
def run(prompt: str, negative_prompt: Optional[str] = None)

Produces images based on the given prompt.

Arguments:

• prompt: The prompt to generate images from.
• negative_prompt: A description of what you want to omit in
  the generated images.

Returns:

A dictionary with the following keys:

• images: A list of ByteStream objects, each containing an image.

Module haystack_integrations.components.generators.google_vertex.question_answering

VertexAIImageQA

@component
class VertexAIImageQA()

This component enables text generation (image captioning) using Google Vertex AI generative models.

Authenticates using Google Cloud Application Default Credentials (ADCs).
For more information see the official Google documentation.

Usage example:

from haystack.dataclasses.byte_stream import ByteStream
from haystack_integrations.components.generators.google_vertex import VertexAIImageQA

qa = VertexAIImageQA(project_id=project_id)

image = ByteStream.from_file_path("dog.jpg")

res = qa.run(image=image, question="What color is this dog")

print(res["answers"][0])

>>> white

VertexAIImageQA.__init__

def __init__(*,
             model: str = "imagetext",
             project_id: str,
             location: Optional[str] = None,
             **kwargs)

Answers questions about an image using a Google Vertex AI model.

Authenticates using Google Cloud Application Default Credentials (ADCs).
For more information see the official Google documentation.

Arguments:

• project_id: ID of the GCP project to use.
• model: Name of the model to use.
• location: The default location to use when making API calls, if not set uses us-central-1.
• kwargs: Additional keyword arguments to pass to the model.
  For a list of supported arguments see the ImageTextModel.ask_question() documentation.

VertexAIImageQA.to_dict

def to_dict() -> Dict[str, Any]

Serializes the component to a dictionary.

Returns:

Dictionary with serialized data.

VertexAIImageQA.from_dict

@classmethod
def from_dict(cls, data: Dict[str, Any]) -> "VertexAIImageQA"

Deserializes the component from a dictionary.

Arguments:

• data: Dictionary to deserialize from.

Returns:

Deserialized component.

VertexAIImageQA.run

@component.output_types(answers=List[str])
def run(image: ByteStream, question: str)

Prompts model to answer a question about an image.

Arguments:

• image: The image to ask the question about.
• question: The question to ask.

Returns:

A dictionary with the following keys:

• answers: A list of answers to the question.

Module haystack_integrations.components.generators.google_vertex.text_generator

VertexAITextGenerator

@component
class VertexAITextGenerator()

This component enables text generation using Google Vertex AI generative models.

VertexAITextGenerator supports text-bison, text-unicorn and text-bison-32k models.

Authenticates using Google Cloud Application Default Credentials (ADCs).
For more information see the official Google documentation.

Usage example:

    from haystack_integrations.components.generators.google_vertex import VertexAITextGenerator

    generator = VertexAITextGenerator(project_id=project_id)
    res = generator.run("Tell me a good interview question for a software engineer.")

    print(res["answers"][0])

    >>> **Question:**
    >>> You are given a list of integers and a target sum.
    >>> Find all unique combinations of numbers in the list that add up to the target sum.
    >>>
    >>> **Example:**
    >>>
    >>> ```
    >>> Input: [1, 2, 3, 4, 5], target = 7
    >>> Output: [[1, 2, 4], [3, 4]]
    >>> ```
    >>>
    >>> **Follow-up:** What if the list contains duplicate numbers?

VertexAITextGenerator.__init__

def __init__(*,
             model: str = "text-bison",
             project_id: str,
             location: Optional[str] = None,
             **kwargs)

Generate text using a Google Vertex AI model.

Authenticates using Google Cloud Application Default Credentials (ADCs).
For more information see the official Google documentation.

Arguments:

• project_id: ID of the GCP project to use.
• model: Name of the model to use.
• location: The default location to use when making API calls, if not set uses us-central-1.
• kwargs: Additional keyword arguments to pass to the model.
  For a list of supported arguments see the TextGenerationModel.predict() documentation.

VertexAITextGenerator.to_dict

def to_dict() -> Dict[str, Any]

Serializes the component to a dictionary.

Returns:

Dictionary with serialized data.

VertexAITextGenerator.from_dict

@classmethod
def from_dict(cls, data: Dict[str, Any]) -> "VertexAITextGenerator"

Deserializes the component from a dictionary.

Arguments:

• data: Dictionary to deserialize from.

Returns:

Deserialized component.

VertexAITextGenerator.run

@component.output_types(answers=List[str],
                        safety_attributes=Dict[str, float],
                        citations=List[Dict[str, Any]])
def run(prompt: str)

Prompts the model to generate text.

Arguments:

• prompt: The prompt to use for text generation.

Returns:

A dictionary with the following keys:

• answers: A list of generated answers.
• safety_attributes: A dictionary with the safety scores
  of each answer.
• citations: A list of citations for each answer.

Module haystack_integrations.components.generators.google_vertex.chat.gemini

VertexAIGeminiChatGenerator

@component
class VertexAIGeminiChatGenerator()

VertexAIGeminiChatGenerator enables chat completion using Google Gemini models.

VertexAIGeminiChatGenerator supports both gemini-pro and gemini-pro-vision models.
Prompting with images requires gemini-pro-vision. Function calling, instead, requires gemini-pro.

Authenticates using Google Cloud Application Default Credentials (ADCs).
For more information see the official Google documentation.

Usage example:

from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator

gemini_chat = VertexAIGeminiChatGenerator(project_id=project_id)

messages = [ChatMessage.from_user("Tell me the name of a movie")]
res = gemini_chat.run(messages)

print(res["replies"][0].content)
>>> The Shawshank Redemption

VertexAIGeminiChatGenerator.__init__

def __init__(*,
             model: str = "gemini-pro",
             project_id: str,
             location: Optional[str] = None,
             generation_config: Optional[Union[GenerationConfig,
                                               Dict[str, Any]]] = None,
             safety_settings: Optional[Dict[HarmCategory,
                                            HarmBlockThreshold]] = None,
             tools: Optional[List[Tool]] = None)

VertexAIGeminiChatGenerator enables chat completion using Google Gemini models.

Authenticates using Google Cloud Application Default Credentials (ADCs).
For more information see the official Google documentation.

Arguments:

• project_id: ID of the GCP project to use.
• model: Name of the model to use, defaults to "gemini-pro-vision".
• location: The default location to use when making API calls, if not set uses us-central-1.
  Defaults to None.
• generation_config: Configuration for the generation process.
  See the [GenerationConfig documentation](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.preview.generative_models.GenerationConfig
  for a list of supported arguments.
• safety_settings: Safety settings to use when generating content. See the documentation
  for HarmBlockThreshold
  and HarmCategory
  for more details.
• tools: List of tools to use when generating content. See the documentation for
  Tool
  the list of supported arguments.

VertexAIGeminiChatGenerator.to_dict

def to_dict() -> Dict[str, Any]

Serializes the component to a dictionary.

Returns:

Dictionary with serialized data.

VertexAIGeminiChatGenerator.from_dict

@classmethod
def from_dict(cls, data: Dict[str, Any]) -> "VertexAIGeminiChatGenerator"

Deserializes the component from a dictionary.

Arguments:

• data: Dictionary to deserialize from.

Returns:

Deserialized component.

VertexAIGeminiChatGenerator.run

@component.output_types(replies=List[ChatMessage])
def run(messages: List[ChatMessage])

Prompts Google Vertex AI Gemini model to generate a response to a list of messages.

Arguments:

• messages: The last message is the prompt, the rest are the history.

Returns:

A dictionary with the following keys:

• replies: A list of ChatMessage objects representing the model's replies.