wip

.github/actions/spelling/allow.txt

@@ -118,6 +118,7 @@ Khanh
 Knopf
 Kubeflow
 Kudrow
+Langraph
 LCEL
 LLMs
 LOOKBACK
@@ -286,6 +287,7 @@ codebase
 codebases
 codelab
 codelabs
+codespell
 colab
 coldline
 coloraxis

gemini/sample-apps/conversational-genai-app-template/README.md

@@ -1,15 +1,15 @@
-# 🚀 Conversational GenAI App Template! 🚀
+# 🚀 Conversational Generative AI App Template! 🚀
 >**Focus on Innovation, not Infrastructure**
 
 This folder is a **starter-pack** to building a Generative AI application on Google Cloud Platform (GCP). 
 
-It is meant to be a template repository to build your own GenAI application.
+It is meant to be a template repository to build your own Generative AI application.
 
 We provide a comprehensive set of resources to guide you through the entire development process, from prototype to production.
 
 ## High-Level Architecture
 
-This template covers all aspects of GenAI app development, from prototyping and evaluation to deployment and monitoring.
+This template covers all aspects of Generative AI app development, from prototyping and evaluation to deployment and monitoring.
 
 ![High Level Architecture](images/high_level_architecture.png "Architecture")
 
@@ -39,7 +39,7 @@ This template covers all aspects of GenAI app development, from prototyping and
 
 | Description | Visualization |
 |-------------|---------------|
-| The repository showcases how to evaluate GenAI applications using tools like Vertex AI rapid eval SDK and Vertex AI Experiments. | ![Vertex AI Rapid Eval](images/vertex_ai_rapid_eval.png) |
+| The repository showcases how to evaluate Generative AI applications using tools like Vertex AI rapid eval SDK and Vertex AI Experiments. | ![Vertex AI Rapid Eval](images/vertex_ai_rapid_eval.png) |
 
 </details>
 
@@ -60,7 +60,7 @@ This template covers all aspects of GenAI app development, from prototyping and
 
 | Description | Visualization |
 |-------------|---------------|
-| Monitor your GenAI Application's performance. We provide a Looker Studio [dashboard](https://lookerstudio.google.com/u/0/reporting/fa742264-4b4b-4c56-81e6-a667dd0f853f) to monitor application conversation statistics and user feedback. | ![Dashboard1](images/dashboard_1.png) |
+| Monitor your Generative AI Application's performance. We provide a Looker Studio [dashboard](https://lookerstudio.google.com/u/0/reporting/fa742264-4b4b-4c56-81e6-a667dd0f853f) to monitor application conversation statistics and user feedback. | ![Dashboard1](images/dashboard_1.png) |
 | We can also drill down to individual conversations and view the messages exchanged | ![Dashboard2](images/dashboard_2.png) |
 
 </details>
@@ -79,7 +79,7 @@ This template covers all aspects of GenAI app development, from prototyping and
 
 | Description | Visualization |
 |-------------|---------------|
-| Experiment with your GenAI Application in a feature-rich playground, including chat curation, user feedback collection, multimodal input, and more! | ![Streamlit View](images/streamlit_view.png) |
+| Experiment with your Generative AI Application in a feature-rich playground, including chat curation, user feedback collection, multimodal input, and more! | ![Streamlit View](images/streamlit_view.png) |
 
 </details>
 
@@ -97,7 +97,7 @@ This template covers all aspects of GenAI app development, from prototyping and
 ```bash
 gsutil -m cp -r gs://genai-starter-pack-templates/genai-starter-pack-template .
 ```
-Use the downloaded folder as a starting point for your own GenAI application.
+Use the downloaded folder as a starting point for your own Generative AI application.
 
 
 ### Installation
@@ -131,11 +131,11 @@ For full command options and usage, refer to the [Makefile](Makefile).
 
 ## Usage
 
-1. **Prototype Your Chain:** Build your GenAI Application using different methodologies and frameworks. Use Vertex AI Evaluation for assessing the performances of your application and relative chain of steps. **See [`notebooks/getting_started.ipynb`](notebooks/getting_started.ipynb) for a tutorial to get started building and evaluating your chain.**
+1. **Prototype Your Chain:** Build your Generative AI Application using different methodologies and frameworks. Use Vertex AI Evaluation for assessing the performances of your application and relative chain of steps. **See [`notebooks/getting_started.ipynb`](notebooks/getting_started.ipynb) for a tutorial to get started building and evaluating your chain.**
 2. **Integrate into the App:** Import your chain into the app. Edit `app/chain.py` file to add your chain.
 3. **Playground Testing:** Explore your chain's functionality using the Streamlit playground. Take advantage of the comprehensive playground features, such as chat history management, user feedback mechanisms, support for various input types, and additional capabilities. You can run the playground locally with `make playground` command. 
 4. **Deploy with CI/CD:** Configure and trigger the CI/CD pipelines. Edit tests if needed. See the [deployment section](#deployment) below for more details.
-5. **Monitor in Production:** Track performance and gather insights using Cloud Logging, Tracing, and the Looker Studio dashboard. Use the gathered data to iterate on your GenAI application.
+5. **Monitor in Production:** Track performance and gather insights using Cloud Logging, Tracing, and the Looker Studio dashboard. Use the gathered data to iterate on your Generative AI application.
 
 
 ## Deployment

gemini/sample-apps/conversational-genai-app-template/app/README.md

@@ -14,7 +14,7 @@ This folder implements a chatbot application using FastAPI, and Google Cloud ser
 ├── utils/              # Utility functions and classes
 └── eval/               # Evaluation tools and data
 ```
-## GenAI Application Patterns
+## Generative AI Application Patterns
 
 ### 1. Default Chain
 
@@ -45,7 +45,7 @@ All chains have the same interface, allowing for seamless swapping without chang
 
 ### Trace and Log Capture
 
-This application utilizes [OpenTelemetry](https://opentelemetry.io/) and [OpenLLMetry](https://github.com/traceloop/openllmetry) for comprehensive observability, emitting events to Google Cloud Trace and Google Cloud Logging. Every interaction with Langchain and VertexAI is instrumented (see [`server.py`](server.py)), enabling detailed tracing of request flows throughout the application.
+This application utilizes [OpenTelemetry](https://opentelemetry.io/) and [OpenLLMetry](https://github.com/traceloop/openllmetry) for comprehensive observability, emitting events to Google Cloud Trace and Google Cloud Logging. Every interaction with LangChain and VertexAI is instrumented (see [`server.py`](server.py)), enabling detailed tracing of request flows throughout the application.
 
 Leveraging the [CloudTraceSpanExporter](https://cloud.google.com/python/docs/reference/spanner/latest/opentelemetry-tracing), the application captures and exports tracing data. To address the  limitations of Cloud Trace ([256-byte attribute value limit](https://cloud.google.com/trace/docs/quotas#limits_on_spans)) and [Cloud Logging](https://cloud.google.com/logging/quotas) ([256KB log entry size](https://cloud.google.com/logging/quotas)), a custom extension of the CloudTraceSpanExporter is implemented in [`app/utils/tracing.py`](app/utils/tracing.py).
 

gemini/sample-apps/conversational-genai-app-template/app/chain.py

@@ -15,25 +15,22 @@
 from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
 from langchain_google_vertexai import ChatVertexAI, HarmBlockThreshold, HarmCategory
 
-safety_settings = {
-    HarmCategory.HARM_CATEGORY_UNSPECIFIED: HarmBlockThreshold.BLOCK_ONLY_HIGH,
-    HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT: HarmBlockThreshold.BLOCK_ONLY_HIGH,
-    HarmCategory.HARM_CATEGORY_HATE_SPEECH: HarmBlockThreshold.BLOCK_ONLY_HIGH,
-    HarmCategory.HARM_CATEGORY_HARASSMENT: HarmBlockThreshold.BLOCK_ONLY_HIGH,
-    HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT: HarmBlockThreshold.BLOCK_ONLY_HIGH,
-}
 
 llm = ChatVertexAI(
-    model_name="gemini-1.5-flash-001", temperature=0, max_output_tokens=1024,
-    safety_settings=safety_settings
+    model_name="gemini-1.5-flash-001",
+    temperature=0,
+    max_output_tokens=1024,
 )
 
 
 template = ChatPromptTemplate.from_messages(
     [
-        ("system", """You are a conversational bot that produce recipes for users based
-        on a question."""),
-        MessagesPlaceholder(variable_name="messages")
+        (
+            "system",
+            """You are a conversational bot that produce recipes for users based
+        on a question.""",
+        ),
+        MessagesPlaceholder(variable_name="messages"),
     ]
 )
 

gemini/sample-apps/conversational-genai-app-template/app/eval/data/chats.yaml

@@ -39,4 +39,4 @@
   - type: human
     content: Those all sound great! I like the Burnt aubergine veggie chilli
   - type: ai
-    content: That's a great choice! I hope you enjoy it. 
+    content: That's a great choice! I hope you enjoy it.

gemini/sample-apps/conversational-genai-app-template/app/eval/utils.py

@@ -12,20 +12,21 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+from concurrent.futures import ThreadPoolExecutor
+from functools import partial
 import glob
 import logging
 import os
-from concurrent.futures import ThreadPoolExecutor
-from functools import partial
 from typing import Any, Callable, Dict, Iterator, List
 
 import nest_asyncio
 import pandas as pd
-import yaml
 from tqdm import tqdm
+import yaml
 
 nest_asyncio.apply()
 
+
 def load_chats(path: str) -> List[Dict[str, Any]]:
     """
     Loads a list of chats from a directory or file.
@@ -44,6 +45,7 @@ def load_chats(path: str) -> List[Dict[str, Any]]:
             chats = chats + chats_in_file
     return chats
 
+
 def pairwise(iterable: List[Any]) -> Iterator[tuple[Any, Any]]:
     """Creates an iterable with tuples paired together
     e.g s -> (s0, s1), (s2, s3), (s4, s5), ...
@@ -81,11 +83,9 @@ def generate_multiturn_history(df: pd.DataFrame) -> pd.DataFrame:
             message = {
                 "human_message": human_message,
                 "ai_message": ai_message,
-                "conversation_history": conversation_history
+                "conversation_history": conversation_history,
             }
-            conversation_history = conversation_history + [
-                human_message, ai_message
-            ]
+            conversation_history = conversation_history + [human_message, ai_message]
             processed_messages.append(message)
 
     return pd.DataFrame(processed_messages)
@@ -103,7 +103,7 @@ def generate_message(row: tuple[int, Dict[str, Any]], callable: Any) -> Dict[str
     Args:
         row (tuple[int, Dict[str, Any]]): A tuple containing the index and a dictionary
           with message data, including:
-                   - "conversation_history" (List[str]): Optional. List of previous 
+                   - "conversation_history" (List[str]): Optional. List of previous
                       messages
                         in the conversation.
                    - "human_message" (str): The current human message.
@@ -118,7 +118,9 @@ def generate_message(row: tuple[int, Dict[str, Any]], callable: Any) -> Dict[str
               - "response_obj" (Any): The usage metadata of the response from the callable.
     """
     index, message = row
-    messages = message["conversation_history"] if "conversation_history" in message else []
+    messages = (
+        message["conversation_history"] if "conversation_history" in message else []
+    )
     messages.append(message["human_message"])
     input_callable = {"messages": messages}
     response = callable.invoke(input_callable)
@@ -130,7 +132,7 @@ def generate_message(row: tuple[int, Dict[str, Any]], callable: Any) -> Dict[str
 def batch_generate_messages(
     messages: pd.DataFrame,
     callable: Callable[[List[Dict[str, Any]]], Dict[str, Any]],
-    max_workers: int = 4
+    max_workers: int = 4,
 ) -> pd.DataFrame:
     """Generates AI responses to user messages using a provided callable.
 
@@ -152,8 +154,8 @@ def batch_generate_messages(
             ]
             ```
 
-        callable (Callable[[List[Dict[str, Any]]], Dict[str, Any]]): Callable object 
-          (e.g., Langchain Chain) used
+        callable (Callable[[List[Dict[str, Any]]], Dict[str, Any]]): Callable object
+          (e.g., LangChain Chain) used
             for response generation. It should accept a list of message dictionaries
             (as described above) and return a dictionary with the following structure:
 
@@ -202,6 +204,7 @@ def batch_generate_messages(
             predicted_messages.append(message)
     return pd.DataFrame(predicted_messages)
 
+
 def save_df_to_csv(df: pd.DataFrame, dir_path: str, filename: str) -> None:
     """Saves a pandas DataFrame to directory as a CSV file.
 
@@ -233,7 +236,9 @@ def prepare_metrics(metrics: List[str]) -> List[Any]:
             *module_path, metric_name = metric.removeprefix("custom:").split(".")
             metrics_evaluation.append(
                 __import__(".".join(module_path), fromlist=[metric_name]).__dict__[
-                    metric_name])
+                    metric_name
+                ]
+            )
         else:
             metrics_evaluation.append(metric)
     return metrics_evaluation

gemini/sample-apps/conversational-genai-app-template/app/patterns/custom_rag_qa/chain.py

@@ -16,14 +16,13 @@
 import logging
 from typing import Any, Dict, Iterator
 
-import google
-import vertexai
-from langchain_google_community.vertex_rank import VertexAIRank
-from langchain_google_vertexai import ChatVertexAI, VertexAIEmbeddings
-
 from app.patterns.custom_rag_qa.templates import query_rewrite_template, rag_template
 from app.patterns.custom_rag_qa.vector_store import get_vector_store
 from app.utils.output_types import OnChatModelStreamEvent, OnToolEndEvent, custom_chain
+import google
+from langchain_google_community.vertex_rank import VertexAIRank
+from langchain_google_vertexai import ChatVertexAI, VertexAIEmbeddings
+import vertexai
 
 # Configuration
 EMBEDDING_MODEL = "text-embedding-004"
@@ -38,7 +37,7 @@
 embedding = VertexAIEmbeddings(model_name=EMBEDDING_MODEL)
 vector_store = get_vector_store(embedding=embedding)
 retriever = vector_store.as_retriever(search_kwargs={"k": 20})
-reranker = VertexAIRank(
+compressor = VertexAIRank(
     project_id=project_id,
     location_id="global",
     ranking_config="default_ranking_config",
@@ -52,23 +51,25 @@
 
 
 @custom_chain
-def chain(input: Dict[str, Any], **kwargs: Any) -> Iterator[OnToolEndEvent | OnChatModelStreamEvent]:
+def chain(
+    input: Dict[str, Any], **kwargs: Any
+) -> Iterator[OnToolEndEvent | OnChatModelStreamEvent]:
     """
-    Implements a RAG QA chain. Decorated with `custom_chain` to offer Langchain compatible astream_events
+    Implements a RAG QA chain. Decorated with `custom_chain` to offer LangChain compatible astream_events
     and invoke interface and OpenTelemetry tracing.
     """
     # Generate optimized query
     query = query_gen.invoke(input).content
 
     # Retrieve and rank documents
     retrieved_docs = retriever.invoke(query)
-    ranked_docs = reranker.compress_documents(documents=retrieved_docs, query=query)
+    ranked_docs = compressor.compress_documents(documents=retrieved_docs, query=query)
 
     # Yield tool results metadata
     yield OnToolEndEvent(data={"input": {"query": query}, "output": ranked_docs})
 
     # Stream LLM response
     for chunk in response_chain.stream(
-            input={"messages": input["messages"], "relevant_documents": ranked_docs}
+        input={"messages": input["messages"], "relevant_documents": ranked_docs}
     ):
-        yield OnChatModelStreamEvent(data={"chunk": chunk})
+        yield OnChatModelStreamEvent(data={"chunk": chunk})

gemini/sample-apps/conversational-genai-app-template/app/patterns/custom_rag_qa/templates.py

@@ -14,14 +14,22 @@
 
 from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
 
-query_rewrite_template = ChatPromptTemplate.from_messages([
-    ("system", "Rewrite a query to a semantic search engine using the current conversation. "
-               "Provide only the rewritten query as output."),
-    MessagesPlaceholder(variable_name="messages")
-])
+query_rewrite_template = ChatPromptTemplate.from_messages(
+    [
+        (
+            "system",
+            "Rewrite a query to a semantic search engine using the current conversation. "
+            "Provide only the rewritten query as output.",
+        ),
+        MessagesPlaceholder(variable_name="messages"),
+    ]
+)
 
-rag_template = ChatPromptTemplate.from_messages([
-    ("system", """You are an AI assistant for question-answering tasks. Follow these guidelines:
+rag_template = ChatPromptTemplate.from_messages(
+    [
+        (
+            "system",
+            """You are an AI assistant for question-answering tasks. Follow these guidelines:
 1. Use only the provided context to answer the question.
 2. Give clear, accurate responses based on the information available.
 3. If the context is insufficient, state: "I don't have enough information to answer this question."
@@ -39,6 +47,9 @@
 {{ doc.page_content | safe }}
 </Document {{ loop.index0 }}>
 {% endfor %}
-"""),
-    MessagesPlaceholder(variable_name="messages")
-], template_format="jinja2")
+""",
+        ),
+        MessagesPlaceholder(variable_name="messages"),
+    ],
+    template_format="jinja2",
+)

gemini/sample-apps/conversational-genai-app-template/app/patterns/custom_rag_qa/vector_store.py

@@ -40,8 +40,8 @@ def load_and_split_documents(url: str) -> List[Document]:
 
 
 def get_vector_store(
-        embedding: Embeddings, persist_path: str = PERSIST_PATH, url: str = URL
-    ) -> SKLearnVectorStore:
+    embedding: Embeddings, persist_path: str = PERSIST_PATH, url: str = URL
+) -> SKLearnVectorStore:
     """Get or create a vector store."""
     vector_store = SKLearnVectorStore(embedding=embedding, persist_path=persist_path)