examples/bot_building_series/genai_agents_101.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Copyright 2024 Google LLC\n",
    "#\n",
    "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
    "# you may not use this file except in compliance with the License.\n",
    "# You may obtain a copy of the License at\n",
    "#\n",
    "#     https://www.apache.org/licenses/LICENSE-2.0\n",
    "#\n",
    "# Unless required by applicable law or agreed to in writing, software\n",
    "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
    "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
    "# See the License for the specific language governing permissions and\n",
    "# limitations under the License."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction\n",
    "In this notebook, we will show you use the Agent Builder SDK to build a Generative AI Agent + Tools.\n",
    "\n",
    "## Prerequisites\n",
    "- Ensure you have a GCP Service Account key with the Dialogflow API Admin privileges assigned to it.\n",
    "\n",
    "<table align=\"left\">\n",
    "  <td style=\"text-align: center\">\n",
    "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/dfcx-scrapi/blob/main/examples/bot_building_series/bot_building_101.ipynb\">\n",
    "      <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Google Colaboratory logo\"><br> Run in Colab\n",
    "    </a>\n",
    "  </td>\n",
    "  <td style=\"text-align: center\">\n",
    "    <a href=\"https://github.com/GoogleCloudPlatform/dfcx-scrapi/blob/main/examples/bot_building_series/bot_building_101.ipynb\">\n",
    "      <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\"><br> View on GitHub\n",
    "    </a>\n",
    "  </td>\n",
    "  <td style=\"text-align: center\">\n",
    "    <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/dfcx-scrapi/blob/main/examples/bot_building_series/bot_building_101.ipynb\">\n",
    "      <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\"><br> Open in Vertex AI Workbench\n",
    "    </a>\n",
    "  </td>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#If you haven't already, make sure you install the `dfcx-scrapi` library\n",
    "!pip install dfcx-scrapi\n",
    "\n",
    "import sys\n",
    "\n",
    "# Colab Auth needed to call Client Endpoints (i.e. vertexai)\n",
    "if \"google.colab\" in sys.modules:\n",
    "    # Authenticate user to Google Cloud\n",
    "    from google.colab import auth as google_auth\n",
    "    google_auth.authenticate_user()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Build a New Agent Application\n",
    "The first thing we will do is create a blank Agent Application.<br>\n",
    "This is the fundamental building block for chaining together Agents, Tools, and adding few-shot examples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "PROJECT_ID = \"<YOUR_GCP_PROJECT>\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from dfcx_scrapi.core.agents import Agents\n",
    "\n",
    "a = Agents()\n",
    "agent = a.create_agent(\n",
    "    project_id=PROJECT_ID,\n",
    "    display_name=\"SCRAPI AGENTS\",\n",
    "    gcp_region=\"us-east1\",\n",
    "    playbook_agent=True\n",
    ")\n",
    "\n",
    "panel = \"(playbooks/00000000-0000-0000-0000-000000000000/basics//right-panel:simulator)\"\n",
    "print(f\"AGENT LINK: https://vertexaiconversation.cloud.google.com/{agent.name}/{panel}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Test Your Agent\n",
    "When your Agent application is first created, there is a \"Default Generative Agent\" created, which has no `goal` or `instructions`.<br>\n",
    "However, you can still interact with it like a generic LLM model.\n",
    "\n",
    "One of the great aspects of Vertex Agent Builder is that it automatically handles Session Management for you!<br>\n",
    "All Agent Applications are ***immediately*** Production scalable and ready, because they are backed by production grade, scalable Google infrastructure.<br>\n",
    "\n",
    "Want to open you app to 100,000 users immediately?<br>\n",
    "We've got you covered! And no extra work on your end."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from dfcx_scrapi.core.sessions import Sessions\n",
    "\n",
    "s = Sessions()\n",
    "\n",
    "session_id = s.build_session_id(agent.name)\n",
    "session_id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "<font color='green'><b> USER QUERY:</font></b> hey, how are you?"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<font color='green'><b>AGENT RESPONSE:</font></b> I'm doing well, thank you! How can I help you today?"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "res = s.detect_intent(agent.name, session_id, \"hey, how are you?\")\n",
    "s.parse_result(res)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "<font color='green'><b> USER QUERY:</font></b> what kind of models do you have?"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<font color='green'><b>AGENT RESPONSE:</font></b> We have a variety of models to choose from. What type of vehicle are you interested in?"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "res = s.detect_intent(agent.name, session_id, \"what kind of models do you have?\")\n",
    "s.parse_result(res)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Update the Default Playbook\n",
    "Our Agent isn't very useful at this stage, so let's update the `goal` and `instructions` to do something more worthwhile!<br>\n",
    "\n",
    "We'll first fetch the handy `playbooks_map` so we can easily refernce IDs which are required by the API endpoints."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from dfcx_scrapi.core.playbooks import Playbooks\n",
    "\n",
    "p = Playbooks(agent.name)\n",
    "\n",
    "playbooks_map = p.get_playbooks_map(agent.name, reverse=True)\n",
    "playbooks_map"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "GOAL: Default goal\n",
      "INSTRUCTIONS: \n"
     ]
    }
   ],
   "source": [
    "# You see here that there are is no `goal` or `instruction` sets yet.\n",
    "playbook = p.get_playbook(playbooks_map[\"Default Generative Agent\"])\n",
    "print(f\"GOAL: {playbook.goal}\")\n",
    "print(f\"INSTRUCTIONS: {playbook.instruction}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "GOAL: You are a friendly Tesla service center agent.\n",
      "Your job is to help users book appointments and answer any questions they have.\n",
      "INSTRUCTIONS: steps {\n",
      "  text: \"Greet the user.\"\n",
      "}\n",
      "steps {\n",
      "  text: \"Answer any questions the have to the best of your ability.\"\n",
      "}\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# We'll provide a simple `goal` and `instruction` set to get started.\n",
    "\n",
    "playbook = p.update_playbook(\n",
    "    playbooks_map[\"Default Generative Agent\"],\n",
    "    goal=\"You are a friendly Tesla service center agent.\\nYour job is to help users book appointments and answer any questions they have.\",\n",
    "    instructions=[\"Greet the user.\", \"Answer any questions the have to the best of your ability.\"]\n",
    "    )\n",
    "\n",
    "print(f\"GOAL: {playbook.goal}\")\n",
    "print(f\"INSTRUCTIONS: {playbook.instruction}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test Agent Again\n",
    "Now that we've updated the `goal` and `instruction` set, let's see how our Agent application reacts."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "<font color='green'><b> USER QUERY:</font></b> what kind of models do you have?"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<font color='green'><b>AGENT RESPONSE:</font></b> We have a variety of models to choose from, including the Model S, Model 3, Model X, and Model Y. Which one are you interested in?"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from dfcx_scrapi.core.sessions import Sessions\n",
    "\n",
    "s = Sessions()\n",
    "\n",
    "session_id = s.build_session_id(agent.name)\n",
    "res = s.detect_intent(agent.name, session_id, \"what kind of models do you have?\")\n",
    "s.parse_result(res)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Voila! Now we're getting somewhere!<br>\n",
    "When we update our Agent application's `goal` and `instruction` sets, we are able to steer the Agent towards doing the task we design for it.<br>\n",
    "Notice the difference in responses when we ask the intentionally ambiguous question about \"models\".<br>\n",
    "\n",
    "We can continue to take this further by creating Tools for our Agent to use."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Create Tools\n",
    "Tools allow your Agent to interact with the outside world.<br>\n",
    "There are 3 primary tool types that you can use:\n",
    "- OpenAPI Spec\n",
    "- Functions\n",
    "- Data Stores\n",
    "\n",
    "In this next section, we'll create an `OpenAPI Spec` tool that can do the following:\n",
    "- Find the weather using the [National Weather Service Web API](https://www.weather.gov/documentation/services-web-api)\n",
    "\n",
    "For more information, see [Vertex Agents Tools](https://cloud.google.com/dialogflow/vertex/docs/concept/tools)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Deploy Cloud Run Endpoints\n",
    "Using the following code, you can deploy a Cloud Run endpoint to call the National Weather Service API."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%mkdir get_weather\n",
    "%cd get_weather"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Writing requirements.txt\n"
     ]
    }
   ],
   "source": [
    "%%writefile requirements.txt\n",
    "functions-framework==3.*\n",
    "firebase-admin==6.2.*\n",
    "firebase-functions\n",
    "requests==2.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!wget https://raw.githubusercontent.com/GoogleCloudPlatform/dfcx-scrapi/main/data/get_weather_tool.py\n",
    "!mv get_weather_tool.py main.py\n",
    "!gcloud config set project $PROJECT_ID\n",
    "!gcloud functions deploy get_weather --region \"us-central1\" --runtime python311 --trigger-http --entry-point main --source main.zip --no-allow-unauthenticated --gen2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define OpenAPI Specs\n",
    "Before proceeding, make sure you replace the URL in the `get_weather_spec` with the CLOUD RUN ENDPOINT that you just deployed!<br>\n",
    "Be sure to use the CLOUD RUN ENDPOINT, NOT the Cloud Function endpoint!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "get_weather_spec = \"\"\"\n",
    "openapi: 3.0.0\n",
    "info:\n",
    "  title: get_weather\n",
    "  version: 1.0.0\n",
    "\n",
    "servers:\n",
    "  - url: YOUR_ENDPOINT_HERE\n",
    "\n",
    "paths:\n",
    "  /get_weather_grid:\n",
    "    get:\n",
    "      summary: Returns the current grid information for a city and state\n",
    "      operationId: get_weather_grid\n",
    "      parameters:\n",
    "        - name: latitude\n",
    "          in: query\n",
    "          required: true\n",
    "          schema:\n",
    "            type: string\n",
    "        - name: longitude\n",
    "          in: query\n",
    "          required: true\n",
    "          schema:\n",
    "            type: string\n",
    "      responses:\n",
    "        '200':\n",
    "          description: OK\n",
    "          content:\n",
    "            application/json:\n",
    "              schema:\n",
    "                type: object\n",
    "                properties:\n",
    "                  data:\n",
    "                    type: string\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Call Create Tool\n",
    "For creating the `OpenAPI Spec` tool, we'll use the `build_open_api_tool` helper method to create the proper Tool object, then pass it to the `create_tool` method and capture the result."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "from dfcx_scrapi.core.tools import Tools\n",
    "\n",
    "t = Tools()\n",
    "\n",
    "# Tool 1 - get_weather\n",
    "weather_tool = t.build_open_api_tool(\n",
    "    \"get_weather\",\n",
    "    spec=get_weather_spec,\n",
    "    description=\"Get the current weather for the provided city.\"\n",
    "    )\n",
    "tool1 = t.create_tool(agent.name, weather_tool)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Create Weather Agent\n",
    "Now that we have 2 new tools to work with, let's apply them to a new Agent in our application."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from dfcx_scrapi.core.playbooks import Playbooks\n",
    "\n",
    "p = Playbooks(agent_id=agent.name)\n",
    "\n",
    "instructions = [\n",
    "    \"Use the ${TOOL:get_weather} to get the current city/state weather grid information.\",\n",
    "    \"- If the user only provides the city you can assume the state unless otherwise specified.\"]\n",
    "\n",
    "# New Playbook\n",
    "weather_agent = p.create_playbook(\n",
    "    agent.name,\n",
    "    display_name=\"Weather Agent\",\n",
    "    referenced_tools=[tool1.name],\n",
    "    goal=\"You are a senior weather advisor at a network broadcast station. Your job is to predict the weather!\",\n",
    "    instructions=instructions\n",
    ")\n",
    "\n",
    "playbooks_map = p.get_playbooks_map(agent.name, reverse=True)\n",
    "playbooks_map"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Chain Agents\n",
    "In order for our Agents to work together properly, we need to chain them together and update the instruction set.\n",
    "\n",
    "Remember, our current architecture looks like this:\n",
    "- Default Generative Agent\n",
    "- Weather Agent\n",
    "\n",
    "These are mutually exclusive Agents that cannot interact yet.<br>\n",
    "We want our architecture to look like this instead:\n",
    "- Default Generative Agent -> Weather Agent\n",
    "\n",
    "In order to do this, let's update the instruction set on our `Default Generative Agent`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "p.update_playbook(\n",
    "    playbooks_map[\"Default Generative Agent\"],\n",
    "    instructions=[\"If the user needs help with the weather, call ${AGENT: Weather Agent}\"]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test Agent Again\n",
    "With our Agents chained together, let's test with a new query about weather in a specific city!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "<font color='green'><b> USER QUERY:</font></b> what is the weather like in Austin, Texas?"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<font color='dark red'>TOOL CALL:</font></b> get_weather_grid -> 30.2672"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<font color='yellow'>TOOL RESULT:</font></b> {'temperatureUnit': 'F', 'temperature': 77.0}"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<font color='green'><b>AGENT RESPONSE:</font></b> The current temperature in Austin, Texas is 77 degrees Fahrenheit."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from dfcx_scrapi.core.sessions import Sessions\n",
    "\n",
    "s = Sessions()\n",
    "\n",
    "session_id = s.build_session_id(agent.name)\n",
    "res = s.detect_intent(agent.name, session_id, \"what is the weather like in Austin, Texas?\")\n",
    "s.parse_result(res)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Multi Turn Session\n",
    "Example of a Multi Turn Session with some \"cleaner\" readability for colab testing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "--- TURN 1 --- \n",
      "\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "<font color='green'><b> USER QUERY:</font></b> What colors does the model 3 come in?"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<font color='green'><b>AGENT RESPONSE:</font></b> The Model 3 comes in 5 colors: black, white, blue, red, and gray."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "--- TURN 2 --- \n",
      "\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "<font color='green'><b> USER QUERY:</font></b> What's the weather like in Boston right now? I'm thinking about walking to the Tesla dealership."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<font color='dark red'>TOOL CALL:</font></b> get_weather_grid -> 42.3601"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<font color='yellow'>TOOL RESULT:</font></b> {'temperatureUnit': 'F', 'temperature': 76.0}"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<font color='green'><b>AGENT RESPONSE:</font></b> The current temperature in Boston is 76 degrees Fahrenheit."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "session_id = s.build_session_id(agent.name)\n",
    "\n",
    "conversation = [\n",
    "    \"What colors does the model 3 come in?\",\n",
    "    \"What's the weather like in Boston right now? I'm thinking about walking to the Tesla dealership.\"\n",
    "    ]\n",
    "\n",
    "i = 1\n",
    "for utterance in conversation:\n",
    "  print(f\"\\n--- TURN {i} --- \\n\")\n",
    "  res = s.detect_intent(agent.name, session_id, utterance)\n",
    "  s.parse_result(res)\n",
    "  i+=1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# GenAI Agents 101 End\n",
    "In this notebook we've shown how to do the following:\n",
    "- Create a new Agent Application\n",
    "- Update the default Agent instructions\n",
    "- Create a new Tool\n",
    "- Create a new Agent and apply the new Tool\n",
    "- Chain together our Agents + Tool to make a simple, cohesive application"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "a46639dc4ce59764707f7d29e7ddf4b543fd040f828fe9a3cb5baeb9324df219"
  },
  "kernelspec": {
   "display_name": "Python 3.8.10 64-bit ('scrapi': conda)",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}