From 9b92093b00b9c4e15c3dfc91f68b2a6ffb80191e Mon Sep 17 00:00:00 2001 From: Alex Reibman Date: Fri, 21 Jun 2024 20:05:11 -0700 Subject: [PATCH] LLM Observability documentation fixes: Broken links, grammar, and spelling (#2995) * update markdown hyperlinks to stable urls * update notebook images and text * re-write observability section * Updated section * update wording * added newline * update styling in image tags to be jsx compatible * added text * update link * simplified text --------- Co-authored-by: Braelyn Boynton --- notebook/agentchat_agentops.ipynb | 185 ++++++++++++----------- website/docs/ecosystem/agentops.md | 28 ++-- website/docs/topics/llm-observability.md | 55 +++---- 3 files changed, 138 insertions(+), 130 deletions(-) diff --git a/notebook/agentchat_agentops.ipynb b/notebook/agentchat_agentops.ipynb index 293efa8e4bd..71106e45d3c 100644 --- a/notebook/agentchat_agentops.ipynb +++ b/notebook/agentchat_agentops.ipynb @@ -7,7 +7,7 @@ "collapsed": false }, "source": [ - "# AgentOps" + "# Agent Tracking with AgentOps" ] }, { @@ -15,7 +15,7 @@ "id": "a447802c88c8a240", "metadata": {}, "source": [ - "![logo](https://raw.githubusercontent.com/AgentOps-AI/agentops/35d5682866921a9e28d8ef66ae3c3b3d92d8fa6b/img/logo.png)\n", + "\n", "\n", "[AgentOps](https://agentops.ai/?=autogen) provides session replays, metrics, and monitoring for AI agents.\n", "\n", @@ -27,8 +27,11 @@ "id": "b354c068", "metadata": {}, "source": [ - "### Dashboard\n", - "![Agent Dashboard](https://github.com/AgentOps-AI/agentops/assets/14807319/158e082a-9a7d-49b7-9b41-51a49a1f7d3d)" + "### Overview Dashboard\n", + "\n", + "\n", + "### Session Replays\n", + "" ] }, { @@ -39,7 +42,7 @@ "## Adding AgentOps to an existing Autogen service.\n", "To get started, you'll need to install the AgentOps package and set an API key.\n", "\n", - "AgentOps automatically configures itself when it's initialized. This means your agents will be tracked and logged to your AgentOps account right away." + "AgentOps automatically configures itself when it's initialized meaning your agent run data will be tracked and logged to your AgentOps account right away." ] }, { @@ -69,7 +72,7 @@ "\n", "By default, the AgentOps `init()` function will look for an environment variable named `AGENTOPS_API_KEY`. Alternatively, you can pass one in as an optional parameter.\n", "\n", - "Create an account and API key at [AgentOps.ai](https://agentops.ai/)" + "Create an account and obtain an API key at [AgentOps.ai](https://agentops.ai/settings/projects)" ] }, { @@ -87,12 +90,14 @@ "name": "stderr", "output_type": "stream", "text": [ - "πŸ–‡ AgentOps: \u001B[34m\u001B[34mSession Replay: https://app.agentops.ai/drilldown?session_id=8bfaeed1-fd51-4c68-b3ec-276b1a3ce8a4\u001B[0m\u001B[0m\n" + "πŸ–‡ AgentOps: \u001b[34m\u001b[34mSession Replay: https://app.agentops.ai/drilldown?session_id=8bfaeed1-fd51-4c68-b3ec-276b1a3ce8a4\u001b[0m\u001b[0m\n" ] }, { "data": { - "text/plain": "UUID('8bfaeed1-fd51-4c68-b3ec-276b1a3ce8a4')" + "text/plain": [ + "UUID('8bfaeed1-fd51-4c68-b3ec-276b1a3ce8a4')" + ] }, "execution_count": 1, "metadata": {}, @@ -104,7 +109,7 @@ "\n", "from autogen import ConversableAgent, UserProxyAgent, config_list_from_json\n", "\n", - "agentops.init(api_key=\"7c94212b-b89d-47a6-a20c-23b2077d3226\") # or agentops.init(api_key=\"...\")" + "agentops.init(api_key=\"...\")" ] }, { @@ -144,19 +149,19 @@ "name": "stdout", "output_type": "stream", "text": [ - "\u001B[33magent\u001B[0m (to user):\n", + "\u001b[33magent\u001b[0m (to user):\n", "\n", "How can I help you today?\n", "\n", "--------------------------------------------------------------------------------\n", - "\u001B[33muser\u001B[0m (to agent):\n", + "\u001b[33muser\u001b[0m (to agent):\n", "\n", "2+2\n", "\n", "--------------------------------------------------------------------------------\n", - "\u001B[31m\n", - ">>>>>>>> USING AUTO REPLY...\u001B[0m\n", - "\u001B[33magent\u001B[0m (to user):\n", + "\u001b[31m\n", + ">>>>>>>> USING AUTO REPLY...\u001b[0m\n", + "\u001b[33magent\u001b[0m (to user):\n", "\n", "2 + 2 equals 4.\n", "\n", @@ -168,7 +173,7 @@ "output_type": "stream", "text": [ "πŸ–‡ AgentOps: This run's cost $0.000960\n", - "πŸ–‡ AgentOps: \u001B[34m\u001B[34mSession Replay: https://app.agentops.ai/drilldown?session_id=8bfaeed1-fd51-4c68-b3ec-276b1a3ce8a4\u001B[0m\u001B[0m\n" + "πŸ–‡ AgentOps: \u001b[34m\u001b[34mSession Replay: https://app.agentops.ai/drilldown?session_id=8bfaeed1-fd51-4c68-b3ec-276b1a3ce8a4\u001b[0m\u001b[0m\n" ] } ], @@ -185,7 +190,7 @@ "# Create the agent that represents the user in the conversation.\n", "user_proxy = UserProxyAgent(\"user\", code_execution_config=False)\n", "\n", - "# Let the assistant start the conversation. It will end when the user types exit.\n", + "# Let the assistant start the conversation. It will end when the user types \"exit\".\n", "assistant.initiate_chat(user_proxy, message=\"How can I help you today?\")\n", "\n", "# Close your AgentOps session to indicate that it completed.\n", @@ -204,13 +209,13 @@ }, { "cell_type": "markdown", - "source": [ - "![Session_Overview](https://github.com/AgentOps-AI/agentops/assets/14807319/d7228019-1488-40d3-852f-a61e998658ad)" - ], + "id": "cbd689b0f5617013", "metadata": { "collapsed": false }, - "id": "cbd689b0f5617013" + "source": [ + "![session replay](https://github.com/AgentOps-AI/agentops/blob/main/docs/images/external/app_screenshots/session-overview.png?raw=true)" + ] }, { "cell_type": "markdown", @@ -236,70 +241,70 @@ "name": "stderr", "output_type": "stream", "text": [ - "πŸ–‡ AgentOps: \u001B[34m\u001B[34mSession Replay: https://app.agentops.ai/drilldown?session_id=880c206b-751e-4c23-9313-8684537fc04d\u001B[0m\u001B[0m\n" + "πŸ–‡ AgentOps: \u001b[34m\u001b[34mSession Replay: https://app.agentops.ai/drilldown?session_id=880c206b-751e-4c23-9313-8684537fc04d\u001b[0m\u001b[0m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ - "\u001B[33mUser\u001B[0m (to Assistant):\n", + "\u001b[33mUser\u001b[0m (to Assistant):\n", "\n", "What is (1423 - 123) / 3 + (32 + 23) * 5?\n", "\n", "--------------------------------------------------------------------------------\n", - "\u001B[31m\n", - ">>>>>>>> USING AUTO REPLY...\u001B[0m\n", - "\u001B[33mAssistant\u001B[0m (to User):\n", + "\u001b[31m\n", + ">>>>>>>> USING AUTO REPLY...\u001b[0m\n", + "\u001b[33mAssistant\u001b[0m (to User):\n", "\n", - "\u001B[32m***** Suggested tool call (call_aINcGyo0Xkrh9g7buRuhyCz0): calculator *****\u001B[0m\n", + "\u001b[32m***** Suggested tool call (call_aINcGyo0Xkrh9g7buRuhyCz0): calculator *****\u001b[0m\n", "Arguments: \n", "{\n", " \"a\": 1423,\n", " \"b\": 123,\n", " \"operator\": \"-\"\n", "}\n", - "\u001B[32m***************************************************************************\u001B[0m\n", + "\u001b[32m***************************************************************************\u001b[0m\n", "\n", "--------------------------------------------------------------------------------\n", - "\u001B[35m\n", - ">>>>>>>> EXECUTING FUNCTION calculator...\u001B[0m\n", - "\u001B[33mUser\u001B[0m (to Assistant):\n", + "\u001b[35m\n", + ">>>>>>>> EXECUTING FUNCTION calculator...\u001b[0m\n", + "\u001b[33mUser\u001b[0m (to Assistant):\n", "\n", - "\u001B[33mUser\u001B[0m (to Assistant):\n", + "\u001b[33mUser\u001b[0m (to Assistant):\n", "\n", - "\u001B[32m***** Response from calling tool (call_aINcGyo0Xkrh9g7buRuhyCz0) *****\u001B[0m\n", + "\u001b[32m***** Response from calling tool (call_aINcGyo0Xkrh9g7buRuhyCz0) *****\u001b[0m\n", "1300\n", - "\u001B[32m**********************************************************************\u001B[0m\n", + "\u001b[32m**********************************************************************\u001b[0m\n", "\n", "--------------------------------------------------------------------------------\n", - "\u001B[31m\n", - ">>>>>>>> USING AUTO REPLY...\u001B[0m\n", - "\u001B[33mAssistant\u001B[0m (to User):\n", + "\u001b[31m\n", + ">>>>>>>> USING AUTO REPLY...\u001b[0m\n", + "\u001b[33mAssistant\u001b[0m (to User):\n", "\n", - "\u001B[32m***** Suggested tool call (call_prJGf8V0QVT7cbD91e0Fcxpb): calculator *****\u001B[0m\n", + "\u001b[32m***** Suggested tool call (call_prJGf8V0QVT7cbD91e0Fcxpb): calculator *****\u001b[0m\n", "Arguments: \n", "{\n", " \"a\": 1300,\n", " \"b\": 3,\n", " \"operator\": \"/\"\n", "}\n", - "\u001B[32m***************************************************************************\u001B[0m\n", + "\u001b[32m***************************************************************************\u001b[0m\n", "\n", "--------------------------------------------------------------------------------\n", - "\u001B[35m\n", - ">>>>>>>> EXECUTING FUNCTION calculator...\u001B[0m\n", - "\u001B[33mUser\u001B[0m (to Assistant):\n", + "\u001b[35m\n", + ">>>>>>>> EXECUTING FUNCTION calculator...\u001b[0m\n", + "\u001b[33mUser\u001b[0m (to Assistant):\n", "\n", - "\u001B[33mUser\u001B[0m (to Assistant):\n", + "\u001b[33mUser\u001b[0m (to Assistant):\n", "\n", - "\u001B[32m***** Response from calling tool (call_prJGf8V0QVT7cbD91e0Fcxpb) *****\u001B[0m\n", + "\u001b[32m***** Response from calling tool (call_prJGf8V0QVT7cbD91e0Fcxpb) *****\u001b[0m\n", "433\n", - "\u001B[32m**********************************************************************\u001B[0m\n", + "\u001b[32m**********************************************************************\u001b[0m\n", "\n", "--------------------------------------------------------------------------------\n", - "\u001B[31m\n", - ">>>>>>>> USING AUTO REPLY...\u001B[0m\n" + "\u001b[31m\n", + ">>>>>>>> USING AUTO REPLY...\u001b[0m\n" ] }, { @@ -316,94 +321,94 @@ "name": "stdout", "output_type": "stream", "text": [ - "\u001B[33mAssistant\u001B[0m (to User):\n", + "\u001b[33mAssistant\u001b[0m (to User):\n", "\n", - "\u001B[32m***** Suggested tool call (call_CUIgHRsySLjayDKuUphI1TGm): calculator *****\u001B[0m\n", + "\u001b[32m***** Suggested tool call (call_CUIgHRsySLjayDKuUphI1TGm): calculator *****\u001b[0m\n", "Arguments: \n", "{\n", " \"a\": 32,\n", " \"b\": 23,\n", " \"operator\": \"+\"\n", "}\n", - "\u001B[32m***************************************************************************\u001B[0m\n", + "\u001b[32m***************************************************************************\u001b[0m\n", "\n", "--------------------------------------------------------------------------------\n", - "\u001B[35m\n", - ">>>>>>>> EXECUTING FUNCTION calculator...\u001B[0m\n", - "\u001B[33mUser\u001B[0m (to Assistant):\n", + "\u001b[35m\n", + ">>>>>>>> EXECUTING FUNCTION calculator...\u001b[0m\n", + "\u001b[33mUser\u001b[0m (to Assistant):\n", "\n", - "\u001B[33mUser\u001B[0m (to Assistant):\n", + "\u001b[33mUser\u001b[0m (to Assistant):\n", "\n", - "\u001B[32m***** Response from calling tool (call_CUIgHRsySLjayDKuUphI1TGm) *****\u001B[0m\n", + "\u001b[32m***** Response from calling tool (call_CUIgHRsySLjayDKuUphI1TGm) *****\u001b[0m\n", "55\n", - "\u001B[32m**********************************************************************\u001B[0m\n", + "\u001b[32m**********************************************************************\u001b[0m\n", "\n", "--------------------------------------------------------------------------------\n", - "\u001B[31m\n", - ">>>>>>>> USING AUTO REPLY...\u001B[0m\n", - "\u001B[33mAssistant\u001B[0m (to User):\n", + "\u001b[31m\n", + ">>>>>>>> USING AUTO REPLY...\u001b[0m\n", + "\u001b[33mAssistant\u001b[0m (to User):\n", "\n", - "\u001B[32m***** Suggested tool call (call_L7pGtBLUf9V0MPL90BASyesr): calculator *****\u001B[0m\n", + "\u001b[32m***** Suggested tool call (call_L7pGtBLUf9V0MPL90BASyesr): calculator *****\u001b[0m\n", "Arguments: \n", "{\n", " \"a\": 55,\n", " \"b\": 5,\n", " \"operator\": \"*\"\n", "}\n", - "\u001B[32m***************************************************************************\u001B[0m\n", + "\u001b[32m***************************************************************************\u001b[0m\n", "\n", "--------------------------------------------------------------------------------\n", - "\u001B[35m\n", - ">>>>>>>> EXECUTING FUNCTION calculator...\u001B[0m\n", - "\u001B[33mUser\u001B[0m (to Assistant):\n", + "\u001b[35m\n", + ">>>>>>>> EXECUTING FUNCTION calculator...\u001b[0m\n", + "\u001b[33mUser\u001b[0m (to Assistant):\n", "\n", - "\u001B[33mUser\u001B[0m (to Assistant):\n", + "\u001b[33mUser\u001b[0m (to Assistant):\n", "\n", - "\u001B[32m***** Response from calling tool (call_L7pGtBLUf9V0MPL90BASyesr) *****\u001B[0m\n", + "\u001b[32m***** Response from calling tool (call_L7pGtBLUf9V0MPL90BASyesr) *****\u001b[0m\n", "275\n", - "\u001B[32m**********************************************************************\u001B[0m\n", + "\u001b[32m**********************************************************************\u001b[0m\n", "\n", "--------------------------------------------------------------------------------\n", - "\u001B[31m\n", - ">>>>>>>> USING AUTO REPLY...\u001B[0m\n", - "\u001B[33mAssistant\u001B[0m (to User):\n", + "\u001b[31m\n", + ">>>>>>>> USING AUTO REPLY...\u001b[0m\n", + "\u001b[33mAssistant\u001b[0m (to User):\n", "\n", - "\u001B[32m***** Suggested tool call (call_Ygo6p4XfcxRjkYBflhG3UVv6): calculator *****\u001B[0m\n", + "\u001b[32m***** Suggested tool call (call_Ygo6p4XfcxRjkYBflhG3UVv6): calculator *****\u001b[0m\n", "Arguments: \n", "{\n", " \"a\": 433,\n", " \"b\": 275,\n", " \"operator\": \"+\"\n", "}\n", - "\u001B[32m***************************************************************************\u001B[0m\n", + "\u001b[32m***************************************************************************\u001b[0m\n", "\n", "--------------------------------------------------------------------------------\n", - "\u001B[35m\n", - ">>>>>>>> EXECUTING FUNCTION calculator...\u001B[0m\n", - "\u001B[33mUser\u001B[0m (to Assistant):\n", + "\u001b[35m\n", + ">>>>>>>> EXECUTING FUNCTION calculator...\u001b[0m\n", + "\u001b[33mUser\u001b[0m (to Assistant):\n", "\n", - "\u001B[33mUser\u001B[0m (to Assistant):\n", + "\u001b[33mUser\u001b[0m (to Assistant):\n", "\n", - "\u001B[32m***** Response from calling tool (call_Ygo6p4XfcxRjkYBflhG3UVv6) *****\u001B[0m\n", + "\u001b[32m***** Response from calling tool (call_Ygo6p4XfcxRjkYBflhG3UVv6) *****\u001b[0m\n", "708\n", - "\u001B[32m**********************************************************************\u001B[0m\n", + "\u001b[32m**********************************************************************\u001b[0m\n", "\n", "--------------------------------------------------------------------------------\n", - "\u001B[31m\n", - ">>>>>>>> USING AUTO REPLY...\u001B[0m\n", - "\u001B[33mAssistant\u001B[0m (to User):\n", + "\u001b[31m\n", + ">>>>>>>> USING AUTO REPLY...\u001b[0m\n", + "\u001b[33mAssistant\u001b[0m (to User):\n", "\n", "The result of the calculation is 708.\n", "\n", "--------------------------------------------------------------------------------\n", - "\u001B[33mUser\u001B[0m (to Assistant):\n", + "\u001b[33mUser\u001b[0m (to Assistant):\n", "\n", "\n", "\n", "--------------------------------------------------------------------------------\n", - "\u001B[31m\n", - ">>>>>>>> USING AUTO REPLY...\u001B[0m\n", - "\u001B[33mAssistant\u001B[0m (to User):\n", + "\u001b[31m\n", + ">>>>>>>> USING AUTO REPLY...\u001b[0m\n", + "\u001b[33mAssistant\u001b[0m (to User):\n", "\n", "TERMINATE\n", "\n", @@ -415,7 +420,7 @@ "output_type": "stream", "text": [ "πŸ–‡ AgentOps: This run's cost $0.001800\n", - "πŸ–‡ AgentOps: \u001B[34m\u001B[34mSession Replay: https://app.agentops.ai/drilldown?session_id=880c206b-751e-4c23-9313-8684537fc04d\u001B[0m\u001B[0m\n" + "πŸ–‡ AgentOps: \u001b[34m\u001b[34mSession Replay: https://app.agentops.ai/drilldown?session_id=880c206b-751e-4c23-9313-8684537fc04d\u001b[0m\u001b[0m\n" ] } ], @@ -474,7 +479,7 @@ " description=\"A simple calculator\", # A description of the tool.\n", ")\n", "\n", - "# Let the assistant start the conversation. It will end when the user types exit.\n", + "# Let the assistant start the conversation. It will end when the user types \"exit\".\n", "user_proxy.initiate_chat(assistant, message=\"What is (1423 - 123) / 3 + (32 + 23) * 5?\")\n", "\n", "agentops.end_session(\"Success\")" @@ -493,13 +498,13 @@ }, { "cell_type": "markdown", - "source": [ - "![Session Drilldown](https://github.com/AgentOps-AI/agentops/assets/14807319/561d59f3-c441-4066-914b-f6cfe32a598c)" - ], + "id": "a922a52ab5fce31", "metadata": { "collapsed": false }, - "id": "a922a52ab5fce31" + "source": [ + "![Session Drilldown](https://github.com/AgentOps-AI/agentops/blob/main/docs/images/external/app_screenshots/session-replay.png?raw=true)" + ] } ], "metadata": { diff --git a/website/docs/ecosystem/agentops.md b/website/docs/ecosystem/agentops.md index 76995b6eb5e..581fb2671e9 100644 --- a/website/docs/ecosystem/agentops.md +++ b/website/docs/ecosystem/agentops.md @@ -1,29 +1,37 @@ -# AgentOps πŸ–‡οΈ +# Agent Monitoring and Debugging with AgentOps -![logo](https://raw.githubusercontent.com/AgentOps-AI/agentops/35d5682866921a9e28d8ef66ae3c3b3d92d8fa6b/img/logo.png) +AgentOps logo -[AgentOps](https://agentops.ai/?=autogen) provides session replays, metrics, and monitoring for agents. +[AgentOps](https://agentops.ai/?=autogen) provides session replays, metrics, and monitoring for AI agents. At a high level, AgentOps gives you the ability to monitor LLM calls, costs, latency, agent failures, multi-agent interactions, tool usage, session-wide statistics, and more. For more info, check out the [AgentOps Repo](https://github.com/AgentOps-AI/agentops). +| | | +| ------------------------------------- | ------------------------------------------------------------- | +| πŸ“Š **Replay Analytics and Debugging** | Step-by-step agent execution graphs | +| πŸ’Έ **LLM Cost Management** | Track spend with LLM foundation model providers | +| πŸ§ͺ **Agent Benchmarking** | Test your agents against 1,000+ evals | +| πŸ” **Compliance and Security** | Detect common prompt injection and data exfiltration exploits | +| 🀝 **Framework Integrations** | Native Integrations with CrewAI, AutoGen, & LangChain | +
- Agent Dashboard + Agent Dashboard - Agent Dashboard + Agent Dashboard
- Session Analytics + Session Analytics - Session Analytics + Session Analytics
- Session Replays + Session Replays - Session Replays + Session Replays
@@ -38,7 +46,7 @@ pip install agentops ``` 2. **Create an API Key:** -Create a user API key here: [Create API Key](https://app.agentops.ai/account) +Create a user API key here: [Create API Key](https://app.agentops.ai/settings/projects) 3. **Configure Your Environment:** Add your API key to your environment variables diff --git a/website/docs/topics/llm-observability.md b/website/docs/topics/llm-observability.md index 6a95d185f97..f80b55ea098 100644 --- a/website/docs/topics/llm-observability.md +++ b/website/docs/topics/llm-observability.md @@ -1,42 +1,37 @@ -# LLM Observability +# Agent Observability -AutoGen supports advanced LLM observability and monitoring through built-in logging and partner providers. +AutoGen supports advanced LLM agent observability and monitoring through built-in logging and partner providers. -## What is LLM Observability -AI agent observability is the ability to monitor, measure, and understand the internal states and behaviors of AI agent systems. -Observability is crucial for ensuring transparency, reliability, and accountability in your agent systems. +## AutoGen Observability Integrations +### Built-In Logging +AutoGen's SQLite and File Logger - [Tutorial Notebook](/docs/notebooks/agentchat_logging) -## Development +### Full-Service Partner Integrations +AutoGen partners with [AgentOps](https://agentops.ai) to provide multi-agent tracking, metrics, and monitoring - [Tutorial Notebook](/docs/notebooks/agentchat_agentops) -### Agent Development in Terminal is Limited -- Lose track of what your agents did in between executions -- Parsing through terminal output searching for LLM completions -- Printing β€œtool called” -### Agent Development Dashboards Enable More -- Visual dashboard so you can see what your agents did in human-readable format -- LLM calls are magically recorded - prompt, completion, timestamps for each - with one line of code -- Agents and their events (including tool calls) are recorded with one more line of code -- Errors are magically associated to its causal event -- Record any other events to your session with two more lines of code -- Tons of other useful data if you’re developing with supported agent frameworks: SDK version +## What is Observability? +Observability provides developers with the necessary insights to understand and improve the internal workings of their agents. Observability is necessary for maintaining reliability, tracking costs, and ensuring AI safety. -## Compliance +**Without observability tools, developers face significant hurdles:** -Observability and monitoring is critical to ensure AI agent systems adhere to laws and regulations in industries like finance and healthcare, preventing violations such as data breaches and privacy issues. +- Tracking agent activities across sessions becomes a complex, error-prone task. +- Manually sifting through verbose terminal outputs to understand LLM interactions is inefficient. +- Pinpointing the exact moments of tool invocations is often like finding a needle in a haystack. -- Insights into AI decision-making, allowing organizations to explain outcomes and build trust with stakeholders. -- Helps detect anomalies and unintended behaviors early, mitigating operational, financial, and reputational risks. -- Ensures compliance with data privacy regulations, preventing unauthorized access and misuse of sensitive information. -- Quick identification and response to compliance violations, supporting incident analysis and prevention. -## Available Observability Integrations +**Key Features of Observability Dashboards:** +- Human-readable overview analytics and replays of agent activities. +- LLM cost, prompt, completion, timestamp, and metadata tracking for performance monitoring. +- Tool invocation, events, and agent-to-agent interactions for workflow monitoring. +- Error flagging and notifications for faster debugging. +- Access to a wealth of data for developers using supported agent frameworks, such as environments, SDK versions, and more. -### Logging -- Autogen SQLite and File Logger - [Tutorial](/docs/notebooks/agentchat_logging) +### Compliance -### Full-Service Partners -Autogen is currently partnered with [AgentOps](https://agentops.ai) for seamless observability integration. - -[Learn how to install AgentOps](/docs/notebooks/agentchat_agentops) +Observability is not just a development convenienceβ€”it's a compliance necessity, especially in regulated industries: +- It offers insights into AI decision-making processes, fostering trust and transparency. +- Anomalies and unintended behaviors are detected promptly, reducing various risks. +- Ensuring adherence to data privacy regulations, thereby safeguarding sensitive information. +- Compliance violations are quickly identified and addressed, enhancing incident management.