From cc65371ce5eeeceb8856efb31bcbc938e697994f Mon Sep 17 00:00:00 2001 From: Liudmila Molkova Date: Sun, 28 Apr 2024 16:35:06 -0700 Subject: [PATCH] make role optional --- docs/gen-ai/llm-spans.md | 64 +++++++++++++++++++++++----------------- 1 file changed, 37 insertions(+), 27 deletions(-) diff --git a/docs/gen-ai/llm-spans.md b/docs/gen-ai/llm-spans.md index 6ad95c08c9..7a3d052049 100644 --- a/docs/gen-ai/llm-spans.md +++ b/docs/gen-ai/llm-spans.md @@ -7,6 +7,7 @@ linkTitle: LLM requests **Status**: [Experimental][DocumentStatus] + - [Configuration](#configuration) @@ -99,6 +100,8 @@ Telemetry consumers SHOULD expect to receive unknown payload fields. ### System message This event describes the instructions passed to the GenAI model. +It's NOT RECOMMENDED to report this event when capturing sensitive content is disabled, unless there are additional system-specific properties +instrumentation adds to the event payload. The event name MUST be `gen_ai.system.message`. @@ -116,12 +119,14 @@ The event name MUST be `gen_ai.system.message`. | Body Field | Type | Description | Examples | Requirement Level | Sensitive | |---|---|---|---|---|---| -| `role` | string | The role of the messages author | `system` | `Required` | | +| `role` | string | The role of the messages author | `system` | `Opt-In` | | | `content` | string | The contents of the system message. | `"You're a friendly bot that answers questions about OpenTelemetry."` | `Opt-In` | ![Sensitive](https://img.shields.io/badge/-sensitive-red) | ### User message This event describes the prompt message specified by the user. +It's NOT RECOMMENDED to report this event when capturing sensitive content is disabled, unless there are additional system-specific properties +instrumentation adds to the event payload. The event name MUST be `gen_ai.user.message`. @@ -139,7 +144,7 @@ The event name MUST be `gen_ai.user.message`. | Body Field | Type | Description | Examples | Requirement Level | Sensitive | |---|---|---|---|---|---| -| `role` | string | The role of the messages author | `user` | `Required` | | +| `role` | string | The role of the messages author | `user` | `Opt-In` | | | `content` | `AnyValue` | The contents of the user message. | `What telemetry is reported by OpenAI instrumentations?` | `Opt-In` | ![Sensitive](https://img.shields.io/badge/-sensitive-red) | ### Assistant message @@ -162,9 +167,9 @@ The event name MUST be `gen_ai.assistant.message`. | Body Field | Type | Description | Examples | Requirement Level | Sensitive | |---|---|---|---|---|---| -| `role` | string | The role of the messages author | `assistant` | `Required` | | +| `role` | string | The role of the messages author | `assistant` | `Conditionally Required`: when the message is returned by GenAI model. | | | `content` | `AnyValue` | The contents of the assistant message. | `Spans, events, metrics defined by the GenAI semantic conventions.` | `Opt-In` | ![Sensitive](https://img.shields.io/badge/-sensitive-red) | -| `tool_calls` | [ToolCall](#toolcall-object)[] | The tool calls generated by the model, such as function calls. | `[{"id":"call_mszuSIzqtI65i1wAUOE8w5H4", "function":{"name":"get_link_to_otel_semconv", "arguments":"{\"semconv\":\"gen_ai\"}"}, "type":"function"}]` | `Conditionally Required: if available` | ![Mixed](https://img.shields.io/badge/-mixed-orange) | +| `tool_calls` | [ToolCall](#toolcall-object)[] | The tool calls generated by the model, such as function calls. | `[{"id":"call_mszuSIzqtI65i1wAUOE8w5H4", "function":{"name":"get_link_to_otel_semconv", "arguments":"{\"semconv\":\"gen_ai\"}"}, "type":"function"}]` | `Conditionally Required`: if available | ![Mixed](https://img.shields.io/badge/-mixed-orange) | #### `ToolCall` object @@ -201,7 +206,7 @@ The event name MUST be `gen_ai.tool.message`. | Body Field | Type | Description | Examples | Requirement Level | Sensitive | |---|---|---|---|---|---| -| `role` | string | The role of the messages author | `tool`, `function` | `Required` | | +| `role` | string | The role of the messages author | `tool`, `function` | `Conditionally Required`: when the message is returned by GenAI model. | | | `content` | string | The contents of the tool message. | `opentelemetry.io` | `Opt-In` | ![Sensitive](https://img.shields.io/badge/-sensitive-red) | | `tool_call_id` | string | Tool call that this message is responding to. | `call_mszuSIzqtI65i1wAUOE8w5H4` | `Required` | | @@ -240,8 +245,6 @@ Response event payload has the following fields: The message structure matches one of the messages defined in this document depending on the role: -- [System message](#system-message) -- [User message](#user-message) - [Assistant message](#assistant-message) - [Tool message](#tool-message) @@ -252,8 +255,10 @@ The message structure matches one of the messages defined in this document depen This example covers the following scenario: - user requests chat completion from OpenAI GPT-4 model for the following prompt: + - System message: `You're a friendly bot that answers questions about OpenTelemetry.` - User message: `How to instrument GenAI library with OTel?` + - The model responds with `"Follow GenAI semantic conventions available at opentelemetry.io."` message Span: @@ -267,35 +272,36 @@ Span: | `gen_ai.request.top_p`| `1.0` | | `gen_ai.response.id`| `"chatcmpl-9J3uIL87gldCFtiIbyaOvTeYBRA3l"` | | `gen_ai.response.model`| `"gpt-4-0613"` | -| `gen_ai.usage.completion_tokens`| `47` | -| `gen_ai.usage.prompt_tokens`| `52` | +| `gen_ai.usage.completion_tokens`| `47` | +| `gen_ai.usage.prompt_tokens`| `52` | | `gen_ai.response.finish_reason`| `"stop"` | Events: -1. `gen_ai.system.message` +1. `gen_ai.system.message`. + Not reported if when capturing sensitive content is not enabled. | Property | Value | |---------------------|-------------------------------------------------------| | `gen_ai.system` | `"openai"` | - | Event payload (full) | `{"role":"system","content":"You're a friendly bot that answers questions about OpenTelemetry."}` | - | Event payload (without sensitive content) | `{"role":"system", "content":"REDACTED"}` | + | Event payload | `{"content":"You're a friendly bot that answers questions about OpenTelemetry."}` | 2. `gen_ai.user.message` + Not reported if when capturing sensitive content is not enabled. | Property | Value | |---------------------|-------------------------------------------------------| | `gen_ai.system` | `"openai"` | - | Event payload (full) | `{"role":"user","content":"How to instrument GenAI library with OTel?"}` | - | Event payload (without sensitive content) | `{"role":"user", "content":"REDACTED"}` | + | Event payload | `{"content":"How to instrument GenAI library with OTel?"}` | 3. `gen_ai.choice` + May be reported if when capturing sensitive content is not enabled. | Property | Value | |---------------------|-------------------------------------------------------| | `gen_ai.system` | `"openai"` | - | Event payload (full) | `{"index":0,"finish_reason":"stop","message":{"role":"assistant","content":"Follow GenAI semantic conventions available at opentelemetry.io."}}` | - | Event payload (without sensitive content) | `{"index":0,"finish_reason":"stop","message":{"role":"assistant","content":"REDACTED"}}` | + | Event payload (with sensitive content) | `{"index":0,"finish_reason":"stop","message":{"role":"assistant","content":"Follow GenAI semantic conventions available at opentelemetry.io."}}` | + | Event payload (without sensitive content) | `{"index":0,"finish_reason":"stop","message":{"role":"assistant"}}` | ### Tools @@ -331,19 +337,20 @@ Here's the telemetry generated for each step in this scenario: Events parented to this span: - - `gen_ai.user.message` + - `gen_ai.user.message` (not reported when capturing sensitive content is disabled) | Property | Value | |---------------------|-------------------------------------------------------| | `gen_ai.system` | `"openai"` | - | Event payload | `{"role":"user","content":"How to instrument GenAI library with OTel?"}` | + | Event payload | `{"content":"How to instrument GenAI library with OTel?"}` | - - `gen_ai.choice` + - `gen_ai.choice` (not reported when capturing sensitive content is disabled) | Property | Value | |---------------------|-------------------------------------------------------| | `gen_ai.system` | `"openai"` | - | Event payload | `{"index":0,"finish_reason":"tool_calls","message":{"role":"assistant","tool_calls":[{"id":"call_VSPygqKTWdrhaFErNvMV18Yl","function":{"name":"get_link_to_otel_semconv","arguments":"{\"semconv\":\"GenAI\"}"},"type":"function"}]}` | + | Event payload (with sensitive content) | `{"index":0,"finish_reason":"tool_calls","message":{"role":"assistant","tool_calls":[{"id":"call_VSPygqKTWdrhaFErNvMV18Yl","function":{"name":"get_link_to_otel_semconv","arguments":"{\"semconv\":\"GenAI\"}"},"type":"function"}]}` | + | Event payload (withput sensitive content) | `{"index":0,"finish_reason":"tool_calls","message":{"role":"assistant","tool_calls":[{"id":"call_VSPygqKTWdrhaFErNvMV18Yl","function":{"name":"get_link_to_otel_semconv"},"type":"function"}]}` | 2. Application executes the tool call. Application may create span which is not covered by this semantic convention. 3. Final chat completion call @@ -364,32 +371,35 @@ Here's the telemetry generated for each step in this scenario: Events parented to this span: (in this example, the event content matches the original messages, but applications may also drop messages or change their content) - - `gen_ai.user.message` + - `gen_ai.user.message` (not reported when capturing sensitive content is disabled) | Property | Value | |---------------------|-------------------------------------------------------| | `gen_ai.system` | `"openai"` | | Event payload | `{"role":"user","content":"How to instrument GenAI library with OTel?"}` | - - `gen_ai.assistant.message` + - `gen_ai.assistant.message` (could be reported when capturing sensitive content is disabled) | Property | Value | |---------------------|-------------------------------------------------------| | `gen_ai.system` | `"openai"` | - | Event payload | `{"role":"assistant","tool_calls":[{"id":"call_VSPygqKTWdrhaFErNvMV18Yl","function":{"name":"get_link_to_otel_semconv","arguments":"{\"semconv\":\"GenAI\"}"},"type":"function"}]}` | + | Event payload (with sensitive content) | `{"role":"assistant","tool_calls":[{"id":"call_VSPygqKTWdrhaFErNvMV18Yl","function":{"name":"get_link_to_otel_semconv","arguments":"{\"semconv\":\"GenAI\"}"},"type":"function"}]}` | + | Event payload (without sensitive content) | `{"role":"assistant","tool_calls":[{"id":"call_VSPygqKTWdrhaFErNvMV18Yl","function":{"name":"get_link_to_otel_semconv","type":"function"}]}` | - - `gen_ai.tool.message` + - `gen_ai.tool.message` (could be reported when capturing sensitive content is disabled) | Property | Value | |---------------------|-------------------------------------------------------| | `gen_ai.system` | `"openai"` | - | Event payload | `{"role":"tool","content":"opentelemetry.io/semconv/gen-ai","tool_call_id":"call_VSPygqKTWdrhaFErNvMV18Yl"}` | + | Event payload (with sensitive content) | `{"role":"tool","content":"opentelemetry.io/semconv/gen-ai","tool_call_id":"call_VSPygqKTWdrhaFErNvMV18Yl"}` | + | Event payload (without sensitive content) | `{"role":"tool","tool_call_id":"call_VSPygqKTWdrhaFErNvMV18Yl"}` | - - `gen_ai.choice` + - `gen_ai.choice` (could be reported when capturing sensitive content is disabled) | Property | Value | |---------------------|-------------------------------------------------------| | `gen_ai.system` | `"openai"` | - | Event payload | `{"index":0,"finish_reason":"stop","message":{"role":"assistant","content":"Follow OTel semconv available at opentelemetry.io/semconv/gen-ai"}}` | + | Event payload (with sensitive content) | `{"index":0,"finish_reason":"stop","message":{"role":"assistant","content":"Follow OTel semconv available at opentelemetry.io/semconv/gen-ai"}}` | + | Event payload (without sensitive content) | `{"index":0,"finish_reason":"stop","message":{"role":"assistant"}}` | [DocumentStatus]: https://github.com/open-telemetry/opentelemetry-specification/tree/v1.31.0/specification/document-status.md