diff --git a/.chloggen/980.yaml b/.chloggen/980.yaml new file mode 100644 index 0000000000..e8bc588dfe --- /dev/null +++ b/.chloggen/980.yaml @@ -0,0 +1,4 @@ +change_type: breaking +component: gen_ai +note: Deprecate `gen_ai.prompt` and `gen_ai.completion` attributes, introduce log-based events for GenAI inputs and outputs. +issues: [834, 980] diff --git a/docs/attributes-registry/gen-ai.md b/docs/attributes-registry/gen-ai.md index 0dc935e462..12c5283980 100644 --- a/docs/attributes-registry/gen-ai.md +++ b/docs/attributes-registry/gen-ai.md @@ -14,34 +14,28 @@ This document defines the attributes used to describe telemetry in the context of Generative Artificial Intelligence (GenAI) Models requests and responses. -| Attribute | Type | Description | Examples | Stability | -| ---------------------------------- | -------- | ------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------- | ---------------------------------------------------------------- | -| `gen_ai.completion` | string | The full response received from the GenAI model. [1] | `[{'role': 'assistant', 'content': 'The capital of France is Paris.'}]` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `gen_ai.operation.name` | string | The name of the operation being performed. [2] | `chat`; `text_completion` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `gen_ai.prompt` | string | The full prompt sent to the GenAI model. [3] | `[{'role': 'user', 'content': 'What is the capital of France?'}]` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `gen_ai.request.frequency_penalty` | double | The frequency penalty setting for the GenAI request. | `0.1` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `gen_ai.request.max_tokens` | int | The maximum number of tokens the model generates for a request. | `100` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `gen_ai.request.model` | string | The name of the GenAI model a request is being made to. | `gpt-4` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `gen_ai.request.presence_penalty` | double | The presence penalty setting for the GenAI request. | `0.1` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `gen_ai.request.stop_sequences` | string[] | List of sequences that the model will use to stop generating further tokens. | `["forest", "lived"]` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `gen_ai.request.temperature` | double | The temperature setting for the GenAI request. | `0.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `gen_ai.request.top_k` | double | The top_k sampling setting for the GenAI request. | `1.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `gen_ai.request.top_p` | double | The top_p sampling setting for the GenAI request. | `1.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `gen_ai.response.finish_reasons` | string[] | Array of reasons the model stopped generating tokens, corresponding to each generation received. | `["stop"]`; `["stop", "length"]` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `gen_ai.response.id` | string | The unique identifier for the completion. | `chatcmpl-123` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `gen_ai.response.model` | string | The name of the model that generated the response. | `gpt-4-0613` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `gen_ai.system` | string | The Generative AI product as identified by the client or server instrumentation. [4] | `openai` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `gen_ai.token.type` | string | The type of token being counted. | `input`; `output` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `gen_ai.usage.input_tokens` | int | The number of tokens used in the GenAI input (prompt). | `100` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `gen_ai.usage.output_tokens` | int | The number of tokens used in the GenAI response (completion). | `180` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | - -**[1]:** It's RECOMMENDED to format completions as JSON string matching [OpenAI messages format](https://platform.openai.com/docs/guides/text-generation) - -**[2]:** If one of the predefined values applies, but specific system uses a different name it's RECOMMENDED to document it in the semantic conventions for specific GenAI system and use system-specific name in the instrumentation. If a different name is not documented, instrumentation libraries SHOULD use applicable predefined value. - -**[3]:** It's RECOMMENDED to format prompts as JSON string matching [OpenAI messages format](https://platform.openai.com/docs/guides/text-generation) - -**[4]:** The `gen_ai.system` describes a family of GenAI models with specific model identified +| Attribute | Type | Description | Examples | Stability | +| ---------------------------------- | -------- | ------------------------------------------------------------------------------------------------ | -------------------------------- | ---------------------------------------------------------------- | +| `gen_ai.operation.name` | string | The name of the operation being performed. [1] | `chat`; `text_completion` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `gen_ai.request.frequency_penalty` | double | The frequency penalty setting for the GenAI request. | `0.1` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `gen_ai.request.max_tokens` | int | The maximum number of tokens the model generates for a request. | `100` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `gen_ai.request.model` | string | The name of the GenAI model a request is being made to. | `gpt-4` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `gen_ai.request.presence_penalty` | double | The presence penalty setting for the GenAI request. | `0.1` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `gen_ai.request.stop_sequences` | string[] | List of sequences that the model will use to stop generating further tokens. | `["forest", "lived"]` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `gen_ai.request.temperature` | double | The temperature setting for the GenAI request. | `0.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `gen_ai.request.top_k` | double | The top_k sampling setting for the GenAI request. | `1.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `gen_ai.request.top_p` | double | The top_p sampling setting for the GenAI request. | `1.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `gen_ai.response.finish_reasons` | string[] | Array of reasons the model stopped generating tokens, corresponding to each generation received. | `["stop"]`; `["stop", "length"]` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `gen_ai.response.id` | string | The unique identifier for the completion. | `chatcmpl-123` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `gen_ai.response.model` | string | The name of the model that generated the response. | `gpt-4-0613` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `gen_ai.system` | string | The Generative AI product as identified by the client or server instrumentation. [2] | `openai` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `gen_ai.token.type` | string | The type of token being counted. | `input`; `output` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `gen_ai.usage.input_tokens` | int | The number of tokens used in the GenAI input (prompt). | `100` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `gen_ai.usage.output_tokens` | int | The number of tokens used in the GenAI response (completion). | `180` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | + +**[1]:** If one of the predefined values applies, but specific system uses a different name it's RECOMMENDED to document it in the semantic conventions for specific GenAI system and use system-specific name in the instrumentation. If a different name is not documented, instrumentation libraries SHOULD use applicable predefined value. + +**[2]:** The `gen_ai.system` describes a family of GenAI models with specific model identified by `gen_ai.request.model` and `gen_ai.response.model` attributes. The actual GenAI product may differ from the one identified by the client. @@ -104,7 +98,9 @@ Thie group defines attributes for OpenAI. Describes deprecated `gen_ai` attributes. -| Attribute | Type | Description | Examples | Stability | -| -------------------------------- | ---- | ----------------------------------------------------- | -------- | ------------------------------------------------------------------------------------------------------------------ | -| `gen_ai.usage.completion_tokens` | int | Deprecated, use `gen_ai.usage.output_tokens` instead. | `42` | ![Deprecated](https://img.shields.io/badge/-deprecated-red)
Replaced by `gen_ai.usage.output_tokens` attribute. | -| `gen_ai.usage.prompt_tokens` | int | Deprecated, use `gen_ai.usage.input_tokens` instead. | `42` | ![Deprecated](https://img.shields.io/badge/-deprecated-red)
Replaced by `gen_ai.usage.input_tokens` attribute. | +| Attribute | Type | Description | Examples | Stability | +| -------------------------------- | ------ | --------------------------------------------------------- | ----------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ | +| `gen_ai.completion` | string | Deprecated, use Event API to report completions contents. | `[{'role': 'assistant', 'content': 'The capital of France is Paris.'}]` | ![Deprecated](https://img.shields.io/badge/-deprecated-red)
Removed, no replacement at this time. | +| `gen_ai.prompt` | string | Deprecated, use Event API to report prompt contents. | `[{'role': 'user', 'content': 'What is the capital of France?'}]` | ![Deprecated](https://img.shields.io/badge/-deprecated-red)
Removed, no replacement at this time. | +| `gen_ai.usage.completion_tokens` | int | Deprecated, use `gen_ai.usage.output_tokens` instead. | `42` | ![Deprecated](https://img.shields.io/badge/-deprecated-red)
Replaced by `gen_ai.usage.output_tokens` attribute. | +| `gen_ai.usage.prompt_tokens` | int | Deprecated, use `gen_ai.usage.input_tokens` instead. | `42` | ![Deprecated](https://img.shields.io/badge/-deprecated-red)
Replaced by `gen_ai.usage.input_tokens` attribute. | diff --git a/docs/gen-ai/README.md b/docs/gen-ai/README.md index 086a6d327f..020fd0a4ca 100644 --- a/docs/gen-ai/README.md +++ b/docs/gen-ai/README.md @@ -16,6 +16,7 @@ use the conventions in limited non-critical workloads and share the feedback Semantic conventions for Generative AI operations are defined for the following signals: +* [Events](gen-ai-events.md): Semantic Conventions for Generative AI inputs and outputs - *events*. * [Metrics](gen-ai-metrics.md): Semantic Conventions for Generative AI operations - *metrics*. * [Spans](gen-ai-spans.md): Semantic Conventions for Generative AI requests - *spans*. diff --git a/docs/gen-ai/gen-ai-events.md b/docs/gen-ai/gen-ai-events.md new file mode 100644 index 0000000000..ea394a7c3c --- /dev/null +++ b/docs/gen-ai/gen-ai-events.md @@ -0,0 +1,383 @@ + + +# Semantic Conventions for GenAI events + +**Status**: [Experimental][DocumentStatus] + + + + + +- [Common attributes](#common-attributes) +- [System event](#system-event) +- [User event](#user-event) +- [Assistant event](#assistant-event) + - [`ToolCall` object](#toolcall-object) + - [`Function` object](#function-object) +- [Tool event](#tool-event) +- [Choice event](#choice-event) + - [`Message` object](#message-object) +- [Custom events](#custom-events) +- [Examples](#examples) + - [Chat completion](#chat-completion) + - [Tools](#tools) + - [Chat completion with multiple choices](#chat-completion-with-multiple-choices) + + + +GenAI instrumentations MAY capture user inputs sent to the model and responses received from it as [events](https://github.com/open-telemetry/opentelemetry-specification/tree/v1.33.0/specification/logs/event-api.md). + +> Note: +> Event API is experimental and not yet available in some languages. Check [spec-compliance matrix](https://github.com/open-telemetry/opentelemetry-specification/blob/main/spec-compliance-matrix.md#events) to see the implementation status in corresponding language. + +Instrumentations MAY capture inputs and outputs if and only if application has enabled the collection of this data. +This is for three primary reasons: + +1. Data privacy concerns. End users of GenAI applications may input sensitive information or personally identifiable information (PII) that they do not wish to be sent to a telemetry backend. +2. Data size concerns. Although there is no specified limit to sizes, there are practical limitations in programming languages and telemetry systems. Some GenAI systems allow for extremely large context windows that end users may take full advantage of. +3. Performance concerns. Sending large amounts of data to a telemetry backend may cause performance issues for the application. + +Body fields that contain user input, model output, or other potentially sensitive and verbose data +SHOULD NOT be captured by default. + +Semantic conventions for individual systems which extend content events SHOULD document all additional body fields and specify whether they +should be captured by default or need application to opt into capturing them. + +Telemetry consumers SHOULD expect to receive unknown body fields. + +Instrumentations SHOULD NOT capture undocumented body fields and MUST follow the documented defaults for known fields. +Instrumentations MAY offer configuration options allowing to disable events or allowing to capture all fields. + +## Common attributes + +The following attributes apply to all GenAI events. + + + + + + + + +| Attribute | Type | Description | Examples | [Requirement Level](https://opentelemetry.io/docs/specs/semconv/general/attribute-requirement-level/) | Stability | +|---|---|---|---|---|---| +| [`gen_ai.system`](/docs/attributes-registry/gen-ai.md) | string | The Generative AI product as identified by the client or server instrumentation. [1] | `openai` | `Recommended` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | + +**[1]:** The `gen_ai.system` describes a family of GenAI models with specific model identified +by `gen_ai.request.model` and `gen_ai.response.model` attributes. + +The actual GenAI product may differ from the one identified by the client. +For example, when using OpenAI client libraries to communicate with Mistral, the `gen_ai.system` +is set to `openai` based on the instrumentation's best knowledge. + +For custom model, a custom friendly name SHOULD be used. +If none of these options apply, the `gen_ai.system` SHOULD be set to `_OTHER`. + + + +`gen_ai.system` has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used. + +| Value | Description | Stability | +|---|---|---| +| `anthropic` | Anthropic | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `cohere` | Cohere | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `openai` | OpenAI | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `vertex_ai` | Vertex AI | ![Experimental](https://img.shields.io/badge/-experimental-blue) | + + + + + + + + +## System event + +This event describes the instructions passed to the GenAI model. + +The event name MUST be `gen_ai.system.message`. + +| Body Field | Type | Description | Examples | Requirement Level | +|---|---|---|---|---| +| `role` | string | The actual role of the message author as passed in the message. | `"system"`, `"instructions"` | `Conditionally Required`: if available and not equal to `system` | +| `content` | `AnyValue` | The contents of the system message. | `"You're a friendly bot that answers questions about OpenTelemetry."` | `Opt-In` | + +## User event + +This event describes the prompt message specified by the user. + +The event name MUST be `gen_ai.user.message`. + +| Body Field | Type | Description | Examples | Requirement Level | +|---|---|---|---|---| +| `role` | string | The actual role of the message author as passed in the message. | `"user"`, `"customer"` | `Conditionally Required`: if available and if not equal to `user` | +| `content` | `AnyValue` | The contents of the user message. | `What telemetry is reported by OpenAI instrumentations?` | `Opt-In` | + +## Assistant event + +This event describes the assistant message. + +The event name MUST be `gen_ai.assistant.message`. + +| Body Field | Type | Description | Examples | [Requirement Level](https://opentelemetry.io/docs/specs/semconv/general/attribute-requirement-level/) | +|--------------|--------------------------------|----------------------------------------|-------------------------------------------------|-------------------| +| `role` | string | The actual role of the message author as passed in the message. | `"assistant"`, `"bot"` | `Conditionally Required`: if available and if not equal to `assistant` | +| `content` | `AnyValue` | The contents of the assistant message. | `Spans, events, metrics defined by the GenAI semantic conventions.` | `Opt-In` | +| `tool_calls` | [ToolCall](#toolcall-object)[] | The tool calls generated by the model, such as function calls. | `[{"id":"call_mszuSIzqtI65i1wAUOE8w5H4", "function":{"name":"get_link_to_otel_semconv", "arguments":{"semconv":"gen_ai"}}, "type":"function"}]` | `Conditionally Required`: if available | + +### `ToolCall` object + +| Body Field | Type | Description | Examples | [Requirement Level](https://opentelemetry.io/docs/specs/semconv/general/attribute-requirement-level/) | +|------------|-----------------------------|------------------------------------|-------------------------------------------------|-------------------| +| `id` | string | The id of the tool call | `call_mszuSIzqtI65i1wAUOE8w5H4` | `Required` | +| `type` | string | The type of the tool | `function` | `Required` | +| `function` | [Function](#function-object)| The function that the model called | `{"name":"get_link_to_otel_semconv", "arguments":{"semconv":"gen_ai"}}` | `Required` | + +### `Function` object + +| Body Field | Type | Description | Examples | [Requirement Level](https://opentelemetry.io/docs/specs/semconv/general/attribute-requirement-level/) | +|-------------|------------|----------------------------------------|----------------------------|-------------------| +| `name` | string | The name of the function to call | `get_link_to_otel_semconv` | `Required` | +| `arguments` | `AnyValue` | The arguments to pass the the function | `{"semconv": "gen_ai"}` | `Opt-In` | + +## Tool event + +This event describes the output of the tool or function submitted back to the model. + +The event name MUST be `gen_ai.tool.message`. + +| Body Field | Type | Description | Examples | [Requirement Level](https://opentelemetry.io/docs/specs/semconv/general/attribute-requirement-level/) | +|----------------|--------|-----------------------------------------------|---------------------------------|-------------------| +| `role` | string | The actual role of the message author as passed in the message. | `"tool"`, `"function"` | `Conditionally Required`: if available and if not equal to `tool` | +| `content` | AnyValue | The contents of the tool message. | `opentelemetry.io` | `Opt-In` | +| `id` | string | Tool call that this message is responding to. | `call_mszuSIzqtI65i1wAUOE8w5H4` | `Required` | + +## Choice event + +This event describes model-generated individual chat response (choice). +If GenAI model returns multiple choices, each choice SHOULD be recorded as an individual event. + +When response is streamed, instrumentations that report response events MUST reconstruct and report the full message and MUST NOT report individual chunks as events. +If the request to GenAI model fails with an error before content is received, instrumentation SHOULD report an event with truncated content (if enabled). If `finish_reason` was not received, it MUST be set to `error`. + +The event name MUST be `gen_ai.choice`. + +Choice event body has the following fields: + +| Body Field | Type | Description | Examples | [Requirement Level](https://opentelemetry.io/docs/specs/semconv/general/attribute-requirement-level/) | +|-----------------|----------------------------|-------------------------------------------------|----------------------------------------|-------------------| +| `finish_reason` | string | The reason the model stopped generating tokens. | `stop`, `tool_calls`, `content_filter` | `Required` | +| `index` | int | The index of the choice in the list of choices. | `1` | `Required` | +| `message` | [Message](#message-object) | GenAI response message | `{"content":"The OpenAI semantic conventions are available at opentelemetry.io"}` | `Recommended` | + +### `Message` object + +| Body Field | Type | Description | Examples | [Requirement Level](https://opentelemetry.io/docs/specs/semconv/general/attribute-requirement-level/) | +|----------------|--------------------------------|-----------------------------------------------|---------------------------------|-------------------| +| `role` | string | The actual role of the message author as passed in the message. | `"assistant"`, `"bot"` | `Conditionally Required`: if available and if not equal to `assistant` | +| `content` | `AnyValue` | The contents of the assistant message. | `Spans, events, metrics defined by the GenAI semantic conventions.` | `Opt-In` | +| `tool_calls` | [ToolCall](#toolcall-object)[] | The tool calls generated by the model, such as function calls. | `[{"id":"call_mszuSIzqtI65i1wAUOE8w5H4", "function":{"name":"get_link_to_otel_semconv", "arguments":"{\"semconv\":\"gen_ai\"}"}, "type":"function"}]` | `Conditionally Required`: if available | + +## Custom events + +System-specific events that are not covered in this document SHOULD be documented in corresponding Semantic Conventions extensions and +SHOULD follow `gen_ai.{gen_ai.system}.*` naming pattern for system-specific events. + +## Examples + +### Chat completion + +This example covers the following scenario: + +- user requests chat completion from OpenAI GPT-4 model for the following prompt: + - System message: `You're a friendly bot that answers questions about OpenTelemetry.` + - User message: `How to instrument GenAI library with OTel?` + +- The model responds with `"Follow GenAI semantic conventions available at opentelemetry.io."` message + +Span: + +| Attribute name | Value | +|---------------------------------|--------------------------------------------| +| Span name | `"chat gpt-4"` | +| `gen_ai.system` | `"openai"` | +| `gen_ai.request.model` | `"gpt-4"` | +| `gen_ai.request.max_tokens` | `200` | +| `gen_ai.request.top_p` | `1.0` | +| `gen_ai.response.id` | `"chatcmpl-9J3uIL87gldCFtiIbyaOvTeYBRA3l"` | +| `gen_ai.response.model` | `"gpt-4-0613"` | +| `gen_ai.usage.output_tokens` | `47` | +| `gen_ai.usage.input_tokens` | `52` | +| `gen_ai.response.finish_reasons`| `["stop"]` | + +Events: + +1. `gen_ai.system.message`. + + | Property | Value | + |---------------------|-------------------------------------------------------| + | `gen_ai.system` | `"openai"` | + | Event body | `{"content": "You're a friendly bot that answers questions about OpenTelemetry."}` | + +2. `gen_ai.user.message` + + | Property | Value | + |---------------------|-------------------------------------------------------| + | `gen_ai.system` | `"openai"` | + | Event body | `{"content":"How to instrument GenAI library with OTel?"}` | + +3. `gen_ai.choice` + + | Property | Value | + |---------------------|-------------------------------------------------------| + | `gen_ai.system` | `"openai"` | + | Event body (with content enabled) | `{"index":0,"finish_reason":"stop","message":{"content":"Follow GenAI semantic conventions available at opentelemetry.io."}}` | + | Event body (without content) | `{"index":0,"finish_reason":"stop","message":{}}` | + +### Tools + +This example covers the following scenario: + +1. Application requests chat completion from OpenAI GPT-4 model and provides a function definition. + + - Application provides the following prompt: + - User message: `How to instrument GenAI library with OTel?` + - Application defines a tool (a function) names `get_link_to_otel_semconv` with single string argument named `semconv` + +2. The model responds with a tool call request which application executes +3. The application requests chat completion again now with the tool execution result + +Here's the telemetry generated for each step in this scenario: + +1. Chat completion resulting in a tool call. + + | Attribute name | Value | + |---------------------|-------------------------------------------------------| + | Span name | `"chat gpt-4"` | + | `gen_ai.system` | `"openai"` | + | `gen_ai.request.model`| `"gpt-4"` | + | `gen_ai.request.max_tokens`| `200` | + | `gen_ai.request.top_p`| `1.0` | + | `gen_ai.response.id`| `"chatcmpl-9J3uIL87gldCFtiIbyaOvTeYBRA3l"` | + | `gen_ai.response.model`| `"gpt-4-0613"` | + | `gen_ai.usage.output_tokens`| `17` | + | `gen_ai.usage.input_tokens`| `47` | + | `gen_ai.response.finish_reasons`| `["tool_calls"]` | + + Events parented to this span: + + - `gen_ai.user.message` (not reported when capturing content is disabled) + + | Property | Value | + |---------------------|-------------------------------------------------------| + | `gen_ai.system` | `"openai"` | + | Event body | `{"content":"How to instrument GenAI library with OTel?"}` | + + - `gen_ai.choice` + + | Property | Value | + |---------------------|-------------------------------------------------------| + | `gen_ai.system` | `"openai"` | + | Event body (with content) | `{"index":0,"finish_reason":"tool_calls","message":{"tool_calls":[{"id":"call_VSPygqKTWdrhaFErNvMV18Yl","function":{"name":"get_link_to_otel_semconv","arguments":"{\"semconv\":\"GenAI\"}"},"type":"function"}]}` | + | Event body (without content) | `{"index":0,"finish_reason":"tool_calls","message":{"tool_calls":[{"id":"call_VSPygqKTWdrhaFErNvMV18Yl","function":{"name":"get_link_to_otel_semconv"},"type":"function"}]}` | + +2. Application executes the tool call. Application may create span which is not covered by this semantic convention. +3. Final chat completion call + + | Attribute name | Value | + |---------------------------------|-------------------------------------------------------| + | Span name | `"chat gpt-4"` | + | `gen_ai.system` | `"openai"` | + | `gen_ai.request.model` | `"gpt-4"` | + | `gen_ai.request.max_tokens` | `200` | + | `gen_ai.request.top_p` | `1.0` | + | `gen_ai.response.id` | `"chatcmpl-call_VSPygqKTWdrhaFErNvMV18Yl"` | + | `gen_ai.response.model` | `"gpt-4-0613"` | + | `gen_ai.usage.output_tokens` | `52` | + | `gen_ai.usage.input_tokens` | `47` | + | `gen_ai.response.finish_reasons`| `["stop"]` | + + Events parented to this span: + (in this example, the event content matches the original messages, but applications may also drop messages or change their content) + + - `gen_ai.user.message` (not reported when capturing content is not enabled) + + | Property | Value | + |----------------------------------|------------------------------------------------------------| + | `gen_ai.system` | `"openai"` | + | Event body | `{"content":"How to instrument GenAI library with OTel?"}` | + + - `gen_ai.assistant.message` + + | Property | Value | + |----------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------| + | `gen_ai.system` | `"openai"` | + | Event body (content enabled) | `{"tool_calls":[{"id":"call_VSPygqKTWdrhaFErNvMV18Yl","function":{"name":"get_link_to_otel_semconv","arguments":"{\"semconv\":\"GenAI\"}"},"type":"function"}]}` | + | Event body (content not enabled) | `{"tool_calls":[{"id":"call_VSPygqKTWdrhaFErNvMV18Yl","function":{"name":"get_link_to_otel_semconv"},"type":"function"}]}` | + + - `gen_ai.tool.message` + + | Property | Value | + |----------------------------------|------------------------------------------------------------------------------------------------| + | `gen_ai.system` | `"openai"` | + | Event body (content enabled) | `{"content":"opentelemetry.io/semconv/gen-ai","id":"call_VSPygqKTWdrhaFErNvMV18Yl"}` | + | Event body (content not enabled) | `{"id":"call_VSPygqKTWdrhaFErNvMV18Yl"}` | + + - `gen_ai.choice` + + | Property | Value | + |----------------------------------|-------------------------------------------------------------------------------------------------------------------------------| + | `gen_ai.system` | `"openai"` | + | Event body (content enabled) | `{"index":0,"finish_reason":"stop","message":{"content":"Follow OTel semconv available at opentelemetry.io/semconv/gen-ai"}}` | + | Event body (content not enabled) | `{"index":0,"finish_reason":"stop","message":{}}` | + +### Chat completion with multiple choices + +This example covers the following scenario: + +- user requests 2 chat completion from OpenAI GPT-4 model for the following prompt: + + - System message: `You're a friendly bot that answers questions about OpenTelemetry.` + - User message: `How to instrument GenAI library with OTel?` + +- The model responds with two choices + + - `"Follow GenAI semantic conventions available at opentelemetry.io."` message + - `"Use OpenAI instrumentation library."` message + +Span: + +| Attribute name | Value | +|---------------------|--------------------------------------------| +| Span name | `"chat gpt-4"` | +| `gen_ai.system` | `"openai"` | +| `gen_ai.request.model`| `"gpt-4"` | +| `gen_ai.request.max_tokens`| `200` | +| `gen_ai.request.top_p`| `1.0` | +| `gen_ai.response.id`| `"chatcmpl-9J3uIL87gldCFtiIbyaOvTeYBRA3l"` | +| `gen_ai.response.model`| `"gpt-4-0613"` | +| `gen_ai.usage.output_tokens`| `77` | +| `gen_ai.usage.input_tokens`| `52` | +| `gen_ai.response.finish_reasons`| `["stop"]` | + +Events: + +1. `gen_ai.system.message`: the same as in the [Chat Completion](#chat-completion) example +2. `gen_ai.user.message`: the same as in the previous example +3. `gen_ai.choice` + + | Property | Value | + |------------------------------|-------------------------------------------------------| + | `gen_ai.system` | `"openai"` | + | Event body (content enabled) | `{"index":0,"finish_reason":"stop","message":{"content":"Follow GenAI semantic conventions available at opentelemetry.io."}}` | + +4. `gen_ai.choice` + + | Property | Value | + |------------------------------|-------------------------------------------------------| + | `gen_ai.system` | `"openai"` | + | Event body (content enabled) | `{"index":1,"finish_reason":"stop","message":{"content":"Use OpenAI instrumentation library."}}` | + +[DocumentStatus]: https://opentelemetry.io/docs/specs/otel/document-status diff --git a/docs/gen-ai/gen-ai-spans.md b/docs/gen-ai/gen-ai-spans.md index 0a3eec44b4..ed63699ae3 100644 --- a/docs/gen-ai/gen-ai-spans.md +++ b/docs/gen-ai/gen-ai-spans.md @@ -2,7 +2,7 @@ linkTitle: Generative AI traces ---> -# Semantic Conventions for GenAI operations +# Semantic Conventions for GenAI spans **Status**: [Experimental][DocumentStatus] @@ -11,9 +11,8 @@ linkTitle: Generative AI traces - [Name](#name) -- [Configuration](#configuration) - [GenAI attributes](#genai-attributes) -- [Events](#events) +- [Capturing inputs and outputs](#capturing-inputs-and-outputs) @@ -27,15 +26,6 @@ GenAI spans MUST follow the overall [guidelines for span names](https://github.c The **span name** SHOULD be `{gen_ai.operation.name} {gen_ai.request.model}`. Semantic conventions for individual GenAI systems and frameworks MAY specify different span name format. -## Configuration - -Instrumentations for Generative AI clients MAY capture prompts and completions. -Instrumentations that support it, MUST offer the ability to turn off capture of prompts and completions. This is for three primary reasons: - -1. Data privacy concerns. End users of GenAI applications may input sensitive information or personally identifiable information (PII) that they do not wish to be sent to a telemetry backend. -2. Data size concerns. Although there is no specified limit to sizes, there are practical limitations in programming languages and telemetry systems. Some GenAI systems allow for extremely large context windows that end users may take full advantage of. -3. Performance concerns. Sending large amounts of data to a telemetry backend may cause performance issues for the application. - ## GenAI attributes These attributes track input data and metadata for a request to an GenAI model. Each attribute represents a concept that is common to most Generative AI clients. @@ -125,54 +115,8 @@ Instrumentations SHOULD document the list of errors they report. -## Events - -In the lifetime of a GenAI span, an event for prompts sent and completions received MAY be created, depending on the configuration of the instrumentation. - - - - - - - - -The event name MUST be `gen_ai.content.prompt`. +## Capturing inputs and outputs -| Attribute | Type | Description | Examples | [Requirement Level](https://opentelemetry.io/docs/specs/semconv/general/attribute-requirement-level/) | Stability | -|---|---|---|---|---|---| -| [`gen_ai.prompt`](/docs/attributes-registry/gen-ai.md) | string | The full prompt sent to the GenAI model. [1] | `[{'role': 'user', 'content': 'What is the capital of France?'}]` | `Conditionally Required` if and only if corresponding event is enabled | ![Experimental](https://img.shields.io/badge/-experimental-blue) | - -**[1]:** It's RECOMMENDED to format prompts as JSON string matching [OpenAI messages format](https://platform.openai.com/docs/guides/text-generation) - - - - - - - - - - - - - - - - -The event name MUST be `gen_ai.content.completion`. - -| Attribute | Type | Description | Examples | [Requirement Level](https://opentelemetry.io/docs/specs/semconv/general/attribute-requirement-level/) | Stability | -|---|---|---|---|---|---| -| [`gen_ai.completion`](/docs/attributes-registry/gen-ai.md) | string | The full response received from the GenAI model. [1] | `[{'role': 'assistant', 'content': 'The capital of France is Paris.'}]` | `Conditionally Required` if and only if corresponding event is enabled | ![Experimental](https://img.shields.io/badge/-experimental-blue) | - -**[1]:** It's RECOMMENDED to format completions as JSON string matching [OpenAI messages format](https://platform.openai.com/docs/guides/text-generation) - - - - - - - - +User inputs and model responses may be recorded as events parented to GenAI operation span. See [Semantic Conventions for GenAI events](./gen-ai-events.md) for the details. -[DocumentStatus]: https://github.com/open-telemetry/opentelemetry-specification/tree/v1.22.0/specification/document-status.md +[DocumentStatus]: https://opentelemetry.io/docs/specs/otel/document-status diff --git a/model/gen-ai/deprecated/registry-deprecated.yaml b/model/gen-ai/deprecated/registry-deprecated.yaml index 04a2968a74..2115482f36 100644 --- a/model/gen-ai/deprecated/registry-deprecated.yaml +++ b/model/gen-ai/deprecated/registry-deprecated.yaml @@ -16,3 +16,15 @@ groups: deprecated: Replaced by `gen_ai.usage.output_tokens` attribute. brief: "Deprecated, use `gen_ai.usage.output_tokens` instead." examples: [42] + - id: gen_ai.prompt + type: string + stability: experimental + deprecated: "Removed, no replacement at this time." + brief: "Deprecated, use Event API to report prompt contents." + examples: ["[{'role': 'user', 'content': 'What is the capital of France?'}]"] + - id: gen_ai.completion + type: string + stability: experimental + deprecated: "Removed, no replacement at this time." + brief: "Deprecated, use Event API to report completions contents." + examples: ["[{'role': 'assistant', 'content': 'The capital of France is Paris.'}]"] diff --git a/model/gen-ai/events.yaml b/model/gen-ai/events.yaml new file mode 100644 index 0000000000..72a4e692f0 --- /dev/null +++ b/model/gen-ai/events.yaml @@ -0,0 +1,48 @@ +groups: + - id: gen_ai.common.event.attributes + type: attribute_group + stability: experimental + brief: > + Describes common Gen AI event attributes. + attributes: + - ref: gen_ai.system + + - id: gen_ai.system.message + name: gen_ai.system.message + type: event + stability: experimental + brief: > + This event describes the instructions passed to the GenAI system inside the prompt. + extends: gen_ai.common.event.attributes + + - id: gen_ai.user.message + name: gen_ai.user.message + type: event + stability: experimental + brief: > + This event describes the prompt message specified by the user. + extends: gen_ai.common.event.attributes + + - id: gen_ai.assistant.message + name: gen_ai.assistant.message + type: event + stability: experimental + brief: > + This event describes the assistant message passed to GenAI system or received from it. + extends: gen_ai.common.event.attributes + + - id: gen_ai.tool.message + name: gen_ai.tool.message + type: event + stability: experimental + brief: > + This event describes the tool or function response message. + extends: gen_ai.common.event.attributes + + - id: gen_ai.choice + name: gen_ai.choice + type: event + stability: experimental + brief: > + This event describes the Gen AI response message. + extends: gen_ai.common.event.attributes diff --git a/model/gen-ai/registry.yaml b/model/gen-ai/registry.yaml index 5b3d1cff79..816f457093 100644 --- a/model/gen-ai/registry.yaml +++ b/model/gen-ai/registry.yaml @@ -119,18 +119,6 @@ groups: brief: 'Output tokens (completion, response, etc.)' brief: The type of token being counted. examples: ['input', 'output'] - - id: gen_ai.prompt - stability: experimental - type: string - brief: The full prompt sent to the GenAI model. - note: It's RECOMMENDED to format prompts as JSON string matching [OpenAI messages format](https://platform.openai.com/docs/guides/text-generation) - examples: ["[{'role': 'user', 'content': 'What is the capital of France?'}]"] - - id: gen_ai.completion - stability: experimental - type: string - brief: The full response received from the GenAI model. - note: It's RECOMMENDED to format completions as JSON string matching [OpenAI messages format](https://platform.openai.com/docs/guides/text-generation) - examples: ["[{'role': 'assistant', 'content': 'The capital of France is Paris.'}]"] - id: gen_ai.operation.name stability: experimental type: diff --git a/model/gen-ai/spans.yaml b/model/gen-ai/spans.yaml index d634d94473..86ddfc4a82 100644 --- a/model/gen-ai/spans.yaml +++ b/model/gen-ai/spans.yaml @@ -54,35 +54,6 @@ groups: The `error.type` SHOULD match the error code returned by the Generative AI provider or the client library, the canonical name of exception that occurred, or another low-cardinality error identifier. Instrumentations SHOULD document the list of errors they report. - events: - - gen_ai.content.prompt - - gen_ai.content.completion - - - id: gen_ai.content.prompt - name: gen_ai.content.prompt - type: event - brief: > - In the lifetime of an GenAI span, events for prompts sent and completions received - may be created, depending on the configuration of the instrumentation. - attributes: - - ref: gen_ai.prompt - requirement_level: - conditionally_required: if and only if corresponding event is enabled - note: > - It's RECOMMENDED to format prompts as JSON string matching [OpenAI messages format](https://platform.openai.com/docs/guides/text-generation) - - - id: gen_ai.content.completion - name: gen_ai.content.completion - type: event - brief: > - In the lifetime of an GenAI span, events for prompts sent and completions received - may be created, depending on the configuration of the instrumentation. - attributes: - - ref: gen_ai.completion - requirement_level: - conditionally_required: if and only if corresponding event is enabled - note: > - It's RECOMMENDED to format completions as JSON string matching [OpenAI messages format](https://platform.openai.com/docs/guides/text-generation) - id: trace.gen_ai.client extends: trace.gen_ai.client.common