Skip to content

Commit

Permalink
make role optional
Browse files Browse the repository at this point in the history
  • Loading branch information
@lmolkova
lmolkova committed Apr 28, 2024
1 parent 6ebc2e2 commit cc65371
Showing 1 changed file with 37 additions and 27 deletions.
64 changes: 37 additions & 27 deletions docs/gen-ai/llm-spans.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ linkTitle: LLM requests
**Status**: [Experimental][DocumentStatus]

<!-- Re-generate TOC with `markdown-toc --no-first-h1 -i` -->

<!-- toc -->

- [Configuration](#configuration)
Expand Down Expand Up @@ -99,6 +100,8 @@ Telemetry consumers SHOULD expect to receive unknown payload fields.
### System message

This event describes the instructions passed to the GenAI model.
It's NOT RECOMMENDED to report this event when capturing sensitive content is disabled, unless there are additional system-specific properties
instrumentation adds to the event payload.

<!-- semconv gen_ai.system.message(full) -->
The event name MUST be `gen_ai.system.message`.
Expand All @@ -116,12 +119,14 @@ The event name MUST be `gen_ai.system.message`.

| Body Field | Type | Description | Examples | Requirement Level | Sensitive |
|---|---|---|---|---|---|
| `role` | string | The role of the messages author | `system` | `Required` | |
| `role` | string | The role of the messages author | `system` | `Opt-In` | |
| `content` | string | The contents of the system message. | `"You're a friendly bot that answers questions about OpenTelemetry."` | `Opt-In` | ![Sensitive](https://img.shields.io/badge/-sensitive-red) |

### User message

This event describes the prompt message specified by the user.
It's NOT RECOMMENDED to report this event when capturing sensitive content is disabled, unless there are additional system-specific properties
instrumentation adds to the event payload.

<!-- semconv gen_ai.user.message(full) -->
The event name MUST be `gen_ai.user.message`.
Expand All @@ -139,7 +144,7 @@ The event name MUST be `gen_ai.user.message`.

| Body Field | Type | Description | Examples | Requirement Level | Sensitive |
|---|---|---|---|---|---|
| `role` | string | The role of the messages author | `user` | `Required` | |
| `role` | string | The role of the messages author | `user` | `Opt-In` | |
| `content` | `AnyValue` | The contents of the user message. | `What telemetry is reported by OpenAI instrumentations?` | `Opt-In` | ![Sensitive](https://img.shields.io/badge/-sensitive-red) |

### Assistant message
Expand All @@ -162,9 +167,9 @@ The event name MUST be `gen_ai.assistant.message`.

| Body Field | Type | Description | Examples | Requirement Level | Sensitive |
|---|---|---|---|---|---|
| `role` | string | The role of the messages author | `assistant` | `Required` | |
| `role` | string | The role of the messages author | `assistant` | `Conditionally Required`: when the message is returned by GenAI model. | |
| `content` | `AnyValue` | The contents of the assistant message. | `Spans, events, metrics defined by the GenAI semantic conventions.` | `Opt-In` | ![Sensitive](https://img.shields.io/badge/-sensitive-red) |
| `tool_calls` | [ToolCall](#toolcall-object)[] | The tool calls generated by the model, such as function calls. | `[{"id":"call_mszuSIzqtI65i1wAUOE8w5H4", "function":{"name":"get_link_to_otel_semconv", "arguments":"{\"semconv\":\"gen_ai\"}"}, "type":"function"}]` | `Conditionally Required: if available` | ![Mixed](https://img.shields.io/badge/-mixed-orange) |
| `tool_calls` | [ToolCall](#toolcall-object)[] | The tool calls generated by the model, such as function calls. | `[{"id":"call_mszuSIzqtI65i1wAUOE8w5H4", "function":{"name":"get_link_to_otel_semconv", "arguments":"{\"semconv\":\"gen_ai\"}"}, "type":"function"}]` | `Conditionally Required`: if available | ![Mixed](https://img.shields.io/badge/-mixed-orange) |

#### `ToolCall` object

Expand Down Expand Up @@ -201,7 +206,7 @@ The event name MUST be `gen_ai.tool.message`.

| Body Field | Type | Description | Examples | Requirement Level | Sensitive |
|---|---|---|---|---|---|
| `role` | string | The role of the messages author | `tool`, `function` | `Required` | |
| `role` | string | The role of the messages author | `tool`, `function` | `Conditionally Required`: when the message is returned by GenAI model. | |
| `content` | string | The contents of the tool message. | `opentelemetry.io` | `Opt-In` | ![Sensitive](https://img.shields.io/badge/-sensitive-red) |
| `tool_call_id` | string | Tool call that this message is responding to. | `call_mszuSIzqtI65i1wAUOE8w5H4` | `Required` | |

Expand Down Expand Up @@ -240,8 +245,6 @@ Response event payload has the following fields:

The message structure matches one of the messages defined in this document depending on the role:

- [System message](#system-message)
- [User message](#user-message)
- [Assistant message](#assistant-message)
- [Tool message](#tool-message)

Expand All @@ -252,8 +255,10 @@ The message structure matches one of the messages defined in this document depen
This example covers the following scenario:

- user requests chat completion from OpenAI GPT-4 model for the following prompt:

- System message: `You're a friendly bot that answers questions about OpenTelemetry.`
- User message: `How to instrument GenAI library with OTel?`

- The model responds with `"Follow GenAI semantic conventions available at opentelemetry.io."` message

Span:
Expand All @@ -267,35 +272,36 @@ Span:
| `gen_ai.request.top_p`| `1.0` |
| `gen_ai.response.id`| `"chatcmpl-9J3uIL87gldCFtiIbyaOvTeYBRA3l"` |
| `gen_ai.response.model`| `"gpt-4-0613"` |
| `gen_ai.usage.completion_tokens`| `47` |
| `gen_ai.usage.prompt_tokens`| `52` |
| `gen_ai.usage.completion_tokens`| `47` |
| `gen_ai.usage.prompt_tokens`| `52` |
| `gen_ai.response.finish_reason`| `"stop"` |

Events:

1. `gen_ai.system.message`
1. `gen_ai.system.message`.
Not reported if when capturing sensitive content is not enabled.

| Property | Value |
|---------------------|-------------------------------------------------------|
| `gen_ai.system` | `"openai"` |
| Event payload (full) | `{"role":"system","content":"You're a friendly bot that answers questions about OpenTelemetry."}` |
| Event payload (without sensitive content) | `{"role":"system", "content":"REDACTED"}` |
| Event payload | `{"content":"You're a friendly bot that answers questions about OpenTelemetry."}` |

2. `gen_ai.user.message`
Not reported if when capturing sensitive content is not enabled.

| Property | Value |
|---------------------|-------------------------------------------------------|
| `gen_ai.system` | `"openai"` |
| Event payload (full) | `{"role":"user","content":"How to instrument GenAI library with OTel?"}` |
| Event payload (without sensitive content) | `{"role":"user", "content":"REDACTED"}` |
| Event payload | `{"content":"How to instrument GenAI library with OTel?"}` |

3. `gen_ai.choice`
May be reported if when capturing sensitive content is not enabled.

| Property | Value |
|---------------------|-------------------------------------------------------|
| `gen_ai.system` | `"openai"` |
| Event payload (full) | `{"index":0,"finish_reason":"stop","message":{"role":"assistant","content":"Follow GenAI semantic conventions available at opentelemetry.io."}}` |
| Event payload (without sensitive content) | `{"index":0,"finish_reason":"stop","message":{"role":"assistant","content":"REDACTED"}}` |
| Event payload (with sensitive content) | `{"index":0,"finish_reason":"stop","message":{"role":"assistant","content":"Follow GenAI semantic conventions available at opentelemetry.io."}}` |
| Event payload (without sensitive content) | `{"index":0,"finish_reason":"stop","message":{"role":"assistant"}}` |

### Tools

Expand Down Expand Up @@ -331,19 +337,20 @@ Here's the telemetry generated for each step in this scenario:

Events parented to this span:

- `gen_ai.user.message`
- `gen_ai.user.message` (not reported when capturing sensitive content is disabled)

| Property | Value |
|---------------------|-------------------------------------------------------|
| `gen_ai.system` | `"openai"` |
| Event payload | `{"role":"user","content":"How to instrument GenAI library with OTel?"}` |
| Event payload | `{"content":"How to instrument GenAI library with OTel?"}` |

- `gen_ai.choice`
- `gen_ai.choice` (not reported when capturing sensitive content is disabled)

| Property | Value |
|---------------------|-------------------------------------------------------|
| `gen_ai.system` | `"openai"` |
| Event payload | `{"index":0,"finish_reason":"tool_calls","message":{"role":"assistant","tool_calls":[{"id":"call_VSPygqKTWdrhaFErNvMV18Yl","function":{"name":"get_link_to_otel_semconv","arguments":"{\"semconv\":\"GenAI\"}"},"type":"function"}]}` |
| Event payload (with sensitive content) | `{"index":0,"finish_reason":"tool_calls","message":{"role":"assistant","tool_calls":[{"id":"call_VSPygqKTWdrhaFErNvMV18Yl","function":{"name":"get_link_to_otel_semconv","arguments":"{\"semconv\":\"GenAI\"}"},"type":"function"}]}` |
| Event payload (withput sensitive content) | `{"index":0,"finish_reason":"tool_calls","message":{"role":"assistant","tool_calls":[{"id":"call_VSPygqKTWdrhaFErNvMV18Yl","function":{"name":"get_link_to_otel_semconv"},"type":"function"}]}` |

2. Application executes the tool call. Application may create span which is not covered by this semantic convention.
3. Final chat completion call
Expand All @@ -364,32 +371,35 @@ Here's the telemetry generated for each step in this scenario:
Events parented to this span:
(in this example, the event content matches the original messages, but applications may also drop messages or change their content)

- `gen_ai.user.message`
- `gen_ai.user.message` (not reported when capturing sensitive content is disabled)

| Property | Value |
|---------------------|-------------------------------------------------------|
| `gen_ai.system` | `"openai"` |
| Event payload | `{"role":"user","content":"How to instrument GenAI library with OTel?"}` |

- `gen_ai.assistant.message`
- `gen_ai.assistant.message` (could be reported when capturing sensitive content is disabled)

| Property | Value |
|---------------------|-------------------------------------------------------|
| `gen_ai.system` | `"openai"` |
| Event payload | `{"role":"assistant","tool_calls":[{"id":"call_VSPygqKTWdrhaFErNvMV18Yl","function":{"name":"get_link_to_otel_semconv","arguments":"{\"semconv\":\"GenAI\"}"},"type":"function"}]}` |
| Event payload (with sensitive content) | `{"role":"assistant","tool_calls":[{"id":"call_VSPygqKTWdrhaFErNvMV18Yl","function":{"name":"get_link_to_otel_semconv","arguments":"{\"semconv\":\"GenAI\"}"},"type":"function"}]}` |
| Event payload (without sensitive content) | `{"role":"assistant","tool_calls":[{"id":"call_VSPygqKTWdrhaFErNvMV18Yl","function":{"name":"get_link_to_otel_semconv","type":"function"}]}` |

- `gen_ai.tool.message`
- `gen_ai.tool.message` (could be reported when capturing sensitive content is disabled)

| Property | Value |
|---------------------|-------------------------------------------------------|
| `gen_ai.system` | `"openai"` |
| Event payload | `{"role":"tool","content":"opentelemetry.io/semconv/gen-ai","tool_call_id":"call_VSPygqKTWdrhaFErNvMV18Yl"}` |
| Event payload (with sensitive content) | `{"role":"tool","content":"opentelemetry.io/semconv/gen-ai","tool_call_id":"call_VSPygqKTWdrhaFErNvMV18Yl"}` |
| Event payload (without sensitive content) | `{"role":"tool","tool_call_id":"call_VSPygqKTWdrhaFErNvMV18Yl"}` |

- `gen_ai.choice`
- `gen_ai.choice` (could be reported when capturing sensitive content is disabled)

| Property | Value |
|---------------------|-------------------------------------------------------|
| `gen_ai.system` | `"openai"` |
| Event payload | `{"index":0,"finish_reason":"stop","message":{"role":"assistant","content":"Follow OTel semconv available at opentelemetry.io/semconv/gen-ai"}}` |
| Event payload (with sensitive content) | `{"index":0,"finish_reason":"stop","message":{"role":"assistant","content":"Follow OTel semconv available at opentelemetry.io/semconv/gen-ai"}}` |
| Event payload (without sensitive content) | `{"index":0,"finish_reason":"stop","message":{"role":"assistant"}}` |

[DocumentStatus]: https://github.com/open-telemetry/opentelemetry-specification/tree/v1.31.0/specification/document-status.md

0 comments on commit cc65371

Please sign in to comment.