From ada3a9ae32846ff251321244979f8d5bcbf54733 Mon Sep 17 00:00:00 2001 From: Karthik Kalyanaraman Date: Wed, 22 May 2024 08:08:42 -0700 Subject: [PATCH 01/20] Add additional LLM span attributes --- docs/attributes-registry/gen-ai.md | 2 ++ model/registry/gen-ai.yaml | 12 ++++++++++++ model/trace/gen-ai.yaml | 4 ++++ 3 files changed, 18 insertions(+) diff --git a/docs/attributes-registry/gen-ai.md b/docs/attributes-registry/gen-ai.md index a58d9c3989..66d20d606e 100644 --- a/docs/attributes-registry/gen-ai.md +++ b/docs/attributes-registry/gen-ai.md @@ -18,6 +18,8 @@ This document defines the attributes used to describe telemetry in the context o | `gen_ai.request.model` | string | The name of the LLM a request is being made to. | `gpt-4` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `gen_ai.request.temperature` | double | The temperature setting for the LLM request. | `0.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `gen_ai.request.top_p` | double | The top_p sampling setting for the LLM request. | `1.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `gen_ai.request.stop` | string | Up to 4 sequences where the API will stop generating further tokens. | `1.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `gen_ai.request.top_k` | double | The top_k sampling setting for the LLM request. | `1.0` | `gen_ai.response.finish_reasons` | string[] | Array of reasons the model stopped generating tokens, corresponding to each generation received. | `stop` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `gen_ai.response.id` | string | The unique identifier for the completion. | `chatcmpl-123` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `gen_ai.response.model` | string | The name of the LLM a response was generated from. | `gpt-4-0613` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | diff --git a/model/registry/gen-ai.yaml b/model/registry/gen-ai.yaml index ab2ea3700c..73abc385e2 100644 --- a/model/registry/gen-ai.yaml +++ b/model/registry/gen-ai.yaml @@ -45,6 +45,18 @@ groups: brief: The top_p sampling setting for the LLM request. examples: [1.0] tag: llm-generic-request + - id: request.top_k + stability: experimental + type: double + brief: The top_k sampling setting for the LLM request. + examples: [1.0] + tag: llm-generic-request + - id: request.stop + stability: experimental + type: double + brief: The stop sequences to provide. + examples: ['\n'] + tag: llm-generic-request - id: response.id stability: experimental type: string diff --git a/model/trace/gen-ai.yaml b/model/trace/gen-ai.yaml index bf1d112e37..ef493b1a99 100644 --- a/model/trace/gen-ai.yaml +++ b/model/trace/gen-ai.yaml @@ -23,6 +23,10 @@ groups: requirement_level: recommended - ref: gen_ai.request.top_p requirement_level: recommended + - ref: gen_ai.request.top_k + requirement_level: recommended + - ref: gen_ai.request.stop + requirement_level: recommended - ref: gen_ai.response.id requirement_level: recommended - ref: gen_ai.response.model From 6dd78a10810344e86f5e1f5fa2e809c42872d172 Mon Sep 17 00:00:00 2001 From: Karthik Kalyanaraman Date: Wed, 22 May 2024 08:17:36 -0700 Subject: [PATCH 02/20] add change log --- .chloggen/839.yaml | 4 ++++ 1 file changed, 4 insertions(+) create mode 100644 .chloggen/839.yaml diff --git a/.chloggen/839.yaml b/.chloggen/839.yaml new file mode 100644 index 0000000000..68557d8dcc --- /dev/null +++ b/.chloggen/839.yaml @@ -0,0 +1,4 @@ +change_type: enhancement +component: gen-ai +note: Adding top_k and stop to the list of required attributes for the gen_ai.request.model event. +issues: [839] \ No newline at end of file From 988a1167276b29ad2815e583d39ce0cfbcea8b6e Mon Sep 17 00:00:00 2001 From: Karthik Kalyanaraman Date: Wed, 22 May 2024 08:24:27 -0700 Subject: [PATCH 03/20] Add presence and frequency penalty --- docs/attributes-registry/gen-ai.md | 5 ++++- model/registry/gen-ai.yaml | 12 ++++++++++++ model/trace/gen-ai.yaml | 4 ++++ 3 files changed, 20 insertions(+), 1 deletion(-) diff --git a/docs/attributes-registry/gen-ai.md b/docs/attributes-registry/gen-ai.md index 66d20d606e..56710feb2e 100644 --- a/docs/attributes-registry/gen-ai.md +++ b/docs/attributes-registry/gen-ai.md @@ -19,7 +19,10 @@ This document defines the attributes used to describe telemetry in the context o | `gen_ai.request.temperature` | double | The temperature setting for the LLM request. | `0.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `gen_ai.request.top_p` | double | The top_p sampling setting for the LLM request. | `1.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `gen_ai.request.stop` | string | Up to 4 sequences where the API will stop generating further tokens. | `1.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `gen_ai.request.top_k` | double | The top_k sampling setting for the LLM request. | `1.0` +| `gen_ai.request.top_k` | double | The top_k sampling setting for the LLM request. | `1.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `gen_ai.request.frequency_penalty` | double | The frequency penalty setting for the LLM request. | `1.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `gen_ai.request.presence_penalty` | double | The presence penalty setting for the LLM request. | `1.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `gen_ai.request.top_p` | double | The top_p sampling setting for the LLM request. | `1.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `gen_ai.response.finish_reasons` | string[] | Array of reasons the model stopped generating tokens, corresponding to each generation received. | `stop` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `gen_ai.response.id` | string | The unique identifier for the completion. | `chatcmpl-123` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `gen_ai.response.model` | string | The name of the LLM a response was generated from. | `gpt-4-0613` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | diff --git a/model/registry/gen-ai.yaml b/model/registry/gen-ai.yaml index 73abc385e2..c6707b0607 100644 --- a/model/registry/gen-ai.yaml +++ b/model/registry/gen-ai.yaml @@ -57,6 +57,18 @@ groups: brief: The stop sequences to provide. examples: ['\n'] tag: llm-generic-request + - id: request.frequency_penalty + stability: experimental + type: double + brief: The frequency penalty to provide. + examples: ['0.1'] + tag: llm-generic-request + - id: request.presence_penalty + stability: experimental + type: double + brief: The presence penalty to provide. + examples: ['0.1'] + tag: llm-generic-request - id: response.id stability: experimental type: string diff --git a/model/trace/gen-ai.yaml b/model/trace/gen-ai.yaml index ef493b1a99..a003eb8f08 100644 --- a/model/trace/gen-ai.yaml +++ b/model/trace/gen-ai.yaml @@ -27,6 +27,10 @@ groups: requirement_level: recommended - ref: gen_ai.request.stop requirement_level: recommended + - ref: gen_ai.request.frequncy_penalty + requirement_level: recommended + - ref: gen_ai.request.presence_penalty + requirement_level: recommended - ref: gen_ai.response.id requirement_level: recommended - ref: gen_ai.response.model From 1ffde119db50d84309ac67904a1a42321459970b Mon Sep 17 00:00:00 2001 From: Karthik Kalyanaraman Date: Wed, 22 May 2024 08:27:27 -0700 Subject: [PATCH 04/20] Add presence and frequency penalty --- docs/attributes-registry/gen-ai.md | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/attributes-registry/gen-ai.md b/docs/attributes-registry/gen-ai.md index 56710feb2e..9f33f62bd0 100644 --- a/docs/attributes-registry/gen-ai.md +++ b/docs/attributes-registry/gen-ai.md @@ -22,7 +22,6 @@ This document defines the attributes used to describe telemetry in the context o | `gen_ai.request.top_k` | double | The top_k sampling setting for the LLM request. | `1.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `gen_ai.request.frequency_penalty` | double | The frequency penalty setting for the LLM request. | `1.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `gen_ai.request.presence_penalty` | double | The presence penalty setting for the LLM request. | `1.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `gen_ai.request.top_p` | double | The top_p sampling setting for the LLM request. | `1.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `gen_ai.response.finish_reasons` | string[] | Array of reasons the model stopped generating tokens, corresponding to each generation received. | `stop` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `gen_ai.response.id` | string | The unique identifier for the completion. | `chatcmpl-123` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `gen_ai.response.model` | string | The name of the LLM a response was generated from. | `gpt-4-0613` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | From 77bdde88d2b6261093574981548155357372de4f Mon Sep 17 00:00:00 2001 From: Karthik Kalyanaraman Date: Wed, 22 May 2024 08:28:31 -0700 Subject: [PATCH 05/20] add new line --- .chloggen/839.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.chloggen/839.yaml b/.chloggen/839.yaml index 68557d8dcc..7e62b3a83f 100644 --- a/.chloggen/839.yaml +++ b/.chloggen/839.yaml @@ -1,4 +1,4 @@ change_type: enhancement component: gen-ai note: Adding top_k and stop to the list of required attributes for the gen_ai.request.model event. -issues: [839] \ No newline at end of file +issues: [839] From a210669cc5440fe87de1d66ce9b444d14ecd4bb7 Mon Sep 17 00:00:00 2001 From: Karthik Kalyanaraman <105607645+karthikscale3@users.noreply.github.com> Date: Wed, 29 May 2024 16:21:22 -0700 Subject: [PATCH 06/20] Update .chloggen/839.yaml Co-authored-by: Liudmila Molkova --- .chloggen/839.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.chloggen/839.yaml b/.chloggen/839.yaml index 7e62b3a83f..1da8ca5ba8 100644 --- a/.chloggen/839.yaml +++ b/.chloggen/839.yaml @@ -1,4 +1,4 @@ change_type: enhancement component: gen-ai -note: Adding top_k and stop to the list of required attributes for the gen_ai.request.model event. +note: Adding `gen_ai.request.top_k`, `gen_ai.request.presence_penalty`, `gen_ai.request.frequency_penalty` and `gen_ai.request.stop_sequences` attributes. issues: [839] From 57f4dce7a6e1baabf57452971e7fa88d1842afdb Mon Sep 17 00:00:00 2001 From: Karthik Kalyanaraman <105607645+karthikscale3@users.noreply.github.com> Date: Wed, 29 May 2024 16:21:27 -0700 Subject: [PATCH 07/20] Update model/registry/gen-ai.yaml Co-authored-by: Liudmila Molkova --- model/registry/gen-ai.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/model/registry/gen-ai.yaml b/model/registry/gen-ai.yaml index c6707b0607..99fa011007 100644 --- a/model/registry/gen-ai.yaml +++ b/model/registry/gen-ai.yaml @@ -54,7 +54,7 @@ groups: - id: request.stop stability: experimental type: double - brief: The stop sequences to provide. + brief: The stop sequences provided in the request. examples: ['\n'] tag: llm-generic-request - id: request.frequency_penalty From c7506aaa9d7088abac0eb49bf74c46166fa32454 Mon Sep 17 00:00:00 2001 From: Karthik Kalyanaraman <105607645+karthikscale3@users.noreply.github.com> Date: Wed, 29 May 2024 16:21:33 -0700 Subject: [PATCH 08/20] Update model/registry/gen-ai.yaml Co-authored-by: Liudmila Molkova --- model/registry/gen-ai.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/model/registry/gen-ai.yaml b/model/registry/gen-ai.yaml index 99fa011007..2e9deb8d8a 100644 --- a/model/registry/gen-ai.yaml +++ b/model/registry/gen-ai.yaml @@ -53,7 +53,7 @@ groups: tag: llm-generic-request - id: request.stop stability: experimental - type: double + type: string[] brief: The stop sequences provided in the request. examples: ['\n'] tag: llm-generic-request From 5dfddfe48f78ef53d7036b61790b076f1e477bb4 Mon Sep 17 00:00:00 2001 From: Karthik Kalyanaraman <105607645+karthikscale3@users.noreply.github.com> Date: Wed, 29 May 2024 16:21:37 -0700 Subject: [PATCH 09/20] Update model/registry/gen-ai.yaml Co-authored-by: Liudmila Molkova --- model/registry/gen-ai.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/model/registry/gen-ai.yaml b/model/registry/gen-ai.yaml index 2e9deb8d8a..f85f125ca2 100644 --- a/model/registry/gen-ai.yaml +++ b/model/registry/gen-ai.yaml @@ -51,7 +51,7 @@ groups: brief: The top_k sampling setting for the LLM request. examples: [1.0] tag: llm-generic-request - - id: request.stop + - id: request.stop_sequences stability: experimental type: string[] brief: The stop sequences provided in the request. From b4392e89c29323a8179e0d0a308bf6ce3a4e1f85 Mon Sep 17 00:00:00 2001 From: Karthik Kalyanaraman <105607645+karthikscale3@users.noreply.github.com> Date: Wed, 29 May 2024 16:21:42 -0700 Subject: [PATCH 10/20] Update model/registry/gen-ai.yaml Co-authored-by: Liudmila Molkova --- model/registry/gen-ai.yaml | 1 - 1 file changed, 1 deletion(-) diff --git a/model/registry/gen-ai.yaml b/model/registry/gen-ai.yaml index f85f125ca2..e4508936b0 100644 --- a/model/registry/gen-ai.yaml +++ b/model/registry/gen-ai.yaml @@ -50,7 +50,6 @@ groups: type: double brief: The top_k sampling setting for the LLM request. examples: [1.0] - tag: llm-generic-request - id: request.stop_sequences stability: experimental type: string[] From eeae488f88348136bb6541a2427adfb42fa68945 Mon Sep 17 00:00:00 2001 From: Karthik Kalyanaraman <105607645+karthikscale3@users.noreply.github.com> Date: Wed, 29 May 2024 16:24:25 -0700 Subject: [PATCH 11/20] Update gen-ai.yaml --- model/registry/gen-ai.yaml | 3 --- 1 file changed, 3 deletions(-) diff --git a/model/registry/gen-ai.yaml b/model/registry/gen-ai.yaml index bc70dfd9f5..6a04790ebc 100644 --- a/model/registry/gen-ai.yaml +++ b/model/registry/gen-ai.yaml @@ -55,19 +55,16 @@ groups: type: string[] brief: The stop sequences provided in the request. examples: ['\n'] - tag: llm-generic-request - id: request.frequency_penalty stability: experimental type: double brief: The frequency penalty to provide. examples: ['0.1'] - tag: llm-generic-request - id: request.presence_penalty stability: experimental type: double brief: The presence penalty to provide. examples: ['0.1'] - tag: llm-generic-request - id: response.id stability: experimental type: string From 65df716871e71a32189ba96cee43eaac344418d3 Mon Sep 17 00:00:00 2001 From: Karthik Kalyanaraman Date: Tue, 18 Jun 2024 23:07:44 -0700 Subject: [PATCH 12/20] Generate md --- docs/gen-ai/gen-ai-spans.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/gen-ai/gen-ai-spans.md b/docs/gen-ai/gen-ai-spans.md index 1dadd5e43b..02f871a33f 100644 --- a/docs/gen-ai/gen-ai-spans.md +++ b/docs/gen-ai/gen-ai-spans.md @@ -47,8 +47,12 @@ These attributes track input data and metadata for a request to an GenAI model. |---|---|---|---|---|---| | [`gen_ai.request.model`](/docs/attributes-registry/gen-ai.md) | string | The name of the GenAI model a request is being made to. [1] | `gpt-4` | `Required` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | [`gen_ai.system`](/docs/attributes-registry/gen-ai.md) | string | The Generative AI product as identified by the client instrumentation. [2] | `openai` | `Required` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| [`gen_ai.request.frequency_penalty`](/docs/attributes-registry/gen-ai.md) | double | The frequency penalty to provide. | `0.1` | `Recommended` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | [`gen_ai.request.max_tokens`](/docs/attributes-registry/gen-ai.md) | int | The maximum number of tokens the model generates for a request. | `100` | `Recommended` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| [`gen_ai.request.presence_penalty`](/docs/attributes-registry/gen-ai.md) | double | The presence penalty to provide. | `0.1` | `Recommended` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| [`gen_ai.request.stop_sequences`](/docs/attributes-registry/gen-ai.md) | string[] | The stop sequences provided in the request. | `["\\n"]` | `Recommended` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | [`gen_ai.request.temperature`](/docs/attributes-registry/gen-ai.md) | double | The temperature setting for the GenAI request. | `0.0` | `Recommended` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| [`gen_ai.request.top_k`](/docs/attributes-registry/gen-ai.md) | double | The top_k sampling setting for the LLM request. | `1.0` | `Recommended` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | [`gen_ai.request.top_p`](/docs/attributes-registry/gen-ai.md) | double | The top_p sampling setting for the GenAI request. | `1.0` | `Recommended` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | [`gen_ai.response.finish_reasons`](/docs/attributes-registry/gen-ai.md) | string[] | Array of reasons the model stopped generating tokens, corresponding to each generation received. | `["stop"]` | `Recommended` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | [`gen_ai.response.id`](/docs/attributes-registry/gen-ai.md) | string | The unique identifier for the completion. | `chatcmpl-123` | `Recommended` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | From 76c3814b6900f07f0558ea483557194e36279214 Mon Sep 17 00:00:00 2001 From: Karthik Kalyanaraman Date: Tue, 18 Jun 2024 23:16:39 -0700 Subject: [PATCH 13/20] Update markdowns --- docs/attributes-registry/gen-ai.md | 2 +- docs/gen-ai/gen-ai-spans.md | 7 ++++--- model/registry/gen-ai.yaml | 8 ++++---- 3 files changed, 9 insertions(+), 8 deletions(-) diff --git a/docs/attributes-registry/gen-ai.md b/docs/attributes-registry/gen-ai.md index 5e683e06e4..96aee39049 100644 --- a/docs/attributes-registry/gen-ai.md +++ b/docs/attributes-registry/gen-ai.md @@ -18,7 +18,7 @@ This document defines the attributes used to describe telemetry in the context o | `gen_ai.request.model` | string | The name of the LLM a request is being made to. | `gpt-4` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `gen_ai.request.temperature` | double | The temperature setting for the LLM request. | `0.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `gen_ai.request.top_p` | double | The top_p sampling setting for the LLM request. | `1.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `gen_ai.request.stop` | string[] | List of sequences that the model will use to stop generating further tokens. | `['forest', 'lived']` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `gen_ai.request.stop_sequences` | string[] | List of sequences that the model will use to stop generating further tokens. | `['forest', 'lived']` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `gen_ai.request.top_k` | double | The top_k sampling setting for the LLM request. | `1.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `gen_ai.request.frequency_penalty` | double | The frequency penalty setting for the LLM request. | `1.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `gen_ai.request.presence_penalty` | double | The presence penalty setting for the LLM request. | `1.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | diff --git a/docs/gen-ai/gen-ai-spans.md b/docs/gen-ai/gen-ai-spans.md index 02f871a33f..9d88a06884 100644 --- a/docs/gen-ai/gen-ai-spans.md +++ b/docs/gen-ai/gen-ai-spans.md @@ -10,9 +10,10 @@ linkTitle: Generative AI traces -- [Configuration](#configuration) -- [GenAI attributes](#genai-attributes) -- [Events](#events) +- [Semantic Conventions for GenAI operations](#semantic-conventions-for-genai-operations) + - [Configuration](#configuration) + - [GenAI attributes](#genai-attributes) + - [Events](#events) diff --git a/model/registry/gen-ai.yaml b/model/registry/gen-ai.yaml index 55dad26b05..021bf513ec 100644 --- a/model/registry/gen-ai.yaml +++ b/model/registry/gen-ai.yaml @@ -63,18 +63,18 @@ groups: - id: request.stop_sequences stability: experimental type: string[] - brief: The stop sequences provided in the request. - examples: ['\n'] + brief: List of sequences that the model will use to stop generating further tokens. + examples: ['forest', 'lived'] - id: request.frequency_penalty stability: experimental type: double - brief: The frequency penalty to provide. + brief: The frequency penalty setting for the LLM request. examples: [0.1] tag: llm-generic-request - id: request.presence_penalty stability: experimental type: double - brief: The presence penalty to provide. + brief: The presence penalty setting for the LLM request. examples: [0.1] tag: llm-generic-request - id: response.id From 279989a4554299695e0535058ea5ac0d39886a0d Mon Sep 17 00:00:00 2001 From: Karthik Kalyanaraman Date: Wed, 19 Jun 2024 08:07:25 -0700 Subject: [PATCH 14/20] Update md --- docs/attributes-registry/gen-ai.md | 39 +++++++++++++++--------------- docs/gen-ai/gen-ai-spans.md | 7 +++--- 2 files changed, 23 insertions(+), 23 deletions(-) diff --git a/docs/attributes-registry/gen-ai.md b/docs/attributes-registry/gen-ai.md index 96aee39049..3de507cfb2 100644 --- a/docs/attributes-registry/gen-ai.md +++ b/docs/attributes-registry/gen-ai.md @@ -10,25 +10,26 @@ This document defines the attributes used to describe telemetry in the context of Generative Artificial Intelligence (GenAI) Models requests and responses. -| Attribute | Type | Description | Examples | Stability | -| -------------------------------- | -------- | ------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------- | ---------------------------------------------------------------- | -| `gen_ai.completion` | string | The full response received from the LLM. [1] | `[{'role': 'assistant', 'content': 'The capital of France is Paris.'}]` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `gen_ai.prompt` | string | The full prompt sent to an LLM. [2] | `[{'role': 'user', 'content': 'What is the capital of France?'}]` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `gen_ai.request.max_tokens` | int | The maximum number of tokens the LLM generates for a request. | `100` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `gen_ai.request.model` | string | The name of the LLM a request is being made to. | `gpt-4` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `gen_ai.request.temperature` | double | The temperature setting for the LLM request. | `0.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `gen_ai.request.top_p` | double | The top_p sampling setting for the LLM request. | `1.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `gen_ai.request.stop_sequences` | string[] | List of sequences that the model will use to stop generating further tokens. | `['forest', 'lived']` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `gen_ai.request.top_k` | double | The top_k sampling setting for the LLM request. | `1.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `gen_ai.request.frequency_penalty` | double | The frequency penalty setting for the LLM request. | `1.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `gen_ai.request.presence_penalty` | double | The presence penalty setting for the LLM request. | `1.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `gen_ai.response.finish_reasons` | string[] | Array of reasons the model stopped generating tokens, corresponding to each generation received. | `stop` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `gen_ai.response.id` | string | The unique identifier for the completion. | `chatcmpl-123` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `gen_ai.response.model` | string | The name of the model that generated the response. | `gpt-4-0613` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `gen_ai.system` | string | The Generative AI product as identified by the client instrumentation. [3] | `openai` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `gen_ai.token.type` | string | The type of token being counted. | `input`; `output` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `gen_ai.usage.completion_tokens` | int | The number of tokens used in the GenAI response (completion). | `180` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `gen_ai.usage.prompt_tokens` | int | The number of tokens used in the GenAI input or prompt. | `100` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| Attribute | Type | Description | Examples | Stability | +| ---------------------------------- | -------- | ------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------- | ---------------------------------------------------------------- | +| `gen_ai.completion` | string | The full response received from the GenAI model. [1] | `[{'role': 'assistant', 'content': 'The capital of France is Paris.'}]` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `gen_ai.operation.name` | string | The name of the operation being performed. | `chat`; `completion` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `gen_ai.prompt` | string | The full prompt sent to the GenAI model. [2] | `[{'role': 'user', 'content': 'What is the capital of France?'}]` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `gen_ai.request.frequency_penalty` | double | The frequency penalty setting for the LLM request. | `0.1` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `gen_ai.request.max_tokens` | int | The maximum number of tokens the model generates for a request. | `100` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `gen_ai.request.model` | string | The name of the GenAI model a request is being made to. | `gpt-4` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `gen_ai.request.presence_penalty` | double | The presence penalty setting for the LLM request. | `0.1` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `gen_ai.request.stop_sequences` | string[] | List of sequences that the model will use to stop generating further tokens. | `["forest", "lived"]` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `gen_ai.request.temperature` | double | The temperature setting for the GenAI request. | `0.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `gen_ai.request.top_k` | double | The top_k sampling setting for the LLM request. | `1.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `gen_ai.request.top_p` | double | The top_p sampling setting for the GenAI request. | `1.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `gen_ai.response.finish_reasons` | string[] | Array of reasons the model stopped generating tokens, corresponding to each generation received. | `["stop"]` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `gen_ai.response.id` | string | The unique identifier for the completion. | `chatcmpl-123` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `gen_ai.response.model` | string | The name of the model that generated the response. | `gpt-4-0613` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `gen_ai.system` | string | The Generative AI product as identified by the client instrumentation. [3] | `openai` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `gen_ai.token.type` | string | The type of token being counted. | `input`; `output` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `gen_ai.usage.completion_tokens` | int | The number of tokens used in the GenAI response (completion). | `180` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `gen_ai.usage.prompt_tokens` | int | The number of tokens used in the GenAI input or prompt. | `100` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | **[1]:** It's RECOMMENDED to format completions as JSON string matching [OpenAI messages format](https://platform.openai.com/docs/guides/text-generation) diff --git a/docs/gen-ai/gen-ai-spans.md b/docs/gen-ai/gen-ai-spans.md index 9d88a06884..02f871a33f 100644 --- a/docs/gen-ai/gen-ai-spans.md +++ b/docs/gen-ai/gen-ai-spans.md @@ -10,10 +10,9 @@ linkTitle: Generative AI traces -- [Semantic Conventions for GenAI operations](#semantic-conventions-for-genai-operations) - - [Configuration](#configuration) - - [GenAI attributes](#genai-attributes) - - [Events](#events) +- [Configuration](#configuration) +- [GenAI attributes](#genai-attributes) +- [Events](#events) From d0138636a132be51bf827e33b9bea050fd49eb55 Mon Sep 17 00:00:00 2001 From: Karthik Kalyanaraman <105607645+karthikscale3@users.noreply.github.com> Date: Wed, 19 Jun 2024 11:48:14 -0700 Subject: [PATCH 15/20] Update model/registry/gen-ai.yaml Co-authored-by: Liudmila Molkova --- model/registry/gen-ai.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/model/registry/gen-ai.yaml b/model/registry/gen-ai.yaml index 021bf513ec..4f64c5d87d 100644 --- a/model/registry/gen-ai.yaml +++ b/model/registry/gen-ai.yaml @@ -74,7 +74,7 @@ groups: - id: request.presence_penalty stability: experimental type: double - brief: The presence penalty setting for the LLM request. + brief: The presence penalty setting for the GenAI request. examples: [0.1] tag: llm-generic-request - id: response.id From 5fd94141f118e8e7f184c124f17118fdce261d65 Mon Sep 17 00:00:00 2001 From: Karthik Kalyanaraman <105607645+karthikscale3@users.noreply.github.com> Date: Wed, 19 Jun 2024 11:48:19 -0700 Subject: [PATCH 16/20] Update model/registry/gen-ai.yaml Co-authored-by: Liudmila Molkova --- model/registry/gen-ai.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/model/registry/gen-ai.yaml b/model/registry/gen-ai.yaml index 4f64c5d87d..a6149c2744 100644 --- a/model/registry/gen-ai.yaml +++ b/model/registry/gen-ai.yaml @@ -68,7 +68,7 @@ groups: - id: request.frequency_penalty stability: experimental type: double - brief: The frequency penalty setting for the LLM request. + brief: The frequency penalty setting for the GenAI request. examples: [0.1] tag: llm-generic-request - id: request.presence_penalty From a14affc90f461b748269d57a73ab1eed7e1200b7 Mon Sep 17 00:00:00 2001 From: Karthik Kalyanaraman <105607645+karthikscale3@users.noreply.github.com> Date: Wed, 19 Jun 2024 11:48:24 -0700 Subject: [PATCH 17/20] Update model/registry/gen-ai.yaml Co-authored-by: Liudmila Molkova --- model/registry/gen-ai.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/model/registry/gen-ai.yaml b/model/registry/gen-ai.yaml index a6149c2744..25f068d048 100644 --- a/model/registry/gen-ai.yaml +++ b/model/registry/gen-ai.yaml @@ -58,7 +58,7 @@ groups: - id: request.top_k stability: experimental type: double - brief: The top_k sampling setting for the LLM request. + brief: The top_k sampling setting for the GenAI request. examples: [1.0] - id: request.stop_sequences stability: experimental From 1bdb9292838142223de69ce5104c8708e86ce908 Mon Sep 17 00:00:00 2001 From: Karthik Kalyanaraman <105607645+karthikscale3@users.noreply.github.com> Date: Wed, 19 Jun 2024 11:48:29 -0700 Subject: [PATCH 18/20] Update model/registry/gen-ai.yaml Co-authored-by: Liudmila Molkova --- model/registry/gen-ai.yaml | 1 - 1 file changed, 1 deletion(-) diff --git a/model/registry/gen-ai.yaml b/model/registry/gen-ai.yaml index 25f068d048..d2629179ba 100644 --- a/model/registry/gen-ai.yaml +++ b/model/registry/gen-ai.yaml @@ -70,7 +70,6 @@ groups: type: double brief: The frequency penalty setting for the GenAI request. examples: [0.1] - tag: llm-generic-request - id: request.presence_penalty stability: experimental type: double From 68a0c5fa06d5d288622e80d9760e931f313d9686 Mon Sep 17 00:00:00 2001 From: Karthik Kalyanaraman <105607645+karthikscale3@users.noreply.github.com> Date: Wed, 19 Jun 2024 11:48:35 -0700 Subject: [PATCH 19/20] Update model/registry/gen-ai.yaml Co-authored-by: Liudmila Molkova --- model/registry/gen-ai.yaml | 1 - 1 file changed, 1 deletion(-) diff --git a/model/registry/gen-ai.yaml b/model/registry/gen-ai.yaml index d2629179ba..394f053a21 100644 --- a/model/registry/gen-ai.yaml +++ b/model/registry/gen-ai.yaml @@ -75,7 +75,6 @@ groups: type: double brief: The presence penalty setting for the GenAI request. examples: [0.1] - tag: llm-generic-request - id: response.id stability: experimental type: string From e580b8eda4ec7ace5755298116cc96c7edbca988 Mon Sep 17 00:00:00 2001 From: Karthik Kalyanaraman Date: Wed, 19 Jun 2024 20:02:24 -0700 Subject: [PATCH 20/20] Update markdowns --- docs/attributes-registry/gen-ai.md | 6 +++--- docs/gen-ai/gen-ai-spans.md | 8 ++++---- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/docs/attributes-registry/gen-ai.md b/docs/attributes-registry/gen-ai.md index 3de507cfb2..36a69ad7dc 100644 --- a/docs/attributes-registry/gen-ai.md +++ b/docs/attributes-registry/gen-ai.md @@ -15,13 +15,13 @@ This document defines the attributes used to describe telemetry in the context o | `gen_ai.completion` | string | The full response received from the GenAI model. [1] | `[{'role': 'assistant', 'content': 'The capital of France is Paris.'}]` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `gen_ai.operation.name` | string | The name of the operation being performed. | `chat`; `completion` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `gen_ai.prompt` | string | The full prompt sent to the GenAI model. [2] | `[{'role': 'user', 'content': 'What is the capital of France?'}]` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `gen_ai.request.frequency_penalty` | double | The frequency penalty setting for the LLM request. | `0.1` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `gen_ai.request.frequency_penalty` | double | The frequency penalty setting for the GenAI request. | `0.1` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `gen_ai.request.max_tokens` | int | The maximum number of tokens the model generates for a request. | `100` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `gen_ai.request.model` | string | The name of the GenAI model a request is being made to. | `gpt-4` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `gen_ai.request.presence_penalty` | double | The presence penalty setting for the LLM request. | `0.1` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `gen_ai.request.presence_penalty` | double | The presence penalty setting for the GenAI request. | `0.1` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `gen_ai.request.stop_sequences` | string[] | List of sequences that the model will use to stop generating further tokens. | `["forest", "lived"]` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `gen_ai.request.temperature` | double | The temperature setting for the GenAI request. | `0.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `gen_ai.request.top_k` | double | The top_k sampling setting for the LLM request. | `1.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `gen_ai.request.top_k` | double | The top_k sampling setting for the GenAI request. | `1.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `gen_ai.request.top_p` | double | The top_p sampling setting for the GenAI request. | `1.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `gen_ai.response.finish_reasons` | string[] | Array of reasons the model stopped generating tokens, corresponding to each generation received. | `["stop"]` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `gen_ai.response.id` | string | The unique identifier for the completion. | `chatcmpl-123` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | diff --git a/docs/gen-ai/gen-ai-spans.md b/docs/gen-ai/gen-ai-spans.md index 02f871a33f..d6837106df 100644 --- a/docs/gen-ai/gen-ai-spans.md +++ b/docs/gen-ai/gen-ai-spans.md @@ -47,12 +47,12 @@ These attributes track input data and metadata for a request to an GenAI model. |---|---|---|---|---|---| | [`gen_ai.request.model`](/docs/attributes-registry/gen-ai.md) | string | The name of the GenAI model a request is being made to. [1] | `gpt-4` | `Required` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | [`gen_ai.system`](/docs/attributes-registry/gen-ai.md) | string | The Generative AI product as identified by the client instrumentation. [2] | `openai` | `Required` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| [`gen_ai.request.frequency_penalty`](/docs/attributes-registry/gen-ai.md) | double | The frequency penalty to provide. | `0.1` | `Recommended` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| [`gen_ai.request.frequency_penalty`](/docs/attributes-registry/gen-ai.md) | double | The frequency penalty setting for the GenAI request. | `0.1` | `Recommended` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | [`gen_ai.request.max_tokens`](/docs/attributes-registry/gen-ai.md) | int | The maximum number of tokens the model generates for a request. | `100` | `Recommended` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| [`gen_ai.request.presence_penalty`](/docs/attributes-registry/gen-ai.md) | double | The presence penalty to provide. | `0.1` | `Recommended` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| [`gen_ai.request.stop_sequences`](/docs/attributes-registry/gen-ai.md) | string[] | The stop sequences provided in the request. | `["\\n"]` | `Recommended` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| [`gen_ai.request.presence_penalty`](/docs/attributes-registry/gen-ai.md) | double | The presence penalty setting for the GenAI request. | `0.1` | `Recommended` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| [`gen_ai.request.stop_sequences`](/docs/attributes-registry/gen-ai.md) | string[] | List of sequences that the model will use to stop generating further tokens. | `["forest", "lived"]` | `Recommended` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | [`gen_ai.request.temperature`](/docs/attributes-registry/gen-ai.md) | double | The temperature setting for the GenAI request. | `0.0` | `Recommended` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| [`gen_ai.request.top_k`](/docs/attributes-registry/gen-ai.md) | double | The top_k sampling setting for the LLM request. | `1.0` | `Recommended` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| [`gen_ai.request.top_k`](/docs/attributes-registry/gen-ai.md) | double | The top_k sampling setting for the GenAI request. | `1.0` | `Recommended` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | [`gen_ai.request.top_p`](/docs/attributes-registry/gen-ai.md) | double | The top_p sampling setting for the GenAI request. | `1.0` | `Recommended` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | [`gen_ai.response.finish_reasons`](/docs/attributes-registry/gen-ai.md) | string[] | Array of reasons the model stopped generating tokens, corresponding to each generation received. | `["stop"]` | `Recommended` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | [`gen_ai.response.id`](/docs/attributes-registry/gen-ai.md) | string | The unique identifier for the completion. | `chatcmpl-123` | `Recommended` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |