From 6074e1c0f4711b45ca67716296d4b9f04ed589ba Mon Sep 17 00:00:00 2001 From: Tao Chen Date: Wed, 8 May 2024 22:20:11 -0700 Subject: [PATCH 1/5] Extend gen-ai system constant values --- model/registry/gen-ai.yaml | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/model/registry/gen-ai.yaml b/model/registry/gen-ai.yaml index 100523a3b8..f8956e7f0a 100644 --- a/model/registry/gen-ai.yaml +++ b/model/registry/gen-ai.yaml @@ -14,6 +14,26 @@ groups: stability: experimental value: "openai" brief: 'OpenAI' + - id: microsoft + stability: experimental + value: "microsoft" + brief: 'Microsoft' + - id: huggingface + stability: experimental + value: "huggingface" + brief: 'Hugging Face' + - id: google + stability: experimental + value: "google" + brief: 'Google' + - id: mistral + stability: experimental + value: "mistral" + brief: 'Mistral' + - id: cohere + stability: experimental + value: "cohere" + brief: 'Cohere' brief: The name of the LLM foundation model vendor. examples: 'openai' tag: llm-generic-request From 469bcb3b7ca42ba69a6635541fc9e40ebc8670d4 Mon Sep 17 00:00:00 2001 From: Tao Chen Date: Thu, 9 May 2024 05:29:07 +0000 Subject: [PATCH 2/5] Add more providers and generate markdowns --- docs/attributes-registry/gen-ai.md | 15 +++++++++++---- docs/gen-ai/llm-spans.md | 7 +++++++ model/registry/gen-ai.yaml | 8 ++++++++ 3 files changed, 26 insertions(+), 4 deletions(-) diff --git a/docs/attributes-registry/gen-ai.md b/docs/attributes-registry/gen-ai.md index e5853b5363..7564e1aad1 100644 --- a/docs/attributes-registry/gen-ai.md +++ b/docs/attributes-registry/gen-ai.md @@ -21,7 +21,7 @@ This document defines the attributes used to describe telemetry in the context o | `gen_ai.response.finish_reasons` | string[] | Array of reasons the model stopped generating tokens, corresponding to each generation received. | `stop` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `gen_ai.response.id` | string | The unique identifier for the completion. | `chatcmpl-123` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `gen_ai.response.model` | string | The name of the LLM a response was generated from. | `gpt-4-0613` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `gen_ai.system` | string | The name of the LLM foundation model vendor. | `openai` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `gen_ai.system` | string | The name of the LLM foundation model vendor. | `openai`; `microsoft`; `huggingface` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `gen_ai.usage.completion_tokens` | int | The number of tokens used in the LLM response (completion). | `180` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `gen_ai.usage.prompt_tokens` | int | The number of tokens used in the LLM prompt. | `100` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | @@ -30,6 +30,13 @@ This document defines the attributes used to describe telemetry in the context o `gen_ai.system` has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used. -| Value | Description | Stability | -| -------- | ----------- | ---------------------------------------------------------------- | -| `openai` | OpenAI | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| Value | Description | Stability | +| ------------- | ------------ | ---------------------------------------------------------------- | +| `openai` | OpenAI | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `microsoft` | Microsoft | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `huggingface` | Hugging Face | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `google` | Google | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `meta` | Meta | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `anthropic` | Anthropic | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `mistral` | Mistral | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `cohere` | Cohere | ![Experimental](https://img.shields.io/badge/-experimental-blue) | diff --git a/docs/gen-ai/llm-spans.md b/docs/gen-ai/llm-spans.md index 5cf34e5ffd..bda4e10db5 100644 --- a/docs/gen-ai/llm-spans.md +++ b/docs/gen-ai/llm-spans.md @@ -61,6 +61,13 @@ These attributes track input data and metadata for a request to an LLM. Each att | Value | Description | Stability | |---|---|---| | `openai` | OpenAI | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `microsoft` | Microsoft | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `huggingface` | Hugging Face | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `google` | Google | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `meta` | Meta | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `anthropic` | Anthropic | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `mistral` | Mistral | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `cohere` | Cohere | ![Experimental](https://img.shields.io/badge/-experimental-blue) | ## Events diff --git a/model/registry/gen-ai.yaml b/model/registry/gen-ai.yaml index f8956e7f0a..04c4edf61a 100644 --- a/model/registry/gen-ai.yaml +++ b/model/registry/gen-ai.yaml @@ -26,6 +26,14 @@ groups: stability: experimental value: "google" brief: 'Google' + - id: meta + stability: experimental + value: "meta" + brief: 'Meta' + - id: anthropic + stability: experimental + value: "anthropic" + brief: "Anthropic" - id: mistral stability: experimental value: "mistral" From 6b59f94d8d122a9d80a059c9b54d1db03dfb06e5 Mon Sep 17 00:00:00 2001 From: Tao Chen Date: Thu, 30 May 2024 14:58:32 +0000 Subject: [PATCH 3/5] Remove Azure OAI --- docs/attributes-registry/gen-ai.md | 13 ++++++------- docs/gen-ai/llm-spans.md | 1 - model/registry/gen-ai.yaml | 4 ---- 3 files changed, 6 insertions(+), 12 deletions(-) diff --git a/docs/attributes-registry/gen-ai.md b/docs/attributes-registry/gen-ai.md index e46c15a6f7..01742675ab 100644 --- a/docs/attributes-registry/gen-ai.md +++ b/docs/attributes-registry/gen-ai.md @@ -33,10 +33,9 @@ This document defines the attributes used to describe telemetry in the context o `gen_ai.system` has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used. -| Value | Description | Stability | -| -------------- | ------------ | ---------------------------------------------------------------- | -| `openai` | OpenAI | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `azure_openai` | Azure OpenAI | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `vertex_ai` | Vertex AI | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `anthropic` | Anthropic | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `cohere` | Cohere | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| Value | Description | Stability | +| ----------- | ----------- | ---------------------------------------------------------------- | +| `openai` | OpenAI | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `vertex_ai` | Vertex AI | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `anthropic` | Anthropic | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `cohere` | Cohere | ![Experimental](https://img.shields.io/badge/-experimental-blue) | diff --git a/docs/gen-ai/llm-spans.md b/docs/gen-ai/llm-spans.md index 4ee968c9bc..7b4972c179 100644 --- a/docs/gen-ai/llm-spans.md +++ b/docs/gen-ai/llm-spans.md @@ -69,7 +69,6 @@ These attributes track input data and metadata for a request to an LLM. Each att | Value | Description | Stability | |---|---|---| | `openai` | OpenAI | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `azure_openai` | Azure OpenAI | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `vertex_ai` | Vertex AI | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `anthropic` | Anthropic | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `cohere` | Cohere | ![Experimental](https://img.shields.io/badge/-experimental-blue) | diff --git a/model/registry/gen-ai.yaml b/model/registry/gen-ai.yaml index a10fd59c40..0438ab9099 100644 --- a/model/registry/gen-ai.yaml +++ b/model/registry/gen-ai.yaml @@ -14,10 +14,6 @@ groups: stability: experimental value: "openai" brief: 'OpenAI' - - id: azure_openai - stability: experimental - value: "azure_openai" - brief: 'Azure OpenAI' - id: vertex_ai stability: experimental value: "vertex_ai" From 17d2468687165de3f7f42a9b9f0ba5b8afca0a68 Mon Sep 17 00:00:00 2001 From: Tao Chen Date: Tue, 4 Jun 2024 15:49:54 +0000 Subject: [PATCH 4/5] Remove whitespaces --- model/registry/gen-ai.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/model/registry/gen-ai.yaml b/model/registry/gen-ai.yaml index bc1187ee12..9e0f3cac12 100644 --- a/model/registry/gen-ai.yaml +++ b/model/registry/gen-ai.yaml @@ -25,7 +25,7 @@ groups: - id: cohere stability: experimental value: "cohere" - brief: 'Cohere' + brief: 'Cohere' brief: The Generative AI product as identified by the client instrumentation. note: > The actual GenAI product may differ from the one identified by the client. From 3822ed57a653890d50c6ecdd26740654df5962df Mon Sep 17 00:00:00 2001 From: Tao Chen Date: Tue, 4 Jun 2024 15:54:10 +0000 Subject: [PATCH 5/5] Add change log --- .chloggen/extend-gen-ai-system-values.yaml | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) create mode 100755 .chloggen/extend-gen-ai-system-values.yaml diff --git a/.chloggen/extend-gen-ai-system-values.yaml b/.chloggen/extend-gen-ai-system-values.yaml new file mode 100755 index 0000000000..619668e7d2 --- /dev/null +++ b/.chloggen/extend-gen-ai-system-values.yaml @@ -0,0 +1,22 @@ +# Use this changelog template to create an entry for release notes. +# +# If your change doesn't affect end users you should instead start +# your pull request title with [chore] or use the "Skip Changelog" label. + +# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix' +change_type: enhancement + +# The name of the area of concern in the attributes-registry, (e.g. http, cloud, db) +component: gen-ai + +# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`). +note: Add 3 well-known gen-ai systems as reference values of the gen-ai system attribute + +# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists. +# The values here must be integers. +issues: [ 1020 ] + +# (Optional) One or more lines of additional information to render under the primary note. +# These lines will be padded with 2 spaces and then inserted directly into the document. +# Use pipe (|) for multiline entries. +subtext: