Feature Flag evaluation event

.chloggen/1440.yaml

@@ -0,0 +1,4 @@
+change_type: enhancement
+component: feature_flag
+note: Define `feature_flag.evaluation` event.
+issues: [1440]

docs/attributes-registry/feature-flag.md

@@ -6,21 +6,61 @@
 
 # Feature Flag
 
+- [Feature Flag Attributes](#feature-flag-attributes)
+- [Deprecated Feature Flag Attributes](#deprecated-feature-flag-attributes)
+
 ## Feature Flag Attributes
 
 This document defines attributes for Feature Flags.
 
 | Attribute | Type | Description | Examples | Stability |
 |---|---|---|---|---|
-| <a id="feature-flag-key" href="#feature-flag-key">`feature_flag.key`</a> | string | The unique identifier of the feature flag. | `logo-color` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
-| <a id="feature-flag-provider-name" href="#feature-flag-provider-name">`feature_flag.provider_name`</a> | string | The name of the service provider that performs the flag evaluation. | `Flag Manager` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
-| <a id="feature-flag-variant" href="#feature-flag-variant">`feature_flag.variant`</a> | string | SHOULD be a semantic identifier for a value. If one is unavailable, a stringified version of the value can be used. [1] | `red`; `true`; `on` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| <a id="feature-flag-context-id" href="#feature-flag-context-id">`feature_flag.context.id`</a> | string | The unique identifier for the flag evaluation context. For example, the targeting key. | `5157782b-2203-4c80-a857-dbbd5e7761db` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| <a id="feature-flag-evaluation-error-code" href="#feature-flag-evaluation-error-code">`feature_flag.evaluation.error.code`</a> | string | An error code which denotes why a flag evaluation failed | `provider_not_ready`; `targeting_key_missing`; `provider_fatal`; `general` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| <a id="feature-flag-evaluation-error-message" href="#feature-flag-evaluation-error-message">`feature_flag.evaluation.error.message`</a> | string | A message explaining the nature of an error occurring during flag evaluation. | `Flag `header-color` expected type `string` but found type `number`` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| <a id="feature-flag-evaluation-reason" href="#feature-flag-evaluation-reason">`feature_flag.evaluation.reason`</a> | string | The reason code which shows how a feature flag value was determined. | `static`; `targeting_match`; `error`; `default` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| <a id="feature-flag-key" href="#feature-flag-key">`feature_flag.key`</a> | string | The lookup key of the feature flag. | `logo-color` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| <a id="feature-flag-set-id" href="#feature-flag-set-id">`feature_flag.set.id`</a> | string | The identifier of the [flag set](https://openfeature.dev/specification/glossary/#flag-set) to which the feature flag belongs. | `proj-1`; `ab98sgs`; `service1/dev` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| <a id="feature-flag-system" href="#feature-flag-system">`feature_flag.system`</a> | string | Identifies the feature flag provider. | `Flag Manager` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| <a id="feature-flag-variant" href="#feature-flag-variant">`feature_flag.variant`</a> | string | A semantic identifier for an evaluated flag value. [1] | `red`; `true`; `on` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| <a id="feature-flag-version" href="#feature-flag-version">`feature_flag.version`</a> | string | The version of the ruleset used during the evaluation. This may be any stable value which uniquely identifies the ruleset. | `1`; `01ABCDEF` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
 
 **[1]:** A semantic identifier, commonly referred to as a variant, provides a means
 for referring to a value without including the value itself. This can
 provide additional context for understanding the meaning behind a value.
 For example, the variant `red` maybe be used for the value `#c05543`.
 
-A stringified version of the value can be used in situations where a
-semantic identifier is unavailable. String representation of the value
-should be determined by the implementer.
+`feature_flag.evaluation.error.code` has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.
+
+| Value  | Description | Stability |
+|---|---|---|
+| `flag_not_found` | The flag could not be found. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| `general` | The error was for a reason not enumerated above. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| `invalid_context` | The evaluation context does not meet provider requirements. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| `parse_error` | An error was encountered parsing data, such as a flag configuration. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| `provider_fatal` | The provider has entered an irrecoverable error state. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| `provider_not_ready` | The value was resolved before the provider was initialized. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| `targeting_key_missing` | The provider requires a targeting key and one was not provided in the evaluation context. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| `type_mismatch` | The type of the flag value does not match the expected type. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+
+`feature_flag.evaluation.reason` has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.
+
+| Value  | Description | Stability |
+|---|---|---|
+| `cached` | The resolved value was retrieved from cache. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| `default` | The resolved value fell back to a pre-configured value (no dynamic evaluation occurred or dynamic evaluation yielded no result). | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| `disabled` | The resolved value was the result of the flag being disabled in the management system. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| `error` | The resolved value was the result of an error. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| `split` | The resolved value was the result of pseudorandom assignment. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| `stale` | The resolved value is non-authoritative or possibly out of date | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| `static` | The resolved value is static (no dynamic evaluation). | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| `targeting_match` | The resolved value was the result of a dynamic evaluation, such as a rule or specific user-targeting. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| `unknown` | The reason for the resolved value could not be determined. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+
+## Deprecated Feature Flag Attributes
+
+Describes deprecated Feature Flag attributes.
+
+| Attribute | Type | Description | Examples | Stability |
+|---|---|---|---|---|
+| <a id="feature-flag-provider-name" href="#feature-flag-provider-name">`feature_flag.provider_name`</a> | string | Deprecated, use `feature_flag.system` instead. | `Flag Manager` | ![Deprecated](https://img.shields.io/badge/-deprecated-red)<br>Replaced by `feature_flag.system`. |

docs/feature-flags/README.md

@@ -14,7 +14,6 @@ evaluations in spans and logs.
 
 Semantic conventions for feature flags are defined for the following signals:
 
-* [Feature Flags in Spans](feature-flags-spans.md): Semantic Conventions for recording feature flags in *spans*.
 * [Feature Flags in Logs](feature-flags-logs.md): Semantic Conventions for recording feature flags in *logs*.
 
 [DocumentStatus]: https://opentelemetry.io/docs/specs/otel/document-status

docs/feature-flags/feature-flags-logs.md

@@ -11,12 +11,16 @@ a [log record](https://github.com/open-telemetry/opentelemetry-specification/tre
 [Logger API](https://github.com/open-telemetry/opentelemetry-specification/tree/v1.37.0/specification/logs/bridge-api.md#emit-a-logrecord).
 This is useful when a flag is evaluated outside of a transaction context
 such as when the application loads or on a timer.
-To record a flag evaluation as a part of a transaction context,
-consider [recording it as a span event](feature-flags-spans.md).
 
-For more information about why it is useful to capture feature flag evaluations,
-refer to the [motivation](feature-flags-spans.md#motivation)
-section of the trace semantic convention for feature flag evaluations.
+## Motivation
+
+Features flags are commonly used in modern applications to decouple feature releases from deployments.
+Many feature flagging tools support the ability to update flag configurations in near real-time from a remote feature flag management service.
+They also commonly allow rulesets to be defined that return values based on contextual information.
+For example, a feature could be enabled only for a specific subset of users based on context (e.g. users email domain, membership tier, country).
+
+Since feature flags are dynamic and affect runtime behavior, it's important to collect relevant feature flag telemetry signals.
+This can be used to determine the impact a feature has on a request, enabling enhanced observability use cases, such as A/B testing or progressive feature releases.
 
 <!-- toc -->
 
@@ -37,7 +41,7 @@ context.
 The table below indicates which attributes should be added to the
 [LogRecord](https://github.com/open-telemetry/opentelemetry-specification/tree/v1.37.0/specification/logs/data-model.md#log-and-event-record-definition) and their types.
 
-<!-- semconv log-feature_flag -->
+<!-- semconv feature_flag.evaluation -->
 <!-- NOTE: THIS TEXT IS AUTOGENERATED. DO NOT EDIT BY HAND. -->
 <!-- see templates/registry/markdown/snippet.md.j2 -->
 <!-- prettier-ignore-start -->
@@ -46,24 +50,67 @@ The table below indicates which attributes should be added to the
 
 **Status:** ![Experimental](https://img.shields.io/badge/-experimental-blue)
 
-The event name MUST be `feature_flag`.
+The event name MUST be `feature_flag.evaluation`.
 
-This event describes feature flag evaluation representation on a Log Record.
+This semantic convention defines the attributes used to represent a feature flag evaluation as an event. A `feature_flag.evaluation` event SHOULD be emitted whenever a feature flag value is evaluated, which may happen many times over the course of an application lifecycle. For example, a website A/B testing different animations may evaluate a flag each time a button is clicked. A `feature_flag.evaluation` event is emitted on each evaluation even if the result is the same.
 
 | Attribute  | Type | Description  | Examples  | [Requirement Level](https://opentelemetry.io/docs/specs/semconv/general/attribute-requirement-level/) | Stability |
 |---|---|---|---|---|---|
-| [`feature_flag.key`](/docs/attributes-registry/feature-flag.md) | string | The unique identifier of the feature flag. | `logo-color` | `Recommended` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
-| [`feature_flag.provider_name`](/docs/attributes-registry/feature-flag.md) | string | The name of the service provider that performs the flag evaluation. | `Flag Manager` | `Recommended` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
-| [`feature_flag.variant`](/docs/attributes-registry/feature-flag.md) | string | SHOULD be a semantic identifier for a value. If one is unavailable, a stringified version of the value can be used. [1] | `red`; `true`; `on` | `Recommended` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| [`feature_flag.key`](/docs/attributes-registry/feature-flag.md) | string | The lookup key of the feature flag. | `logo-color` | `Required` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| [`feature_flag.evaluation.error.code`](/docs/attributes-registry/feature-flag.md) | string | An error code which denotes why a flag evaluation failed | `provider_not_ready`; `targeting_key_missing`; `provider_fatal`; `general` | `Conditionally Required` [1] | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| [`feature_flag.evaluation.error.message`](/docs/attributes-registry/feature-flag.md) | string | A message explaining the nature of an error occurring during flag evaluation. | `Flag `header-color` expected type `string` but found type `number`` | `Conditionally Required` [2] | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| [`feature_flag.variant`](/docs/attributes-registry/feature-flag.md) | string | A semantic identifier for an evaluated flag value. [3] | `red`; `true`; `on` | `Conditionally Required` [4] | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| [`feature_flag.context.id`](/docs/attributes-registry/feature-flag.md) | string | The unique identifier for the flag evaluation context. For example, the targeting key. | `5157782b-2203-4c80-a857-dbbd5e7761db` | `Recommended` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| [`feature_flag.evaluation.reason`](/docs/attributes-registry/feature-flag.md) | string | The reason code which shows how a feature flag value was determined. | `static`; `targeting_match`; `error`; `default` | `Recommended` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| [`feature_flag.set.id`](/docs/attributes-registry/feature-flag.md) | string | The identifier of the [flag set](https://openfeature.dev/specification/glossary/#flag-set) to which the feature flag belongs. | `proj-1`; `ab98sgs`; `service1/dev` | `Recommended` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| [`feature_flag.system`](/docs/attributes-registry/feature-flag.md) | string | Identifies the feature flag provider. | `Flag Manager` | `Recommended` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| [`feature_flag.version`](/docs/attributes-registry/feature-flag.md) | string | The version of the ruleset used during the evaluation. This may be any stable value which uniquely identifies the ruleset. | `1`; `01ABCDEF` | `Recommended` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+
+**[1]:** Required if an error occurred during flag evaluation and `feature_flag.evaluation.error.message` is not set, recommended otherwise.
 
-**[1]:** A semantic identifier, commonly referred to as a variant, provides a means
+**[2]:** Required if an error occurred during flag evaluation and `feature_flag.evaluation.error.code` is not set, recommended otherwise.
+
+**[3]:** A semantic identifier, commonly referred to as a variant, provides a means
 for referring to a value without including the value itself. This can
 provide additional context for understanding the meaning behind a value.
 For example, the variant `red` maybe be used for the value `#c05543`.
 
-A stringified version of the value can be used in situations where a
-semantic identifier is unavailable. String representation of the value
-should be determined by the implementer.
+**[4]:** If feature flag provider supplies a variant or equivalent concept.
+
+`feature_flag.evaluation.error.code` has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.
+
+| Value  | Description | Stability |
+|---|---|---|
+| `flag_not_found` | The flag could not be found. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| `general` | The error was for a reason not enumerated above. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| `invalid_context` | The evaluation context does not meet provider requirements. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| `parse_error` | An error was encountered parsing data, such as a flag configuration. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| `provider_fatal` | The provider has entered an irrecoverable error state. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| `provider_not_ready` | The value was resolved before the provider was initialized. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| `targeting_key_missing` | The provider requires a targeting key and one was not provided in the evaluation context. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| `type_mismatch` | The type of the flag value does not match the expected type. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+
+`feature_flag.evaluation.reason` has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.
+
+| Value  | Description | Stability |
+|---|---|---|
+| `cached` | The resolved value was retrieved from cache. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| `default` | The resolved value fell back to a pre-configured value (no dynamic evaluation occurred or dynamic evaluation yielded no result). | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| `disabled` | The resolved value was the result of the flag being disabled in the management system. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| `error` | The resolved value was the result of an error. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| `split` | The resolved value was the result of pseudorandom assignment. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| `stale` | The resolved value is non-authoritative or possibly out of date | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| `static` | The resolved value is static (no dynamic evaluation). | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| `targeting_match` | The resolved value was the result of a dynamic evaluation, such as a rule or specific user-targeting. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| `unknown` | The reason for the resolved value could not be determined. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+
+**Body fields:**
+
+| Body Field  | Type | Description  | Examples  | [Requirement Level](https://opentelemetry.io/docs/specs/semconv/general/attribute-requirement-level/) | Stability |
+|---|---|---|---|---|---|
+| `value` | undefined | The evaluated value of the feature flag. | `#ff0000`; `1`; `true` | `Conditionally Required` [1] | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+
+**[1]:** If and only if feature flag provider does not supply variant or equivalent concept. Otherwise, `value` should be treated as opt-in.
 
 <!-- markdownlint-restore -->
 <!-- prettier-ignore-end -->