From 9a27d71655408be70e6cedaff72f39a4c42e42c5 Mon Sep 17 00:00:00 2001 From: Josh MacDonald Date: Thu, 5 Nov 2020 15:48:49 -0800 Subject: [PATCH 1/6] Accumulator transient state requirement; Processor requirements outline --- specification/metrics/sdk.md | 78 +++++++++++++++++++++++++++++++++++- 1 file changed, 76 insertions(+), 2 deletions(-) diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index b65ee88a895..d47301590ff 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -250,6 +250,14 @@ synchronous instrument updates. The Accumulator SHOULD NOT hold an exclusive lock while calling an Aggregator (see below), since some Aggregators may have higher concurrency expectations. +State managed in the Accumulator MUST be transient. This requirement +ensures that export pipelines written constructed for stateless +exporters (e.g. Statsd, OTLP with a stateless ExportKindSelector) are +not penalized by permanent state in the Accumulator. This implies +that the use of long-term state in a Metrics export pipeline should be +elective, and such state if present should be managed in the Processor +component. + #### Accumulator: Collect() function The Accumulator MUST implement a Collect method that builds and @@ -268,9 +276,75 @@ Label Set, Resource, and metric Descriptor. TODO: _Are there more Accumulator functional requirements?_ -### Processor +### Processor: Component interface + +The Processor component interface supports interaction from the +Controller and Accumulator. The Controller, responsible for +initiating a new round of collection, informs the Processor when a +collection interval starts and finishes. After finishing the +collection interval, the Controller gets the ExportRecordSet before +calling the Exporter to export data. + +The Accumulator interacts with the Processor during the call to its +`Collect()` method, during which it calls `Process()` once per +ExportRecord on the Processor. + +The Processor component is meant to be used for managing long-term +state; it is also one of the locations in the Metrics export pipeline +where we can impemlement control over cardinality. There are two +reasons that long-term state is typically required in a Metric export +pipeline: + +1. Because the Exporter requests Cumulative aggregation temporality for Sum and/or Histogram data points +2. Because the Exporter requests keeping Memory about all metric label sets, regardless of the requested aggregation temporality. + +Note that both of these behaviors are typically required for a +Prometheus exporter and that when none of these behaviors are +configured, the Metrics export pipeline can be expected not to develop +long-term state. + +#### Basic Processor + +The basic Processor supports two standard ExportKindSelectors and the +independent Memory behavior described above. The default +OpenTelemetry Metrics SDK MUST provide a basic Processor meeting these +requirements. + +##### Basic Processor: CumulativeExportKindSelector + +CumulativeExportKindSelector is the default behavior, which requests +exporting Cumulative aggregation temporality for Sums and Histograms +and implies that label sets used with synchronous instruments will be +remembered indefinitely in the SDK. This ExportKindSelector is the +default in order support downstream Prometheus exporters "out of the +box". + +##### Basic Processor: StatelessExportKindSelector + +The StatelessExportKindSelector configures a Metric export pipeline +with no long-term memory requirements. In this selector, the Counter, +UpDownCounter, ValueRecorder, and ValueObserver instruments are +configured for Delta aggregation temporality while SumObserver and +UpDownSumObserver instruments are configured for Cumulative +aggregation temporality. This basic Processor configuration has no +long-term memory requirements because each instrument is +passed-through without any conversion. + +##### Basic Processor: Memory + +Some metrics exporter configurations request that the Metric export +pipeline maintain long-term state about historically reported Metric +timeseries. This option is a simple boolean that, when set, requires +the Processor to retain memory about all timeseries it has ever +exported. This option is only meaningful when reporting Cumulative +aggregation temporality. + +#### Reducing Processor -TODO _Processor functional requirements_ +The reducing Processor is a Processor interface implementation used in +conjunction with another (e.g., basic) Processor to drop labels in a +Metric export pipeline. The default OpenTelemetry SDK SHOULD provide +a Reducing Processor implementation. ### Controller From f89f78a165f9df338d769b2d347896f4f45f556d Mon Sep 17 00:00:00 2001 From: Josh MacDonald Date: Thu, 12 Nov 2020 10:59:49 -0800 Subject: [PATCH 2/6] Clarify stateless selector; ExportKind->AggregationTemporality --- specification/metrics/sdk.md | 34 +++++++++++++++++----------------- 1 file changed, 17 insertions(+), 17 deletions(-) diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index d47301590ff..75b7ec19c99 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -83,8 +83,8 @@ These are the significant data types used in the model architecture: - **Accumulation**: consists of Instrument, Label Set, Resource, and Aggregator snapshot, output by Accumulator - **Aggregation**: the result of aggregating one or more events by a specific aggregator, output by Processor - **AggregationKind**: describes the kind of read API the Aggregation supports (e.g., Sum) -- **ExportKind**: one of Delta, Cumulative, or Pass-Through -- **ExportKindSelector**: chooses which ExportKind to use for a metric instrument. +- **AggregationTemporality**: one of Delta, Cumulative +- **AggregationTemporalitySelector**: chooses which AggregationTemporality to use for a metric instrument. - **ExportRecord**: consists of Instrument, Label Set, Resource, Timestamp(s), and Aggregation - **ExportRecordSet**: a set of export records. @@ -252,7 +252,7 @@ Aggregators may have higher concurrency expectations. State managed in the Accumulator MUST be transient. This requirement ensures that export pipelines written constructed for stateless -exporters (e.g. Statsd, OTLP with a stateless ExportKindSelector) are +exporters (e.g. Statsd, OTLP with a stateless AggregationTemporalitySelector) are not penalized by permanent state in the Accumulator. This implies that the use of long-term state in a Metrics export pipeline should be elective, and such state if present should be managed in the Processor @@ -305,30 +305,30 @@ long-term state. #### Basic Processor -The basic Processor supports two standard ExportKindSelectors and the +The basic Processor supports two standard AggregationTemporalitySelectors and the independent Memory behavior described above. The default OpenTelemetry Metrics SDK MUST provide a basic Processor meeting these requirements. -##### Basic Processor: CumulativeExportKindSelector +##### Basic Processor: CumulativeAggregationTemporalitySelector -CumulativeExportKindSelector is the default behavior, which requests +CumulativeAggregationTemporalitySelector is the default behavior, which requests exporting Cumulative aggregation temporality for Sums and Histograms and implies that label sets used with synchronous instruments will be -remembered indefinitely in the SDK. This ExportKindSelector is the +remembered indefinitely in the SDK. This AggregationTemporalitySelector is the default in order support downstream Prometheus exporters "out of the box". -##### Basic Processor: StatelessExportKindSelector +##### Basic Processor: StatelessAggregationTemporalitySelector -The StatelessExportKindSelector configures a Metric export pipeline -with no long-term memory requirements. In this selector, the Counter, -UpDownCounter, ValueRecorder, and ValueObserver instruments are -configured for Delta aggregation temporality while SumObserver and +The StatelessAggregationTemporalitySelector configures a Metric export +pipeline with no long-term memory requirements. In this selector, the +Counter, UpDownCounter, ValueRecorder, and ValueObserver instruments +are configured for Delta aggregation temporality while SumObserver and UpDownSumObserver instruments are configured for Cumulative aggregation temporality. This basic Processor configuration has no -long-term memory requirements because each instrument is -passed-through without any conversion. +long-term memory requirements since the Exporter directly uses the +output of the Accumulator for every kind of instrument. ##### Basic Processor: Memory @@ -526,8 +526,8 @@ considered language-specific details. ### Export pipeline detail -TODO: define AggregatorSelector, Aggregator, Accumulation, ExportKind, -ExportKindSelector, Aggregation, AggregationKind ExportRecord, +TODO: define AggregatorSelector, Aggregator, Accumulation, AggregationTemporality, +AggregationTemporalitySelector, Aggregation, AggregationKind ExportRecord, ExportRecordSet ### Processor Detail @@ -536,7 +536,7 @@ TODO: define the Processor interface #### Basic Processor -TODO: define how ExportKind conversion works (delta->cumulative +TODO: define how AggregationTemporality conversion works (delta->cumulative required, cumulative->delta optional), Memory option (to not forget prior collection state). From f748d8014a7002c37ea47798653b98e775679e6e Mon Sep 17 00:00:00 2001 From: Josh MacDonald Date: Wed, 25 Nov 2020 21:29:26 -0800 Subject: [PATCH 3/6] Revert ExportKind->AggregationTemporality --- specification/metrics/sdk.md | 34 +++++++++++++++++----------------- 1 file changed, 17 insertions(+), 17 deletions(-) diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index 75b7ec19c99..d47301590ff 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -83,8 +83,8 @@ These are the significant data types used in the model architecture: - **Accumulation**: consists of Instrument, Label Set, Resource, and Aggregator snapshot, output by Accumulator - **Aggregation**: the result of aggregating one or more events by a specific aggregator, output by Processor - **AggregationKind**: describes the kind of read API the Aggregation supports (e.g., Sum) -- **AggregationTemporality**: one of Delta, Cumulative -- **AggregationTemporalitySelector**: chooses which AggregationTemporality to use for a metric instrument. +- **ExportKind**: one of Delta, Cumulative, or Pass-Through +- **ExportKindSelector**: chooses which ExportKind to use for a metric instrument. - **ExportRecord**: consists of Instrument, Label Set, Resource, Timestamp(s), and Aggregation - **ExportRecordSet**: a set of export records. @@ -252,7 +252,7 @@ Aggregators may have higher concurrency expectations. State managed in the Accumulator MUST be transient. This requirement ensures that export pipelines written constructed for stateless -exporters (e.g. Statsd, OTLP with a stateless AggregationTemporalitySelector) are +exporters (e.g. Statsd, OTLP with a stateless ExportKindSelector) are not penalized by permanent state in the Accumulator. This implies that the use of long-term state in a Metrics export pipeline should be elective, and such state if present should be managed in the Processor @@ -305,30 +305,30 @@ long-term state. #### Basic Processor -The basic Processor supports two standard AggregationTemporalitySelectors and the +The basic Processor supports two standard ExportKindSelectors and the independent Memory behavior described above. The default OpenTelemetry Metrics SDK MUST provide a basic Processor meeting these requirements. -##### Basic Processor: CumulativeAggregationTemporalitySelector +##### Basic Processor: CumulativeExportKindSelector -CumulativeAggregationTemporalitySelector is the default behavior, which requests +CumulativeExportKindSelector is the default behavior, which requests exporting Cumulative aggregation temporality for Sums and Histograms and implies that label sets used with synchronous instruments will be -remembered indefinitely in the SDK. This AggregationTemporalitySelector is the +remembered indefinitely in the SDK. This ExportKindSelector is the default in order support downstream Prometheus exporters "out of the box". -##### Basic Processor: StatelessAggregationTemporalitySelector +##### Basic Processor: StatelessExportKindSelector -The StatelessAggregationTemporalitySelector configures a Metric export -pipeline with no long-term memory requirements. In this selector, the -Counter, UpDownCounter, ValueRecorder, and ValueObserver instruments -are configured for Delta aggregation temporality while SumObserver and +The StatelessExportKindSelector configures a Metric export pipeline +with no long-term memory requirements. In this selector, the Counter, +UpDownCounter, ValueRecorder, and ValueObserver instruments are +configured for Delta aggregation temporality while SumObserver and UpDownSumObserver instruments are configured for Cumulative aggregation temporality. This basic Processor configuration has no -long-term memory requirements since the Exporter directly uses the -output of the Accumulator for every kind of instrument. +long-term memory requirements because each instrument is +passed-through without any conversion. ##### Basic Processor: Memory @@ -526,8 +526,8 @@ considered language-specific details. ### Export pipeline detail -TODO: define AggregatorSelector, Aggregator, Accumulation, AggregationTemporality, -AggregationTemporalitySelector, Aggregation, AggregationKind ExportRecord, +TODO: define AggregatorSelector, Aggregator, Accumulation, ExportKind, +ExportKindSelector, Aggregation, AggregationKind ExportRecord, ExportRecordSet ### Processor Detail @@ -536,7 +536,7 @@ TODO: define the Processor interface #### Basic Processor -TODO: define how AggregationTemporality conversion works (delta->cumulative +TODO: define how ExportKind conversion works (delta->cumulative required, cumulative->delta optional), Memory option (to not forget prior collection state). From f92244c1e0ac50ddba1be4f9b3488db9b1fbf725 Mon Sep 17 00:00:00 2001 From: Josh MacDonald Date: Wed, 2 Dec 2020 14:54:19 -0800 Subject: [PATCH 4/6] Editorial re: Processor state --- specification/metrics/sdk.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index d47301590ff..d786f5b3ef0 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -251,10 +251,10 @@ exclusive lock while calling an Aggregator (see below), since some Aggregators may have higher concurrency expectations. State managed in the Accumulator MUST be transient. This requirement -ensures that export pipelines written constructed for stateless -exporters (e.g. Statsd, OTLP with a stateless ExportKindSelector) are -not penalized by permanent state in the Accumulator. This implies -that the use of long-term state in a Metrics export pipeline should be +ensures that export pipelines constructed for stateless exporters +(e.g. Statsd, OTLP with a stateless ExportKindSelector) are not forced +into the use of permanent state in the Accumulator. This implies that +the use of long-term state in a Metrics export pipeline should be elective, and such state if present should be managed in the Processor component. From eac3c44034ee469d8f2af10b2e624d6d63b2c0a1 Mon Sep 17 00:00:00 2001 From: Josh MacDonald Date: Wed, 2 Dec 2020 23:51:42 -0800 Subject: [PATCH 5/6] Rewrite Accumulator requirements first section --- specification/metrics/sdk.md | 52 +++++++++++++++++++++++++----------- 1 file changed, 37 insertions(+), 15 deletions(-) diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index d786f5b3ef0..4d983942e5d 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -88,6 +88,9 @@ These are the significant data types used in the model architecture: - **ExportRecord**: consists of Instrument, Label Set, Resource, Timestamp(s), and Aggregation - **ExportRecordSet**: a set of export records. +TODO(jmacd): rename ExportKind to AggregationTemporality, +ExportKindSelector to AggregationTemporalitySelector. + The term **SDK instrument** refers to the underlying implementation of an instrument. @@ -228,18 +231,33 @@ Accumulator, with detail shown for synchronous instruments. ![Metrics SDK Accumulator Detail Diagram](img/accumulator-detail.png) -For a synchronous instrument, the Accumulator will: +The Accumulator's primary tasks are to aggregate synchronous metric +events over a collection interval, and then at end of the interval, to +evaluate asynchronous callbacks and finally snapshot current +Aggregators for passing to the Processor. + +The Accumulator MUST be configured with an AggregatorSelector +interface that is used to assign new Aggregators to instruments as +they are needed. -1. Map each active Label Set to a record, consisting of two instances of the same type Aggregator -2. Enter new records into the mapping, calling the AggregationSelector if needed -3. Update the current Aggregator instance, responding to concurrent API events -4. Call Aggregator.SynchronizedMove on the current Aggregator instance to: (a) copy its value into the snapshot Aggregator instance and (b) reset the current Aggregator to the zero state -5. Call Processor.Process for every resulting Accumulation (i.e., Instrument, Label Set, Resource, and Aggregator snapshot) +The Accumulator MUST ensure that metric events for a given instrument +and identical label set that occur within a single collection interval +are passed to the same Aggregator. -The Accumulator MUST provide the option to associate a -[`Resource`](../resource/sdk.md) with the Accumulations that it +The Accumulator MUST snapshot the current value and reset the state of +every Aggregator that was used during a collection interval. The +Aggregator snapshot, together with the instrument descriptor, label +set, and Resource, define an Accumulation and are passed to the +Processor. + +The Accumulator MUST provide the option to configure the +[`Resource`](../resource/sdk.md) associated with Accumulations that it produces. +Aggregators that are not used during a collection interval MUST not +yield Accumulations for that collection interval, when no events or +observations happen. + Synchronous metric instruments are expected to be used concurrently. Unless concurrency is not a feature of the source language, the SDK Accumulator component SHOULD be designed with concurrent performance @@ -250,13 +268,17 @@ synchronous instrument updates. The Accumulator SHOULD NOT hold an exclusive lock while calling an Aggregator (see below), since some Aggregators may have higher concurrency expectations. -State managed in the Accumulator MUST be transient. This requirement -ensures that export pipelines constructed for stateless exporters -(e.g. Statsd, OTLP with a stateless ExportKindSelector) are not forced -into the use of permanent state in the Accumulator. This implies that -the use of long-term state in a Metrics export pipeline should be -elective, and such state if present should be managed in the Processor -component. +The Accumulator MUST eliminate from memory (i.e. "forget") state +associated with label sets used in earlier collection intervals, after +they are not for a suitable amount of time. A "suitable amount of +time" is intentionally not specific, since implementations may wish to +optimize memory management and have to contend with concurrent access. +This requirement ensures that export pipelines constructed for +stateless exporters (e.g. Statsd, OTLP with a stateless +ExportKindSelector) are not forced into the use of permanent state in +the Accumulator. This implies that the use of long-term state in a +Metrics export pipeline should be elective, and such state if present +should be managed in the Processor component. #### Accumulator: Collect() function From 62f2a9e4875c350bb381bbb348c2639896b9c196 Mon Sep 17 00:00:00 2001 From: Josh MacDonald Date: Wed, 2 Dec 2020 23:59:23 -0800 Subject: [PATCH 6/6] Use Accumulations, not 'instrument' for what passes-through here --- specification/metrics/sdk.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index 4d983942e5d..d6e8b520eb5 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -349,8 +349,10 @@ UpDownCounter, ValueRecorder, and ValueObserver instruments are configured for Delta aggregation temporality while SumObserver and UpDownSumObserver instruments are configured for Cumulative aggregation temporality. This basic Processor configuration has no -long-term memory requirements because each instrument is -passed-through without any conversion. +long-term memory requirements because the instrument temporality +matches the aggregation temporality, meaning Accumulations "pass +through" the Processor without requiring additional memory for +temporality conversion. ##### Basic Processor: Memory