Skip to content

Commit

Permalink
Add support AWS Embedded Metric Format Version 0 (#20314)
Browse files Browse the repository at this point in the history 
Adding support for EMF version v0 and a flag for customers to make a difference between Version 0 and Version 1 (e.g EMF v0)
  • Loading branch information
@khanhntd
khanhntd authored Apr 25, 2023
1 parent 5f992df commit 132b116
Show file tree
Hide file tree
Showing 12 changed files with 205 additions and 141 deletions.
16 changes: 16 additions & 0 deletions .chloggen/add-support-emf-version-0.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
change_type: enhancement

# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver)
component: awsemfexporter

# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
note: Add support AWS Embedded Metric Format Version 0

# One or more tracking issues related to the change
issues: [20314]

# (Optional) One or more lines of additional information to render under the primary note.
# These lines will be padded with 2 spaces and then inserted directly into the document.
# Use pipe (|) for multiline entries.
subtext:
7 changes: 4 additions & 3 deletions exporter/awsemfexporter/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -32,12 +32,13 @@ The following exporter configuration parameters are supported.
| `max_retries` | Maximum number of retries before abandoning an attempt to post data. | 1 |
| `dimension_rollup_option` | DimensionRollupOption is the option for metrics dimension rollup. Three options are available: `NoDimensionRollup`, `SingleDimensionRollupOnly` and `ZeroAndSingleDimensionRollup` | "ZeroAndSingleDimensionRollup" (Enable both zero dimension rollup and single dimension rollup) |
| `resource_to_telemetry_conversion` | "resource_to_telemetry_conversion" is the option for converting resource attributes to telemetry attributes. It has only one config onption- `enabled`. For metrics, if `enabled=true`, all the resource attributes will be converted to metric labels by default. See `Resource Attributes to Metric Labels` section below for examples. | `enabled=false` |
| `output_destination` | "output_destination" is an option to specify the EMFExporter output. Currently, two options are available. "cloudwatch" or "stdout" | `cloudwatch` |
| `detailed_metrics` | Retain detailed datapoint values in exported metrics (e.g instead of exporting a quantile as a statistical value, preserve the quantile's population) | `false` |
| `output_destination` | Specify the EMFExporter output. Currently, two options are available. "cloudwatch" or "stdout" | `cloudwatch` |
| `detailed_metrics` | Retain detailed datapoint values in exported metrics (e.g instead of exporting a quantile as a statistical value, preserve the quantile's population) | `false` |
| `version` | Send metrics to CloudWatchLogs with Embedded Metric Format in selected version [(e.g version 1 with _aws)](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Embedded_Metric_Format_Specification.html#CloudWatch_Embedded_Metric_Format_Specification_structure), version 0 without _aws) | `1` |
| `parse_json_encoded_attr_values` | List of attribute keys whose corresponding values are JSON-encoded strings and will be converted to JSON structures in emf logs. For example, the attribute string value "{\\"x\\":5,\\"y\\":6}" will be converted to a json object: ```{"x": 5, "y": 6}``` | [ ] |
| [`metric_declarations`](#metric_declaration) | List of rules for filtering exported metrics and their dimensions. | [ ] |
| [`metric_descriptors`](#metric_descriptor) | List of rules for inserting or updating metric descriptors. | [ ] |
| `retain_initial_value_of_delta_metric` | This option specifies how the first value of a metric is handled. AWS EMF expects metric values to only contain deltas to the previous value. In the default case the first received value is therefor not sent to AWS but only used as a baseline for follow up changes to this metric. This is fine for high throughput metrics with stable labels (e.g. `requests{code=200}`). In this case it does not matter if the first value of this metric is discarded. However when your metric describes infrequent events or events with high label cardinality, then the exporter in default configuration would still drop the first occurrence of this metric. With this configuration value set to `true` the first value of all metrics will instead be send to AWS. | false |
| `retain_initial_value_of_delta_metric` | Specify how the first value of a metric is handled. AWS EMF expects metric values to only contain deltas to the previous value. In the default case the first received value is therefor not sent to AWS but only used as a baseline for follow up changes to this metric. This is fine for high throughput metrics with stable labels (e.g. `requests{code=200}`). In this case it does not matter if the first value of this metric is discarded. However when your metric describes infrequent events or events with high label cardinality, then the exporter in default configuration would still drop the first occurrence of this metric. With this configuration value set to `true` the first value of all metrics will instead be send to AWS. | false |

### metric_declaration
A metric_declaration section characterizes a rule to be used to set dimensions for exported metrics, filtered by the incoming metrics' labels and metric names.
Expand Down
12 changes: 10 additions & 2 deletions exporter/awsemfexporter/config.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ package awsemfexporter // import "github.com/open-telemetry/opentelemetry-collec
import (
"errors"

"go.opentelemetry.io/collector/component"
"go.uber.org/zap"

"github.com/open-telemetry/opentelemetry-collector-contrib/internal/aws/awsutil"
Expand Down Expand Up @@ -79,15 +80,20 @@ type Config struct {
// Note that at the moment in order to use this feature the value "kubernetes" must also be added to the ParseJSONEncodedAttributeValues array in order to be used
EKSFargateContainerInsightsEnabled bool `mapstructure:"eks_fargate_container_insights_enabled"`

// ResourceToTelemetrySettings is the option for converting resource attrihutes to telemetry attributes.
// ResourceToTelemetrySettings is an option for converting resource attrihutes to telemetry attributes.
// "Enabled" - A boolean field to enable/disable this option. Default is `false`.
// If enabled, all the resource attributes will be converted to metric labels by default.
ResourceToTelemetrySettings resourcetotelemetry.Settings `mapstructure:"resource_to_telemetry_conversion"`

// DetailedMetrics is the options for retaining detailed datapoint values in exported metrics (e.g instead of exporting a quantile as a statistical value,
// DetailedMetrics is an option for retaining detailed datapoint values in exported metrics (e.g instead of exporting a quantile as a statistical value,
// preserve the quantile's population)
DetailedMetrics bool `mapstructure:"detailed_metrics"`

// Version is an option for sending metrics to CloudWatchLogs with Embedded Metric Format in selected version (with "_aws")
// https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Embedded_Metric_Format_Specification.html#CloudWatch_Embedded_Metric_Format_Specification_structure
// Otherwise, sending metrics as Embedded Metric Format version 0 (without "_aws")
Version string `mapstructure:"version"`

// logger is the Logger used for writing error/warning logs
logger *zap.Logger
}
Expand All @@ -102,6 +108,8 @@ type MetricDescriptor struct {
Overwrite bool `mapstructure:"overwrite"`
}

var _ component.Config = (*Config)(nil)

// Validate filters out invalid metricDeclarations and metricDescriptors
func (config *Config) Validate() error {
var validDeclarations []*MetricDeclaration
Expand Down
3 changes: 3 additions & 0 deletions exporter/awsemfexporter/config_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ func TestLoadConfig(t *testing.T) {
LogStreamName: "",
DimensionRollupOption: "ZeroAndSingleDimensionRollup",
OutputDestination: "cloudwatch",
Version: "1",
logger: zap.NewNop(),
},
},
Expand All @@ -79,6 +80,7 @@ func TestLoadConfig(t *testing.T) {
LogStreamName: "",
DimensionRollupOption: "ZeroAndSingleDimensionRollup",
OutputDestination: "cloudwatch",
Version: "1",
ResourceToTelemetrySettings: resourcetotelemetry.Settings{Enabled: true},
logger: zap.NewNop(),
},
Expand All @@ -100,6 +102,7 @@ func TestLoadConfig(t *testing.T) {
LogStreamName: "",
DimensionRollupOption: "ZeroAndSingleDimensionRollup",
OutputDestination: "cloudwatch",
Version: "1",
MetricDescriptors: []MetricDescriptor{{
MetricName: "memcached_current_items",
Unit: "Count",
Expand Down
Loading

0 comments on commit 132b116

Please sign in to comment.