Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[receiver/windowsperfcountersreceiver] Update how metrics are established #8376

Merged
merged 26 commits into from
Mar 21, 2022
Merged
Show file tree
Hide file tree
Changes from 22 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@

### 🛑 Breaking changes 🛑

- `windowsperfcountersreceiver`: Added metrics configuration (#8376)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like I can't change this PR, so I can't fix this for you, but this now needs to be moved up to the "unreleased" section.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

- `mongodbatlasreceiver`: rename mislabeled attribute `memory_state` to correct `disk_status` on partition disk metrics (#7747)
- `mongodbatlasreceiver`: Correctly set initial lookback for querying mongodb atlas api (#8246)
- `nginxreceiver`: instrumentation name updated from `otelcol/nginx` to `otelcol/nginxreceiver` (#8255)
Expand Down
89 changes: 67 additions & 22 deletions receiver/windowsperfcountersreceiver/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,6 @@ interface](https://docs.microsoft.com/en-us/windows/win32/perfctrs/using-the-pdh
It is based on the [Telegraf Windows Performance Counters Input
Plugin](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/win_perf_counters).

Metrics will be generated with names and labels that match the performance
counter path, i.e.

- `Memory\Committed Bytes`
- `Processor\% Processor Time`, with a datapoint for each `Instance` label = (`_Total`, `1`, `2`, `3`, ... )

Expand All @@ -25,11 +22,25 @@ be configured:
```yaml
windowsperfcounters:
collection_interval: <duration> # default = "1m"
metrics:
<metric name>:
description: <description>
unit: <unit type>
gauge:
<metric name>:
description: <description>
unit: <unit type>
sum:
aggregation: <cumulative or delta>
monotonic: <true or false>
perfcounters:
- object: <object name>
instances: [<instance name>]*
counters:
- <counter name>
- name: <counter name>
metric: <metric name>
attributes:
<key>: <value>
```

*Note `instances` can have several special values depending on the type of
Expand All @@ -50,63 +61,97 @@ If you would like to scrape some counters at a different frequency than others,
you can configure multiple `windowsperfcounters` receivers with different
`collection_interval` values. For example:

```yaml
```yaml
djaglowski marked this conversation as resolved.
Show resolved Hide resolved
receivers:
windowsperfcounters/memory:
metrics:
bytes.committed:
description: the number of bytes committed to memory
unit: By
gauge:
collection_interval: 30s
perfcounters:
- object: Memory
counters:
- Committed Bytes
- name: Committed Bytes
metric: bytes.committed

windowsperfcounters/processor:
collection_interval: 1m
metrics:
processor.time:
description: active and idle time of the processor
unit: "%"
gauge:
perfcounters:
- object: "Processor"
instances: "*"
counters:
- "% Processor Time"
- name: "% Processor Time"
metric: processor.time
attributes:
state: active
- object: "Processor"
instances: [1, 2]
counters:
- "% Idle Time"
- name: "% Idle Time"
metric: processor.time
attributes:
state: idle

service:
pipelines:
metrics:
receivers: [windowsperfcounters/memory, windowsperfcounters/processor]
```

### Changing metric format
### Defining metric format

To report metrics in the desired output format, define a metric and reference it in the corresponding counter, along with any applicable attributes. The metric's data type can either be `gauge` (default) or `sum`.

| Field Name | Description | Value | Default |
| -- | -- | -- | -- |
| name | The key for the metric. | string | Counter Name |
| description | definition of what the metric measures. | string | |
| unit | what is being measured. | string | `1` |
| sum | representation of a sum metric. | Sum Config | |
| gauge | representation of a gauge metric. | Gauge Config | |

To report metrics in the desired output format, it's recommended you use this
receiver with the [metrics transform
processor](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/metricstransformprocessor).

#### Sum Config

| Field Name | Description | Value | Default |
| -- | -- | -- | -- |
| aggregation | The type of aggregation temporality for the metric. | [`cumulative` or `delta`] | |
| monotonic | whether or not the metric value can decrease. | false | |

#### Gauge Config

A `gauge` config currently accepts no settings. It is specified as an object for forwards compatibility.

e.g. To output the `Memory/Committed Bytes` counter as a metric with the name
`system.memory.usage`:
`bytes.committed`:

```yaml
receivers:
windowsperfcounters:
metrics:
bytes.committed:
description: the number of bytes committed to memory
unit: By
gauge:
value_type: int
djaglowski marked this conversation as resolved.
Show resolved Hide resolved
collection_interval: 30s
perfcounters:
- object: Memory
counters:
- Committed Bytes

processors:
metricstransformprocessor:
transforms:
- metric_name: "Memory/Committed Bytes"
action: update
new_name: system.memory.usage
- name: Committed Bytes
metric: bytes.committed

service:
pipelines:
metrics:
receivers: [windowsperfcounters]
processors: [metricstransformprocessor]
```

## Recommended configuration for common applications
Expand Down
71 changes: 63 additions & 8 deletions receiver/windowsperfcountersreceiver/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,14 +25,37 @@ import (
type Config struct {
scraperhelper.ScraperControllerSettings `mapstructure:",squash"`

PerfCounters []PerfCounterConfig `mapstructure:"perfcounters"`
MetricMetaData map[string]MetricConfig `mapstructure:"metrics"`
PerfCounters []PerfCounterConfig `mapstructure:"perfcounters"`
}

// PerfCounterConfig defines configuration for a perf counter object.
type PerfCounterConfig struct {
Object string `mapstructure:"object"`
Instances []string `mapstructure:"instances"`
Counters []string `mapstructure:"counters"`
Object string `mapstructure:"object"`
Instances []string `mapstructure:"instances"`
Counters []CounterConfig `mapstructure:"counters"`
}

// MetricsConfig defines the configuration for a metric to be created.
type MetricConfig struct {
Unit string `mapstructure:"unit"`
Description string `mapstructure:"description"`
Gauge GaugeMetric `mapstructure:"gauge"`
Sum SumMetric `mapstructure:"sum"`
}

type GaugeMetric struct {
}

type SumMetric struct {
Aggregation string `mapstructure:"aggregation"`
Monotonic bool `mapstructure:"monotonic"`
}

type CounterConfig struct {
Metric string `mapstructure:"metric"`
Name string `mapstructure:"name"`
Attributes map[string]string `mapstructure:"attributes"`
}

func (c *Config) Validate() error {
Expand All @@ -46,23 +69,55 @@ func (c *Config) Validate() error {
errs = multierr.Append(errs, fmt.Errorf("must specify at least one perf counter"))
}

for name, metric := range c.MetricMetaData {
if metric.Unit == "" {
metric.Unit = "1"
}

if (metric.Sum != SumMetric{}) {
if (metric.Gauge != GaugeMetric{}) {
errs = multierr.Append(errs, fmt.Errorf("metric %q provides both a sum config and a gauge config", name))
}

if metric.Sum.Aggregation != "cumulative" && metric.Sum.Aggregation != "delta" {
djaglowski marked this conversation as resolved.
Show resolved Hide resolved
errs = multierr.Append(errs, fmt.Errorf("sum metric %q includes an invalid aggregation", name))
}
}
}

var perfCounterMissingObjectName bool
for _, pc := range c.PerfCounters {
if pc.Object == "" {
perfCounterMissingObjectName = true
continue
}

if len(pc.Counters) == 0 {
errs = multierr.Append(errs, fmt.Errorf("perf counter for object %q does not specify any counters", pc.Object))
}

for _, counter := range pc.Counters {
if counter.Metric == "" {
continue
}

foundMatchingMetric := false
for name := range c.MetricMetaData {
if counter.Metric == name {
foundMatchingMetric = true
}
}
if !foundMatchingMetric {
errs = multierr.Append(errs, fmt.Errorf("perf counter for object %q includes an undefined metric", pc.Object))
}
djaglowski marked this conversation as resolved.
Show resolved Hide resolved
}

for _, instance := range pc.Instances {
if instance == "" {
errs = multierr.Append(errs, fmt.Errorf("perf counter for object %q includes an empty instance", pc.Object))
break
}
}

if len(pc.Counters) == 0 {
errs = multierr.Append(errs, fmt.Errorf("perf counter for object %q does not specify any counters", pc.Object))
}
}

if perfCounterMissingObjectName {
Expand Down
99 changes: 95 additions & 4 deletions receiver/windowsperfcountersreceiver/config_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -43,10 +43,27 @@ func TestLoadConfig(t *testing.T) {

r0 := cfg.Receivers[config.NewComponentID(typeStr)]
defaultConfigSingleObject := factory.CreateDefaultConfig()
defaultConfigSingleObject.(*Config).PerfCounters = []PerfCounterConfig{{Object: "object", Counters: []string{"counter"}}}

counterConfig := CounterConfig{
Name: "counter1",
Metric: "metric",
}
defaultConfigSingleObject.(*Config).PerfCounters = []PerfCounterConfig{{Object: "object", Counters: []CounterConfig{counterConfig}}}
defaultConfigSingleObject.(*Config).MetricMetaData = map[string]MetricConfig{
"metric": {
Description: "desc",
Unit: "1",
Gauge: GaugeMetric{},
},
}

assert.Equal(t, defaultConfigSingleObject, r0)

counterConfig2 := CounterConfig{
Name: "counter2",
Metric: "metric2",
}

r1 := cfg.Receivers[config.NewComponentIDWithName(typeStr, "customname")].(*Config)
expectedConfig := &Config{
ScraperControllerSettings: scraperhelper.ScraperControllerSettings{
Expand All @@ -56,18 +73,92 @@ func TestLoadConfig(t *testing.T) {
PerfCounters: []PerfCounterConfig{
{
Object: "object1",
Counters: []string{"counter1"},
Counters: []CounterConfig{counterConfig},
},
{
Object: "object2",
Counters: []string{"counter1", "counter2"},
Counters: []CounterConfig{counterConfig, counterConfig2},
},
},
MetricMetaData: map[string]MetricConfig{
"metric": {
Description: "desc",
Unit: "1",
Gauge: GaugeMetric{},
},
"metric2": {
Description: "desc",
Unit: "1",
Gauge: GaugeMetric{},
},
},
}

assert.Equal(t, expectedConfig, r1)
}

func TestLoadConfigNoMetrics(t *testing.T) {
testCases := []struct {
TestName string
TestPath string
Expected Config
}{
{
TestName: "NoMetricsDefined",
TestPath: filepath.Join("testdata", "config-nometrics.yaml"),
Expected: Config{
PerfCounters: []PerfCounterConfig{
{
Object: "object",
Counters: []CounterConfig{{Name: "counter1"}},
},
},
},
},
{
TestName: "NoMetricSpecified",
TestPath: filepath.Join("testdata", "config-nometricspecified.yaml"),
Expected: Config{
PerfCounters: []PerfCounterConfig{
{
Object: "object",
Counters: []CounterConfig{{Name: "counter1"}},
},
},
MetricMetaData: map[string]MetricConfig{
"metric": {
Description: "desc",
Unit: "1",
Gauge: GaugeMetric{},
},
},
},
},
}
for _, test := range testCases {
t.Run(test.TestName, func(t *testing.T) {
factories, err := componenttest.NopFactories()
require.NoError(t, err)

factory := NewFactory()
factories.Receivers[typeStr] = factory
cfg, err := servicetest.LoadConfigAndValidate(test.TestPath, factories)

require.NoError(t, err)
require.NotNil(t, cfg)

assert.Equal(t, len(cfg.Receivers), 1)

actualReceiver := cfg.Receivers[config.NewComponentID(typeStr)]
expectedReceiver := factory.CreateDefaultConfig()
expectedReceiver.(*Config).PerfCounters = test.Expected.PerfCounters
expectedReceiver.(*Config).MetricMetaData = test.Expected.MetricMetaData

assert.Equal(t, expectedReceiver, actualReceiver)
})
}
}

func TestLoadConfig_Error(t *testing.T) {
type testCase struct {
name string
Expand Down Expand Up @@ -117,8 +208,8 @@ func TestLoadConfig_Error(t *testing.T) {
"%s: %s; %s; %s; %s",
errorPrefix,
negativeCollectionIntervalErr,
fmt.Sprintf(emptyInstanceErr, "object"),
fmt.Sprintf(noCountersErr, "object"),
fmt.Sprintf(emptyInstanceErr, "object"),
noObjectNameErr,
),
},
Expand Down
Loading