Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce telegrafreceiver #935

Merged
merged 1 commit into from
Mar 3, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions cmd/otelcontribcol/unstable_components_enabled.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,12 @@ import (
"go.opentelemetry.io/collector/component"

"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/filelogreceiver"
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/telegrafreceiver"
)

func extraReceivers() []component.ReceiverFactory {
return []component.ReceiverFactory{
filelogreceiver.NewFactory(),
telegrafreceiver.NewFactory(),
}
}
5 changes: 5 additions & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ require (
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/simpleprometheusreceiver v0.0.0-00010101000000-000000000000
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/splunkhecreceiver v0.0.0-00010101000000-000000000000
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/statsdreceiver v0.0.0-00010101000000-000000000000
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/telegrafreceiver v0.0.0-00010101000000-000000000000
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/wavefrontreceiver v0.0.0-00010101000000-000000000000
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/windowsperfcountersreceiver v0.0.0-00010101000000-000000000000
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/zookeeperreceiver v0.0.0-00010101000000-000000000000
Expand All @@ -67,6 +68,8 @@ require (

replace go.opentelemetry.io/collector => github.com/SumoLogic/opentelemetry-collector v0.21.0-sumo

replace github.com/influxdata/telegraf => github.com/sumologic/telegraf v1.17.3-sumo

// Replace references to modules that are in this repository with their relateive paths
// so that we always build with current (latest) version of the source code.

Expand Down Expand Up @@ -182,6 +185,8 @@ replace github.com/open-telemetry/opentelemetry-collector-contrib/receiver/filel

replace github.com/open-telemetry/opentelemetry-collector-contrib/receiver/memcachedreceiver => ./receiver/memcachedreceiver

replace github.com/open-telemetry/opentelemetry-collector-contrib/receiver/telegrafreceiver => ./receiver/telegrafreceiver

replace github.com/open-telemetry/opentelemetry-collector-contrib/processor/cascadingfilterprocessor => ./processor/cascadingfilterprocessor

replace github.com/open-telemetry/opentelemetry-collector-contrib/processor/groupbytraceprocessor => ./processor/groupbytraceprocessor
Expand Down
310 changes: 302 additions & 8 deletions go.sum

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions receiver/telegrafreceiver/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
include ../../Makefile.Common
57 changes: 57 additions & 0 deletions receiver/telegrafreceiver/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# Telegraf Receiver
pmm-sumo marked this conversation as resolved.
Show resolved Hide resolved

Telegraf receiver for ingesting metrics from various [input plugins][input_plugins]
into otc pipeline.

Supported pipeline types: metrics

Use case: user configures telegraf input plugins in config for ingestion and otc
processors and exporters for data processing and export.

> :construction: This receiver is currently in **BETA** and is considered **unstable**.

[input_plugins]: https://github.com/influxdata/telegraf/tree/master/plugins/inputs

## Configuration

The following settings are required:

- `agent_config`: Telegraf config. For now it allows to provide agent and input
plugins configuration. One can refer to
[telegraf configuration docs][telegraf_config_docs] for full list of
configuration options.

The Following settings are optional:

- `separate_field` (default value is `false`): Specify whether metric field
should be added separately as data point label.

Example:

```yaml
receivers:
telegraf:
separate_field: false
agent_config: |
[agent]
interval = "2s"
flush_interval = "3s"
[[inputs.mem]]
```

The full list of settings exposed for this receiver are documented in
[config.go](./config.go).

[telegraf_config_docs]: https://github.com/influxdata/telegraf/blob/master/docs/CONFIGURATION.md

## Limitations

With its current implementation Telegraf receiver has the following limitations:

- only input plugins can be configured in telegraf agent confugration section
(apart from agent's configuration itself). That means that metrics go straight
from input plugin to the receiver for translation (into otc data model) without
any processing
- ony `telegraf.Gauge` metric data is supported, which translated (loosly) into
`pdata.MetricDataTypeDoubleGauge` and `pdata.MetricDataTypeIntGauge` depending
on the underlying data type
22 changes: 22 additions & 0 deletions receiver/telegrafreceiver/all.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
// Copyright 2021, OpenTelemetry Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package telegrafreceiver

import (
// _ "github.com/influxdata/telegraf/plugins/aggregators/all"
_ "github.com/influxdata/telegraf/plugins/inputs/all"
// _ "github.com/influxdata/telegraf/plugins/outputs/all"
// _ "github.com/influxdata/telegraf/plugins/processors/all"
)
34 changes: 34 additions & 0 deletions receiver/telegrafreceiver/config.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
// Copyright 2021, OpenTelemetry Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package telegrafreceiver

import (
"go.opentelemetry.io/collector/config/configmodels"
)

// Config defines configuration for the telegraf receiver.
type Config struct {
configmodels.ReceiverSettings `mapstructure:",squash"`

// AgentConfig is the yaml config used as telegraf configuration.
// Please note that only inputs should be configured as all metrics gathered
// by them will be passed through to otc pipeline for processing and export.
AgentConfig string `mapstructure:"agent_config"`

// SeparateField controls whether the ingested metrics should have a field
// concatenated with metric name like e.g. metric=mem_available or maybe rather
// have it as a separate label like e.g. metric=mem field=available
SeparateField bool `mapstructure:"separate_field"`
}
126 changes: 126 additions & 0 deletions receiver/telegrafreceiver/converter.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
// Copyright 2021, OpenTelemetry Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package telegrafreceiver

import (
"fmt"

"github.com/influxdata/telegraf"
"go.opentelemetry.io/collector/consumer/pdata"
)

const (
fieldLabel = "field"
)

type MetricConverter interface {
Convert(telegraf.Metric) (pdata.Metrics, error)
}

type metricConverter struct {
separateField bool
}

func newConverter(separateField bool) MetricConverter {
return metricConverter{
separateField: separateField,
}
}

// Convert converts telegraf.Metric to pdata.Metrics.
func (mc metricConverter) Convert(m telegraf.Metric) (pdata.Metrics, error) {
ms := pdata.NewMetrics()
rms := ms.ResourceMetrics()
rms.Resize(1)
rm := rms.At(0)

// Attach tags as attributes - pipe through the metadata
for _, t := range m.TagList() {
rm.Resource().Attributes().InsertString(t.Key, t.Value)
}

rm.InstrumentationLibraryMetrics().Resize(1)
ilm := rm.InstrumentationLibraryMetrics().At(0)

il := ilm.InstrumentationLibrary()
il.SetName(typeStr)
il.SetVersion(versionStr)

tim := m.Time().UnixNano()

metrics := ilm.Metrics()

switch t := m.Type(); t {
case telegraf.Gauge:
for _, f := range m.FieldList() {
pm := pdata.NewMetric()

if mc.separateField {
pm.SetName(m.Name())
} else {
pm.SetName(m.Name() + "_" + f.Key)
}

switch v := f.Value.(type) {
case float64:
pm.SetDataType(pdata.MetricDataTypeDoubleGauge)
dps := pm.DoubleGauge().DataPoints()
dps.Resize(1)
dp := dps.At(0)
dp.SetValue(v)
dp.SetTimestamp(pdata.TimestampUnixNano(tim))
if mc.separateField {
dp.LabelsMap().Insert(fieldLabel, f.Key)
}

case int64, uint64:
pm.SetDataType(pdata.MetricDataTypeIntGauge)
dps := pm.IntGauge().DataPoints()
dps.Resize(1)
dp := dps.At(0)
switch vv := v.(type) {
case int64:
dp.SetValue(vv)
case uint64:
dp.SetValue(int64(vv))
}

dp.SetTimestamp(pdata.TimestampUnixNano(tim))
if mc.separateField {
dp.LabelsMap().Insert(fieldLabel, f.Key)
}

default:
return pdata.Metrics{},
fmt.Errorf("unknown data type in telegraf.Gauge metric: %T", v)
}
metrics.Append(pm)
}

case telegraf.Counter:
return pdata.Metrics{}, fmt.Errorf("unsupported metric type: telegraf.Counter")
case telegraf.Untyped:
return pdata.Metrics{}, fmt.Errorf("unsupported metric type: telegraf.Untyped")
case telegraf.Summary:
return pdata.Metrics{}, fmt.Errorf("unsupported metric type: telegraf.Summary")
case telegraf.Histogram:
return pdata.Metrics{}, fmt.Errorf("unsupported metric type: telegraf.Histogram")

default:
return pdata.Metrics{}, fmt.Errorf("unknown metric type: %T", t)
}

return ms, nil
}
Loading