Skip to content

Commit

Permalink
Implement googlemanagedprometheus exporter (#10925)
Browse files Browse the repository at this point in the history
implement googlemanagedprometheus exporter
  • Loading branch information
dashpole authored Jun 15, 2022
1 parent 4ef7bfe commit 3221691
Show file tree
Hide file tree
Showing 13 changed files with 766 additions and 32 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@

- `expvarreceiver`: Include `expvarreceiver` in components (#10847)
- `googlemanagedprometheusexporter` Add the Google Managed Service for Prometheus exporter. (#10840)
- `googlemanagedprometheusexporter` The Google Managed Service for Prometheus exporter is alpha. (#10925)

### 💡 Enhancements 💡

Expand Down
7 changes: 4 additions & 3 deletions cmd/configschema/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -48,9 +48,10 @@ require (
github.com/DataDog/sketches-go v1.4.1 // indirect
github.com/GehirnInc/crypt v0.0.0-20200316065508-bb7000b8a962 // indirect
github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v0.32.1 // indirect
github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/collector v0.30.2-0.20220512190557-875976f87023 // indirect
github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/trace v1.6.1 // indirect
github.com/GoogleCloudPlatform/opentelemetry-operations-go/internal/resourcemapping v0.30.1 // indirect
github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/collector v0.32.2-0.20220608154027-0c567eed530c // indirect
github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/collector/googlemanagedprometheus v0.32.1 // indirect
github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/trace v1.8.1 // indirect
github.com/GoogleCloudPlatform/opentelemetry-operations-go/internal/resourcemapping v0.32.1 // indirect
github.com/Microsoft/go-winio v0.5.2 // indirect
github.com/ReneKroon/ttlcache/v2 v2.11.0 // indirect
github.com/SAP/go-hdb v0.105.5 // indirect
Expand Down
14 changes: 8 additions & 6 deletions cmd/configschema/go.sum

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

53 changes: 51 additions & 2 deletions exporter/googlemanagedprometheusexporter/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,37 @@

| Status | |
| ------------------------ |-----------------------|
| Stability | [in development](https://github.com/open-telemetry/opentelemetry-collector#in-development) |
| Stability | [alpha](https://github.com/open-telemetry/opentelemetry-collector#alpha) |
| Supported pipeline types | metrics |
| Distributions | [] |

This exporter can be used to send metrics and traces to [Google Cloud Managed Service for Prometheus](https://cloud.google.com/stackdriver/docs/managed-prometheus). The difference between this exporter and the `googlecloud` exporter is that metrics sent with this exporter are queried using [promql](https://prometheus.io/docs/prometheus/latest/querying/basics/#querying-prometheus), rather than standard the standard MQL.

This exporter is not the standard method of ingesting metrics into Google Cloud Managed Service for Prometheus, which is built on a drop-in replacement for the Prometheus server: https://github.com/GoogleCloudPlatform/prometheus. This exporter does not support the full range of Prometheus functionality, including the UI, recording and alerting rules, and can't be used with the GMP Operator, but does support sending metrics.

## Configuration Reference

The following configuration options are supported:

- `project` (optional): GCP project identifier.
- `user_agent` (optional): Override the user agent string sent on requests to Cloud Monitoring (currently only applies to metrics). Specify `{{version}}` to include the application version number. Defaults to `opentelemetry-collector-contrib {{version}}`.
- `endpoint` (optional): Endpoint where metric data is going to be sent to. Replaces `endpoint`.
- `use_insecure` (optional): If true, use gRPC as their communication transport. Only has effect if Endpoint is not "".
- `retry_on_failure` (optional): Configuration for how to handle retries when sending data to Google Cloud fails.
- `enabled` (default = true)
- `initial_interval` (default = 5s): Time to wait after the first failure before retrying; ignored if `enabled` is `false`
- `max_interval` (default = 30s): Is the upper bound on backoff; ignored if `enabled` is `false`
- `max_elapsed_time` (default = 120s): Is the maximum amount of time spent trying to send a batch; ignored if `enabled` is `false`
- `sending_queue` (optional): Configuration for how to buffer traces before sending.
- `enabled` (default = true)
- `num_consumers` (default = 10): Number of consumers that dequeue batches; ignored if `enabled` is `false`
- `queue_size` (default = 5000): Maximum number of batches kept in memory before data; ignored if `enabled` is `false`;
User should calculate this as `num_seconds * requests_per_second` where:
- `num_seconds` is the number of seconds to buffer in case of a backend outage
- `requests_per_second` is the average number of requests per seconds.

Note: These `retry_on_failure` and `sending_queue` are provided (and documented) by the [Exporter Helper](https://github.com/open-telemetry/opentelemetry-collector/tree/main/exporter/exporterhelper#configuration)

## Example Configuration

```yaml
Expand All @@ -18,7 +41,7 @@ receivers:
config:
scrape_configs:
# Add your prometheus scrape configuration here.
# Using kubernetes_sd_configs with namespaced resources
# Using kubernetes_sd_configs with namespaced resources (e.g. pod)
# ensures the namespace is set on your metrics.
- job_name: 'kubernetes-pods'
kubernetes_sd_configs:
Expand Down Expand Up @@ -74,4 +97,30 @@ service:
exporters: [googlemanagedprometheus]
```
## Resource Attribute Handling
The Google Managed Prometheus exporter maps metrics to the
[prometheus_target](https://cloud.google.com/monitoring/api/resources#tag_prometheus_target)
monitored resource. The logic for mapping to monitored resources is designed to
be used with the prometheus receiver, but can be used with other receivers as
well. To avoid collisions (i.e. "duplicate timeseries enountered" errors), you
need to ensure the prometheus_target resource uniquely identifies the source of
metrics. The exporter uses the following resource attributes to determine
monitored resource:
* location: [`location` (see `groupbyattrs` config above), `cloud.availability_zone`, `cloud.region`]
* cluster: [`cluster` (see `groupbyattrs` config above), `k8s.cluster.name`]
* namespace: [`namespace` (see `groupbyattrs` config above), `k8s.namespace.name`]
* job: [`service.name` + `service.namespace`]
* instance: [`service.instance.id`]

In the configuration above, `cloud.availability_zone`, `cloud.region`, and
`k8s.cluster.name` are detected using the `resourcedetection` processor with
the `gcp` detector. The prometheus receiver sets `service.name` to the
configured `job_name`, and `service.instance.id` is set to the scrape target's
`instance`. The prometheus receiver sets `k8s.namespace.name` when using
`role: pod`. The `groupbyattrs` processor promotes `location`, `cluster`, and
`namespace` labels on metrics to resource labels, which allows overriding (e.g.
using prometheus metric_relabel_configs) the attributes discovered by other
receivers or processors. This is useful when metric exporters already have
`location`, `cluster`, or `namespace` labels, such as Kube-state-metrics.
39 changes: 35 additions & 4 deletions exporter/googlemanagedprometheusexporter/config.go
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
// Copyright 2022 Google LLC
// Copyright The OpenTelemetry Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// https://www.apache.org/licenses/LICENSE-2.0
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
Expand All @@ -17,25 +17,56 @@ package googlemanagedprometheusexporter // import "github.com/open-telemetry/ope
import (
"fmt"

"github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/collector"
"github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/collector/googlemanagedprometheus"
"go.opentelemetry.io/collector/config"
"go.opentelemetry.io/collector/exporter/exporterhelper"
)

// Config defines configuration for Google Cloud Managed Service for Prometheus exporter.
type Config struct {
config.ExporterSettings `mapstructure:",squash"`

// TODO: add config for the GMP exporter
GMPConfig `mapstructure:",squash"`

// Timeout for all API calls. If not set, defaults to 12 seconds.
exporterhelper.TimeoutSettings `mapstructure:",squash"` // squash ensures fields are correctly decoded in embedded struct.
exporterhelper.QueueSettings `mapstructure:"sending_queue"`
exporterhelper.RetrySettings `mapstructure:"retry_on_failure"`
}

// GMPConfig is a subset of the collector config applicable to the GMP exporter.
type GMPConfig struct {
ProjectID string `mapstructure:"project"`
UserAgent string `mapstructure:"user_agent"`
ClientConfig collector.ClientConfig `mapstructure:",squash"`
}

func (c *GMPConfig) toCollectorConfig() collector.Config {
// start with whatever the default collector config is.
cfg := collector.DefaultConfig()
// hard-code some config options to make it work with GMP
cfg.MetricConfig.Prefix = "prometheus.googleapis.com"
cfg.MetricConfig.SkipCreateMetricDescriptor = true
cfg.MetricConfig.InstrumentationLibraryLabels = false
cfg.MetricConfig.ServiceResourceLabels = false
// Update metric naming to match GMP conventions
cfg.MetricConfig.GetMetricName = googlemanagedprometheus.GetMetricName
// Map to the prometheus_target monitored resource
cfg.MetricConfig.MapMonitoredResource = googlemanagedprometheus.MapToPrometheusTarget
cfg.MetricConfig.EnableSumOfSquaredDeviation = true
// map the GMP config's fields to the collector config
cfg.ProjectID = c.ProjectID
cfg.UserAgent = c.UserAgent
cfg.MetricConfig.ClientConfig = c.ClientConfig
return cfg
}

func (cfg *Config) Validate() error {
if err := cfg.ExporterSettings.Validate(); err != nil {
return fmt.Errorf("exporter settings are invalid :%w", err)
}
if err := collector.ValidateConfig(cfg.toCollectorConfig()); err != nil {
return fmt.Errorf("exporter settings are invalid :%w", err)
}
return nil
}
69 changes: 69 additions & 0 deletions exporter/googlemanagedprometheusexporter/config_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
// Copyright The OpenTelemetry Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package googlemanagedprometheusexporter // import "github.com/open-telemetry/opentelemetry-collector-contrib/exporter/googlemanagedprometheusexporter"

import (
"path/filepath"
"testing"
"time"

"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"go.opentelemetry.io/collector/component/componenttest"
"go.opentelemetry.io/collector/config"
"go.opentelemetry.io/collector/exporter/exporterhelper"
"go.opentelemetry.io/collector/service/servicetest"
)

func TestLoadConfig(t *testing.T) {
factories, err := componenttest.NopFactories()
assert.Nil(t, err)

factory := NewFactory()
factories.Exporters[typeStr] = factory
cfg, err := servicetest.LoadConfigAndValidate(filepath.Join("testdata", "config.yaml"), factories)

require.NoError(t, err)
require.NotNil(t, cfg)

assert.Equal(t, len(cfg.Exporters), 2)

r0 := cfg.Exporters[config.NewComponentID(typeStr)].(*Config)
assert.Equal(t, r0, factory.CreateDefaultConfig().(*Config))

r1 := cfg.Exporters[config.NewComponentIDWithName(typeStr, "customname")].(*Config)
assert.Equal(t, r1,
&Config{
ExporterSettings: config.NewExporterSettings(config.NewComponentIDWithName(typeStr, "customname")),
TimeoutSettings: exporterhelper.TimeoutSettings{
Timeout: 20 * time.Second,
},
GMPConfig: GMPConfig{
ProjectID: "my-project",
UserAgent: "opentelemetry-collector-contrib {{version}}",
},
RetrySettings: exporterhelper.RetrySettings{
Enabled: true,
InitialInterval: 10 * time.Second,
MaxInterval: 1 * time.Minute,
MaxElapsedTime: 10 * time.Minute,
},
QueueSettings: exporterhelper.QueueSettings{
Enabled: true,
NumConsumers: 2,
QueueSize: 10,
},
})
}
23 changes: 18 additions & 5 deletions exporter/googlemanagedprometheusexporter/factory.go
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
// Copyright 2022 Google LLC
// Copyright The OpenTelemetry Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// https://www.apache.org/licenses/LICENSE-2.0
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
Expand All @@ -16,9 +16,9 @@ package googlemanagedprometheusexporter // import "github.com/open-telemetry/ope

import (
"context"
"fmt"
"time"

"github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/collector"
"go.opentelemetry.io/collector/component"
"go.opentelemetry.io/collector/config"
"go.opentelemetry.io/collector/exporter/exporterhelper"
Expand Down Expand Up @@ -54,6 +54,19 @@ func createMetricsExporter(
ctx context.Context,
params component.ExporterCreateSettings,
cfg config.Exporter) (component.MetricsExporter, error) {
// TODO: implement the metrics exporter
return nil, fmt.Errorf("not implemented")
eCfg := cfg.(*Config)
mExp, err := collector.NewGoogleCloudMetricsExporter(ctx, eCfg.GMPConfig.toCollectorConfig(), params.TelemetrySettings.Logger, params.BuildInfo.Version, eCfg.Timeout)
if err != nil {
return nil, err
}
return exporterhelper.NewMetricsExporter(
cfg,
params,
mExp.PushMetrics,
exporterhelper.WithShutdown(mExp.Shutdown),
// Disable exporterhelper Timeout, since we are using a custom mechanism
// within exporter itself
exporterhelper.WithTimeout(exporterhelper.TimeoutSettings{Timeout: 0}),
exporterhelper.WithQueue(eCfg.QueueSettings),
exporterhelper.WithRetry(eCfg.RetrySettings))
}
51 changes: 51 additions & 0 deletions exporter/googlemanagedprometheusexporter/factory_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
// Copyright The OpenTelemetry Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package googlemanagedprometheusexporter // import "github.com/open-telemetry/opentelemetry-collector-contrib/exporter/googlemanagedprometheusexporter"

import (
"context"
"os"
"testing"

"github.com/stretchr/testify/assert"
"go.opentelemetry.io/collector/component/componenttest"
"go.opentelemetry.io/collector/config/configtest"
)

func TestCreateDefaultConfig(t *testing.T) {
factory := NewFactory()
cfg := factory.CreateDefaultConfig()
assert.NotNil(t, cfg, "failed to create default config")
assert.NoError(t, configtest.CheckConfigStruct(cfg))
}

func TestCreateExporter(t *testing.T) {
if os.Getenv("GOOGLE_APPLICATION_CREDENTIALS") == "" {
t.Skip("Default credentials not set, skip creating Google Cloud exporter")
}
ctx := context.Background()
factory := NewFactory()
cfg := factory.CreateDefaultConfig()
eCfg := cfg.(*Config)
eCfg.ProjectID = "test"

te, err := factory.CreateTracesExporter(ctx, componenttest.NewNopExporterCreateSettings(), eCfg)
assert.NoError(t, err)
assert.NotNil(t, te, "failed to create trace exporter")

me, err := factory.CreateMetricsExporter(ctx, componenttest.NewNopExporterCreateSettings(), eCfg)
assert.NoError(t, err)
assert.NotNil(t, me, "failed to create metrics exporter")
}
Loading

0 comments on commit 3221691

Please sign in to comment.