Skip to content

Commit

Permalink
add OpenCensus metric exporter bridge
Browse files Browse the repository at this point in the history
  • Loading branch information
dashpole committed Jan 11, 2021
1 parent 5ed96e9 commit 11314d9
Show file tree
Hide file tree
Showing 9 changed files with 1,305 additions and 10 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
- `NewSplitDriver` for OTLP exporter that allows sending traces and metrics to different endpoints. (#1418)
- Add codeql worfklow to GitHub Actions (#1428)
- Added Gosec workflow to GitHub Actions (#1429)
- Add an OpenCensus exporter bridge. (#1444)

### Changed

Expand Down
58 changes: 54 additions & 4 deletions bridge/opencensus/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,9 @@

The OpenCensus Bridge helps facilitate the migration of an application from OpenCensus to OpenTelemetry.

## The Problem: Mixing OpenCensus and OpenTelemetry libraries
## Tracing

### The Problem: Mixing OpenCensus and OpenTelemetry libraries

In a perfect world, one would simply migrate their entire go application --including custom instrumentation, libraries, and exporters-- from OpenCensus to OpenTelemetry all at once. In the real world, dependency constraints, third-party ownership of libraries, or other reasons may require mixing OpenCensus and OpenTelemetry libraries in a single application.

Expand Down Expand Up @@ -44,10 +46,12 @@ The bridge implements the OpenCensus trace API using OpenTelemetry. This would

### User Journey

Starting from an application using entirely OpenCensus APIs:

1. Instantiate OpenTelemetry SDK and Exporters
2. Override OpenCensus' DefaultTracer with the bridge
3. Migrate libraries from OpenCensus to OpenTelemetry
4. Remove OpenCensus Exporters
3. Migrate libraries individually from OpenCensus to OpenTelemetry
4. Remove OpenCensus exporters and configuration

To override OpenCensus' DefaultTracer with the bridge:
```golang
Expand All @@ -63,10 +67,56 @@ octrace.DefaultTracer = opencensus.NewTracer(tracer)

Be sure to set the `Tracer` name to your instrumentation package name instead of `"bridge"`.

### Incompatibilities
#### Incompatibilities

OpenCensus and OpenTelemetry APIs are not entirely compatible. If the bridge finds any incompatibilities, it will log them. Incompatibilities include:

* Custom OpenCensus Samplers specified during StartSpan are ignored.
* Links cannot be added to OpenCensus spans.
* OpenTelemetry Debug or Deferred trace flags are dropped after an OpenCensus span is created.

## Metrics

### The problem: mixing libraries without mixing pipelines

The problem for monitoring is simpler than the problem for tracing, since there
are no context propagation issues to deal with. However, it still is difficult
for users to migrate an entire applications' monitoring at once. It
should be possible to send metrics generated by OpenCensus libraries to an
OpenTelemetry pipeline so that migrating a metric does not require maintaining
separate export pipelines for OpenCensus and OpenTelemetry.

### The Exporter "wrapper" solution

The solution we use here is to allow wrapping an OpenTelemetry exporter such
that it implements the OpenCensus exporter interfaces. This allows a single
exporter to be used for metrics from *both* OpenCensus and OpenTelemetry.

### User Journy

Starting from an application using entirely OpenCensus APIs:

1. Instantiate OpenTelemetry SDK and Exporters.
2. Replace OpenCensus exporters with a wrapped OpenTelemetry exporter from step 1.
3. Migrate libraries individually from OpenCensus to OpenTelemetry
4. Remove OpenCensus Exporters and configuration.

For example, to swap out the OpenCensus logging exporter for the OpenTelemetry stdout exporter:
```golang
import (
"go.opencensus.io/metric/metricexport"
"go.opentelemetry.io/otel/bridge/opencensus"
"go.opentelemetry.io/otel/exporters/stdout"
"go.opentelemetry.io/otel"
)
// With OpenCensus, you could have previously configured the logging exporter like this:
// import logexporter "go.opencensus.io/examples/exporter"
// exporter, _ := logexporter.NewLogExporter(logexporter.Options{})
// Instead, we can create an equivalent using the OpenTelemetry stdout exporter:
openTelemetryExporter, _ := stdout.NewExporter(stdout.WithPrettyPrint())
exporter := opencensus.NewMetricExporter(openTelemetryExporter)

// Use the wrapped OpenTelemetry exporter like you normally would with OpenCensus
intervalReader, _ := metricexport.NewIntervalReader(&metricexport.Reader{}, exporter)
intervalReader.Start()
```
174 changes: 174 additions & 0 deletions bridge/opencensus/aggregation.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,174 @@
// Copyright The OpenTelemetry Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package opencensus

import (
"errors"
"fmt"
"time"

"go.opencensus.io/metric/metricdata"

"go.opentelemetry.io/otel/metric/number"
"go.opentelemetry.io/otel/sdk/export/metric/aggregation"
)

var (
errIncompatibleType = errors.New("incompatible type for aggregation")
errEmpty = errors.New("points may not be empty")
errBadPoint = errors.New("point cannot be converted")
)

// aggregationWithEndTime is an aggregation that can also provide the timestamp
// of the last recorded point.
type aggregationWithEndTime interface {
aggregation.Aggregation
end() time.Time
}

// newAggregationFromPoints creates an OpenTelemetry aggregation from
// OpenCensus points. Points may not be empty.
func newAggregationFromPoints(points []metricdata.Point) (aggregationWithEndTime, error) {
if len(points) == 0 {
return nil, errEmpty
}
switch t := points[0].Value.(type) {
case int64:
return newExactAggregator(points)
case float64:
return newExactAggregator(points)
case *metricdata.Distribution:
return newDistributionAggregator(points)
default:
// TODO add *metricdata.Summary support
return nil, fmt.Errorf("%w: %v", errIncompatibleType, t)
}
}

var _ aggregation.Aggregation = &ocExactAggregator{}
var _ aggregation.LastValue = &ocExactAggregator{}
var _ aggregation.Points = &ocExactAggregator{}

// newExactAggregator creates an OpenTelemetry aggreation from OpenCensus points.
// Points may not be empty, and must only contain integers or floats.
func newExactAggregator(pts []metricdata.Point) (aggregationWithEndTime, error) {
points := make([]number.Number, len(pts))
for i, pt := range pts {
switch t := pt.Value.(type) {
case int64:
points[i] = number.NewInt64Number(pt.Value.(int64))
case float64:
points[i] = number.NewFloat64Number(pt.Value.(float64))
default:
return nil, fmt.Errorf("%w: %v", errIncompatibleType, t)
}
}
return &ocExactAggregator{
points: points,
endTime: pts[len(pts)-1].Time,
}, nil
}

type ocExactAggregator struct {
points []number.Number
endTime time.Time
}

// Kind returns the kind of aggregation this is.
func (o *ocExactAggregator) Kind() aggregation.Kind {
return aggregation.ExactKind
}

// Points returns access to the raw data set.
func (o *ocExactAggregator) Points() ([]number.Number, error) {
return o.points, nil
}

// LastValue returns the last point.
func (o *ocExactAggregator) LastValue() (number.Number, time.Time, error) {
last := o.points[len(o.points)-1]
return last, o.endTime, nil
}

// end returns the timestamp of the last point
func (o *ocExactAggregator) end() time.Time {
return o.endTime
}

var _ aggregation.Aggregation = &ocDistAggregator{}
var _ aggregation.Histogram = &ocDistAggregator{}

// newDistributionAggregator creates an OpenTelemetry aggreation from
// OpenCensus points. Points may not be empty, and must only contain
// Distributions. The most recent disribution will be used in the aggregation.
func newDistributionAggregator(pts []metricdata.Point) (aggregationWithEndTime, error) {
// only use the most recent datapoint for now.
pt := pts[len(pts)-1]
val, ok := pt.Value.(*metricdata.Distribution)
if !ok {
return nil, fmt.Errorf("%w: %v", errBadPoint, pt.Value)
}
bucketCounts := make([]uint64, len(val.Buckets))
for i, bucket := range val.Buckets {
if bucket.Count < 0 {
return nil, fmt.Errorf("%w: bucket count may not be negative", errBadPoint)
}
bucketCounts[i] = uint64(bucket.Count)
}
if val.Count < 0 {
return nil, fmt.Errorf("%w: count may not be negative", errBadPoint)
}
return &ocDistAggregator{
sum: number.NewFloat64Number(val.Sum),
count: uint64(val.Count),
buckets: aggregation.Buckets{
Boundaries: val.BucketOptions.Bounds,
Counts: bucketCounts,
},
endTime: pts[len(pts)-1].Time,
}, nil
}

type ocDistAggregator struct {
sum number.Number
count uint64
buckets aggregation.Buckets
endTime time.Time
}

// Kind returns the kind of aggregation this is.
func (o *ocDistAggregator) Kind() aggregation.Kind {
return aggregation.HistogramKind
}

// Sum returns the sum of values.
func (o *ocDistAggregator) Sum() (number.Number, error) {
return o.sum, nil
}

// Count returns the number of values.
func (o *ocDistAggregator) Count() (uint64, error) {
return o.count, nil
}

// Histogram returns the count of events in pre-determined buckets.
func (o *ocDistAggregator) Histogram() (aggregation.Buckets, error) {
return o.buckets, nil
}

// end returns the time the histogram was measured.
func (o *ocDistAggregator) end() time.Time {
return o.endTime
}
Loading

0 comments on commit 11314d9

Please sign in to comment.