Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support cumulative, delta, and pass-through exporters #799

Closed
wants to merge 42 commits into from

Conversation

jmacd
Copy link
Contributor

@jmacd jmacd commented Jun 8, 2020

I'm marking this a draft because it's a very large change. Several substantial portions of this change can be applied as separate PRs because they are independent. Otherwise, this PR is finished and unfortunately still a large change. More, inter-dependent parts of this PR may be broken apart afterward as long as the approach is agreed to.

The most significant changes are in sdk/export/metric, where:

  • aggregation subpackage is renamed from aggregator, new aggregation.Kind is exposed.
  • new aggregation.Aggregation interface is created, will be used by exporters; one Aggregator contains two Aggregation values, the "current" and "checkpoint" state.
  • Aggregator interface has been widened with new support, including access to current and checkpoint Aggregation values, a Swap() primitive, and an optional Subtract() primitive. The semantics of Merge() have changed to update the current state, not the checkpoint.
  • Record is now a type used between the Integrator and Exporter. Record includes the Aggregation appropriate for the exporter and the start and end time of the interval it represents (plus the descriptor, labels, and resource).
  • Accumulation is a new type like the former Record: it's used between the Accumulator and Integrator, refers to an Aggregator and does not contain timestamps.
  • Exporter knows its own kind.
  • Integrator knows the set of (multiple) exporter kinds (in case of multiple exporters)
  • CheckpointSet.ForEach takes the exporter kind as an argument.

I'm concerned about the complexity of the Aggregator API and am option to alternate suggestions. The change in Merge semantics--that it adds into the current state--combined with Swap(), makes it possible to combine values in place w/o extra memory allocations. In the ordinary case, the Accumulator owns one Aggregator per instrument and label set, and the Integrator owns a copy only when it needs to. The Accumulator makes use of the Aggregator Update/Checkpoint synchronization guarantee, while the Integrator doesn't require any locking and uses only the Merge/Swap/Subtract/CheckpointedValue/AccumulatedValue APIs.

With these fundamental changes, the sdk/metric/integrator/simple package is upgraded to handle cumulative instruments and cumulative exporters. It will use state when the instrument and exporter require it, either cumulative instrument with a delta exporter or a delta instrument with a cumulative exporter. All of the substantial changes in this PR happen in this package.

Related TODO: SumObserver monotonicity is not being tested, but it could be done in the Integrator or the Accumulator.

Comment: The "simple" Integrator is no longer simple. Perhaps it should be renamed "basic". I expect that a configurable Integrator can be based off of this code, adding configuration support without having to re-implement the logic here.

Each of the current aggregators had to undergo changes to complete the change, changes which are large but not really functional changes. Only the "Sum" aggregator has been upgraded with support for Subtract(), as needed to expose deltas of cumulative instruments.

Misc cleanups in this PR: Aggregator.Checkpoint() and Integrator.Process() no longer accept Context arguments, they weren't being used and create substantial confusion.

@jmacd jmacd added the area:metrics Part of OpenTelemetry Metrics label Jun 8, 2020
@jmacd
Copy link
Contributor Author

jmacd commented Jun 10, 2020

(I've figured out a better approach for the Aggregator API.)

@MrAlias MrAlias linked an issue Jun 10, 2020 that may be closed by this pull request
@jmacd jmacd closed this Jun 17, 2020
@jmacd jmacd deleted the jmacd/mexport branch June 17, 2020 19:47
@jmacd jmacd restored the jmacd/mexport branch June 17, 2020 19:47
@jmacd jmacd deleted the jmacd/mexport branch June 23, 2020 18:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:metrics Part of OpenTelemetry Metrics
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant