Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: metric aggregation temporality controls #2902

Merged

Conversation

seemk
Copy link
Contributor

@seemk seemk commented Apr 18, 2022

Which problem is this PR solving?

Fixes #2864

Short description of the changes

  • Get rid of getPreferredAggregationTemporality at MetricReader and exporters. They now have a selector, taking in an instrument type, that specifies which temporality to use. The replacement is now getAggregationTemporality(instrumentType: InstrumentType) which invokes the given selector.
  • OTLP exporter can be configured with a preferred temporality, prometheus only uses cumulative.
  • Moved aggregationTemporality from exporters to metric data type itself. This simplifies code - now the aggregation is selected in the metrics processor and passed to the creation of metric data, otherwise conversion code in exporters would have to invoke the same selector again.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

How Has This Been Tested?

  • Updated unit tests

Checklist:

  • Followed the style guidelines of this project
  • Unit tests have been added
  • Documentation has been updated

@seemk seemk requested a review from a team April 18, 2022 10:59
@seemk seemk changed the title feat: aggregation temporality controls feat: metric aggregation temporality controls Apr 18, 2022
@codecov
Copy link

codecov bot commented Apr 18, 2022

Codecov Report

Merging #2902 (a4b1e72) into main (858f6ce) will decrease coverage by 0.22%.
The diff coverage is 59.09%.

@@            Coverage Diff             @@
##             main    #2902      +/-   ##
==========================================
- Coverage   93.01%   92.78%   -0.23%     
==========================================
  Files         183      183              
  Lines        5909     5921      +12     
  Branches     1254     1257       +3     
==========================================
- Hits         5496     5494       -2     
- Misses        413      427      +14     
Impacted Files Coverage Δ
...ntelemetry-sdk-metrics-base/src/aggregator/Drop.ts 100.00% <ø> (ø)
...metry-sdk-metrics-base/src/aggregator/Histogram.ts 100.00% <ø> (ø)
...metry-sdk-metrics-base/src/aggregator/LastValue.ts 100.00% <ø> (ø)
...entelemetry-sdk-metrics-base/src/aggregator/Sum.ts 100.00% <ø> (ø)
...telemetry-sdk-metrics-base/src/aggregator/types.ts 100.00% <ø> (ø)
...-metrics-base/src/export/AggregationTemporality.ts 100.00% <ø> (ø)
...elemetry-sdk-metrics-base/src/export/MetricData.ts 100.00% <ø> (ø)
...etry-sdk-metrics-base/src/export/MetricExporter.ts 23.07% <0.00%> (-4.20%) ⬇️
...emetry-sdk-metrics-base/src/export/MetricReader.ts 100.00% <ø> (ø)
...s-base/src/export/PeriodicExportingMetricReader.ts 91.11% <33.33%> (-4.24%) ⬇️
... and 11 more

Copy link
Member

@legendecas legendecas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM % nits.

@@ -69,7 +69,7 @@ export class TemporalMetricProcessor<T> {
sdkStartTime: HrTime,
collectionTime: HrTime,
): Maybe<MetricData> {
const aggregationTemporality = collector.aggregatorTemporality;
const aggregationTemporality = collector.selectAggregationTemporality(instrumentDescriptor.type);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As the selectAggregationTemporality can call into user codes, I think we should memo the selection result at the creation of the instrument so that it can not be accidentally changed: like returning CUMULATIVE for first call and DELTA for second call.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean basically attaching it to instrument descriptor at the time of creation?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, or attaching it to the internal object MetricStorage, since the spec didn't ask to expose this info.

Copy link
Member

@legendecas legendecas Apr 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Eh, sorry for the turn back. Seems this is not settled yet?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was just wondering that the temporality is currently retrieved via the collector, then at instrument creation we would create a mapping between each collector and temporality (based on instrument type, by calling collector's selector), store this map at the MetricStorage - we'd need to handle the case where a metric reader / collector is added after instrument creation, then go over existing storages and update the collector to temporality mapping?

If this logic seems fine, I can add it, although I'd prefer to do it without all the extra state

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, it can be simpler if we add a new parameter aggregationTemporality to TemporalMetricProcessor.buildMetrics and save the aggregationTemporality to MetricStorage. Or we can construct TemporalMetricProcessor with an aggregationTemporality since we don't want it to be changed anyway.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

although I'd prefer to do it without all the extra state

In this sense, how should we handle the aggregationTemporality changes if the user-provided selector changes the returned value from time to time?

Copy link
Contributor Author

@seemk seemk Apr 29, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, it can be simpler if we add a new parameter aggregationTemporality to TemporalMetricProcessor.buildMetrics and save the aggregationTemporality to MetricStorage. Or we can construct TemporalMetricProcessor with an aggregationTemporality since we don't want it to be changed anyway.

Yup, but in order to save aggregationTemporality to MetricStorage we need to invoke the reader's/collector's selector, right? But if I read it correctly there can be many readers for a single storage, so we'd need a mapping. Just a bit confused here

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right. This is making the problem complicated. I'm ok with this as is.

@legendecas legendecas self-requested a review April 28, 2022 17:22
@legendecas
Copy link
Member

legendecas commented May 9, 2022

@seemk I think this PR is ready to go. would you mind updating the branch on top of the latest main branch and resolving the conflicts? thank you!

@dyladan
Copy link
Member

dyladan commented May 9, 2022

Sorry it took so long to review. I thought I had reviewed this already so I kept skipping it in the list 😨

@legendecas legendecas merged commit 1ee1b28 into open-telemetry:main May 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

AggregationTemporality Controls
4 participants