Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PoC] alternate approach for InMemoryExporter, adding selective copy to Metric (for discussion) #3187

Closed
wants to merge 2 commits into from

Conversation

TimothyMothra
Copy link
Contributor

@TimothyMothra TimothyMothra commented Apr 15, 2022

Alternate approach for #2361

Changes

  • Adds selective copy to Metric and AggregatorStore.
  • Updates InMemoryExporter to copy Metric.

Discussion

This attempts to meet the initial ask of #2361 by providing a selective deep-copy of Metric.
"Selective" meaning the minimum fields/properties that are required for current unit tests.
A SME might recommend additional minimum members.
With this draft, all unit tests pass locally.

Ideally, we should deep-copy ALL members.
I would not recommend this approach without also investing in thorough unit tests to fully validate copying.
I don't want to give the false impression this this is a true deep-copy when members could be missed!
This will likely require some extensive reflection-based comparisons and I want to test the waters before investing in this.

I've already evaluated one library (FluentAssertions) to do this comparison, but this has some limitations.
FluentAssertions won't review private members and it depends on the Equals() method for structs.
I'm open to suggestions for other libraries!

Open Questions

  • Is this selective-copy approach appropriate?
    • This creates a maintenance tax that future developers are responsible for updating Copy() methods if needed.
    • Public customers may find scenarios not covered by our tests and be blocked in their own use-cases.
    • Could we mark certain members public to better communicate what is/isn't copied?
  • Regarding full deep-copy, are there any ideas for better validating?

For significant contributions please make sure you have completed the following items:

  • Appropriate CHANGELOG.md updated for non-trivial changes
  • Design discussion issue #
  • Changes in public API reviewed

@TimothyMothra
Copy link
Contributor Author

I got latest on main and now some OTPL tests broke. This highlights the flaw with the selective approach.

{
this.MetricType = other.MetricType;
this.InstrumentIdentity = other.InstrumentIdentity;
this.aggStore = other.aggStore.Copy();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One concern with this approach. AggregatorStore holds onto a pretty sizeable buffer (sizeof(MetricPoint) * 2000 essentially) so user might OOM if their inmemoryexporter is running long enough?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good point!
It also looks like there's a circular reference with AggregatorStore appearing on both Metric and MetricPoint.

@codecov
Copy link

codecov bot commented Apr 15, 2022

Codecov Report

Merging #3187 (a5a6c47) into main (73cc937) will decrease coverage by 0.21%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #3187      +/-   ##
==========================================
- Coverage   85.17%   84.95%   -0.22%     
==========================================
  Files         259      259              
  Lines        9330     9347      +17     
==========================================
- Hits         7947     7941       -6     
- Misses       1383     1406      +23     
Impacted Files Coverage Δ
...penTelemetry.Exporter.InMemory/InMemoryExporter.cs 100.00% <100.00%> (ø)
src/OpenTelemetry/Metrics/AggregatorStore.cs 83.42% <100.00%> (+0.66%) ⬆️
src/OpenTelemetry/Metrics/HistogramBuckets.cs 100.00% <100.00%> (ø)
src/OpenTelemetry/Metrics/Metric.cs 96.05% <100.00%> (+0.40%) ⬆️
...entation/ExportClient/OtlpGrpcTraceExportClient.cs 35.71% <0.00%> (-42.86%) ⬇️
...xporter.OpenTelemetryProtocol/OtlpTraceExporter.cs 36.36% <0.00%> (-40.91%) ⬇️
...tation/OpenTelemetryProtocolExporterEventSource.cs 75.00% <0.00%> (-10.00%) ⬇️
...metryProtocol/Implementation/ActivityExtensions.cs 88.73% <0.00%> (-2.71%) ⬇️

@TimothyMothra TimothyMothra deleted the 2361_alt_cloning branch December 8, 2022 22:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants