Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Batch observer #1137

Merged
merged 30 commits into from
Jun 30, 2020
Merged

Batch observer #1137

merged 30 commits into from
Jun 30, 2020

Conversation

obecny
Copy link
Member

@obecny obecny commented Jun 3, 2020

Which problem is this PR solving?

Short description of the changes

  • It adds a BatchObserver to support observing multiple instruments in one callback.
  • Observer refactoring
  • Created separate files for each of the metric
  • Renamed Observer to ValueObserver
  • Renamed aggregators

@dyladan
Copy link
Member

dyladan commented Jun 3, 2020

Does this also contain the changes we talked about for the non-batch observer?

@codecov
Copy link

codecov bot commented Jun 3, 2020

Codecov Report

Merging #1137 into master will decrease coverage by 0.37%.
The diff coverage is 90.44%.

@@            Coverage Diff             @@
##           master    #1137      +/-   ##
==========================================
- Coverage   93.29%   92.92%   -0.38%     
==========================================
  Files         126      131       +5     
  Lines        3597     3674      +77     
  Branches      724      742      +18     
==========================================
+ Hits         3356     3414      +58     
- Misses        241      260      +19     
Impacted Files Coverage Δ
packages/opentelemetry-api/src/metrics/Metric.ts 100.00% <ø> (ø)
...s/opentelemetry-metrics/src/UpDownCounterMetric.ts 100.00% <ø> (ø)
packages/opentelemetry-metrics/src/export/types.ts 100.00% <ø> (ø)
...ackages/opentelemetry-api/src/metrics/NoopMeter.ts 66.66% <46.15%> (-3.11%) ⬇️
...pentelemetry-exporter-prometheus/src/prometheus.ts 87.87% <70.00%> (-3.52%) ⬇️
...ckages/opentelemetry-metrics/src/export/Batcher.ts 91.30% <87.50%> (-8.70%) ⬇️
packages/opentelemetry-metrics/src/Meter.ts 92.68% <90.90%> (-1.07%) ⬇️
packages/opentelemetry-metrics/src/Metric.ts 97.05% <92.30%> (-2.95%) ⬇️
...s/opentelemetry-metrics/src/BatchObserverMetric.ts 93.10% <93.10%> (ø)
...es/opentelemetry-metrics/src/BaseObserverMetric.ts 100.00% <100.00%> (ø)
... and 17 more

@obecny
Copy link
Member Author

obecny commented Jun 3, 2020

Does this also contain the changes we talked about for the non-batch observer?

Yes - it has both of them

Copy link
Member

@mayurkale22 mayurkale22 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR is very big, easy to miss the things. Will take a look again.

examples/metrics/metrics/observer.js Outdated Show resolved Hide resolved
packages/opentelemetry-metrics/src/Meter.ts Show resolved Hide resolved
Copy link
Member

@dyladan dyladan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn't have a lot of time to look at this today. It's on my list for first thing tomorrow.

examples/metrics/metrics/observer.js Outdated Show resolved Hide resolved
examples/metrics/metrics/observer.js Outdated Show resolved Hide resolved
examples/metrics/metrics/observer.js Show resolved Hide resolved
packages/opentelemetry-api/src/metrics/NoopMeter.ts Outdated Show resolved Hide resolved
@obecny
Copy link
Member Author

obecny commented Jun 4, 2020

This PR is very big, easy to miss the things. Will take a look again.

I have splited the code from Meter into separate files as Meter was growing bigger and bigger (different classes in one file)

obecny added 3 commits June 4, 2020 17:55

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
@obecny obecny requested a review from naseemkullah as a code owner June 4, 2020 16:21
@obecny obecny added the breaking label Jun 4, 2020
@obecny obecny requested a review from dyladan June 12, 2020 22:30
Copy link
Member

@markwolff markwolff left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM, but I'll take another closer look

Comment on lines +38 to +39
resource: Resource,
metricKind: MetricKind,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: reorder these two to match Metric ctor

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

they already match the other metric like CounterMetric, ValueRecorderMetric etc.

description: this._options.description,
unit: this._options.unit,
description: this._options.description || '',
unit: this._options.unit || '1',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like this was default value previously existed here, but what does '1' mean for a unit? From the spec it seems like just a normal unit of measure, e.g. GBs or whatever. Am I misinterpreting the unit descriptor?

Copy link
Member Author

@obecny obecny Jun 22, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think default '1' is meant to show that by default it is a single count of something, but you do not know what. Could be 1 second (timer), could be 1 transaction (counter), could be one byte, kb, mb, or gb. Without having a unit provided it is basically impossible to infer what the value should be, so it becomes "1 of something".

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dyladan mentioned it correctly.

super(name, options, MetricKind.OBSERVER, resource);
this._maxTimeoutUpdateMS =
options.maxTimeoutUpdateMS ?? MAX_TIMEOUT_UPDATE_MS;
this._callback = callback || NOOP_CALLBACK;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment about type guarding as above.

@obecny obecny requested a review from legendecas as a code owner June 22, 2020 20:16
@obecny
Copy link
Member Author

obecny commented Jun 29, 2020

@open-telemetry/javascript-approvers @open-telemetry/javascript-maintainers ^^

Copy link
Member

@dyladan dyladan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Big change, but I'm satisfied all the concerns are met at this point. Looks good and we can always make minor tweaks later if need be.

examples/metrics/metrics/observer.js Outdated Show resolved Hide resolved
packages/opentelemetry-metrics/README.md Outdated Show resolved Hide resolved
packages/opentelemetry-metrics/README.md Outdated Show resolved Hide resolved
examples/metrics/metrics/observer.js Outdated Show resolved Hide resolved
packages/opentelemetry-metrics/README.md Outdated Show resolved Hide resolved
description: this._options.description,
unit: this._options.unit,
description: this._options.description || '',
unit: this._options.unit || '1',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dyladan mentioned it correctly.

@@ -39,13 +31,16 @@ export abstract class Metric<T extends BaseBoundInstrument>

constructor(
private readonly _name: string,
private readonly _options: MetricOptions,
private readonly _options: api.MetricOptions,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC we have intentionally used MetricOptions instead of api.MetricOptions in order to get ride of Non-Null Assertion Operator!. This way no need to worry about missing values in the SDK and no need to assign default values the way you did here: https://github.com/open-telemetry/opentelemetry-js/pull/1137/files#diff-c46db629f283d0e2d7d273379302ec6fR94. Does it make sense?

Related comment: #415 (comment)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm ok but then the api.MetricOptions was used anyway as an interface in our sdk so this is imho duplication. Also our sdk needs to be compatible with api so our sdk needs to use the api.MetricOptions anyway and you have to sync it manually. With regards to missing values maybe I'm missing something but if we expect our sdk to work with our api we should create a default values anyway not matter if we use a separate options interface or not as our sdk should work with anything (any 3rd party) that is api compatible and vice versa. I might revert it but I really think this is duplication and we endup with casting the interface once as api and once as our sdk and still we will have problem I mentioned.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I largely agree with your comment. This kind of duplication is adding more confusion than initially intended. Also, it is hard to keep them in sync for longer runs.

packages/opentelemetry-metrics/src/types.ts Show resolved Hide resolved
@obecny obecny requested a review from mayurkale22 June 30, 2020 13:31
@mayurkale22
Copy link
Member

Sorry for the delay in reviews. It would be great if you can split PRs into smaller ones to get faster approval and merge. Renaming aggregators and refactoring other metrics would have easily made into separate PR. Although you have mentioned everything in the description, the PR title doesn't justify whole work.

@obecny
Copy link
Member Author

obecny commented Jun 30, 2020

Sorry for the delay in reviews. It would be great if you can split PRs into smaller ones to get faster approval and merge. Renaming aggregators and refactoring other metrics would have easily made into separate PR. Although you have mentioned everything in the description, the PR title doesn't justify whole work.

Sry for confusion, my initial thought was that it will be faster in one PR, but next time I would definitely split this into smaller chunks. Basically some of the things were discovered during coding and was already hard to go back - the temptation to fix things was too big if you know what I mean :D

@mayurkale22 mayurkale22 merged commit 9845b4f into open-telemetry:master Jun 30, 2020
@dyladan
Copy link
Member

dyladan commented Jul 1, 2020

🎉

@obecny obecny deleted the batch-observer branch July 8, 2020 12:13
dyladan pushed a commit to dyladan/opentelemetry-js that referenced this pull request Sep 9, 2022
* chore: adding batch observer, some metrics refactoring

* chore: undo changes after testing

* chore: undo changes after testing

* chore: addressing comments

* chore: renaming observer into value observer, fixing few spotted issues

* chore: missing renamed for ValueObserver

* chore: removing unused class

* chore: cleanup

* chore: refactoring, renaming aggregators

* chore: refactoring observer to have base class that can be extended

* chore: changing aggregator for ValueObserver, exposing batcher so it can be used to override a default one

* chore: addressing comments

* chore: addressing comments

* chore: preventing user from updating observer after timeout or update

* chore: aligning aggregators for value observer and recorder with regards to last spec changes

* chore: fixing test

* chore: fixes after merge

* chore: changes after review

* chore: changes after review with some additional fixes around typing

* chore: changes after review

* chore: lint

* chore: reviews

* chore: typo
dyladan pushed a commit to dyladan/opentelemetry-js that referenced this pull request Sep 9, 2022
* chore: adding batch observer, some metrics refactoring

* chore: undo changes after testing

* chore: undo changes after testing

* chore: addressing comments

* chore: renaming observer into value observer, fixing few spotted issues

* chore: missing renamed for ValueObserver

* chore: removing unused class

* chore: cleanup

* chore: refactoring, renaming aggregators

* chore: refactoring observer to have base class that can be extended

* chore: changing aggregator for ValueObserver, exposing batcher so it can be used to override a default one

* chore: addressing comments

* chore: addressing comments

* chore: preventing user from updating observer after timeout or update

* chore: aligning aggregators for value observer and recorder with regards to last spec changes

* chore: fixing test

* chore: fixes after merge

* chore: changes after review

* chore: changes after review with some additional fixes around typing

* chore: changes after review

* chore: lint

* chore: reviews

* chore: typo
pichlermarc pushed a commit to dynatrace-oss-contrib/opentelemetry-js that referenced this pull request Dec 15, 2023
…1135 (open-telemetry#1137)

Co-authored-by: Daniel Dyla <dyladan@users.noreply.github.com>
Co-authored-by: Valentin Marchaud <contact@vmarchaud.fr>
martinkuba pushed a commit to martinkuba/opentelemetry-js that referenced this pull request Mar 13, 2024
…1135 (open-telemetry#1137)

Co-authored-by: Daniel Dyla <dyladan@users.noreply.github.com>
Co-authored-by: Valentin Marchaud <contact@vmarchaud.fr>
martinkuba pushed a commit to martinkuba/opentelemetry-js that referenced this pull request Mar 16, 2024
…1135 (open-telemetry#1137)

Co-authored-by: Daniel Dyla <dyladan@users.noreply.github.com>
Co-authored-by: Valentin Marchaud <contact@vmarchaud.fr>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement BatchObserver Rename Observer to ValueObserver
5 participants