Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

are all otel metrics being written correctly flushed when the state machine restarts #3140

Closed
garypen opened this issue May 25, 2023 · 0 comments · Fixed by #3143
Closed

are all otel metrics being written correctly flushed when the state machine restarts #3140

garypen opened this issue May 25, 2023 · 0 comments · Fixed by #3143
Assignees
@garypen

Comments

@garypen
Copy link
Contributor

garypen commented May 25, 2023

We have noticed that we don't see any apollo_router_uplink_fetch_duration_seconds metrics with attribute kind:new when we are exporting metrics using OTLP. This doesn't seem to be a problem when metrics are exported using prometheus, so it's possible that OTLP metrics are not flushing correctly when the state machine restarts.
garypen added a commit that referenced this issue May 25, 2023
@garypen
Try to stop OTLP controllers when Telemetry is dropped
db0ba71 
We already have code to specifically drop tracers and we are adding some
additional logic to do the same thing with metrics exporters.

fixes: #3140
@garypen garypen mentioned this issue May 25, 2023
Merged
6 tasks
garypen added a commit that referenced this issue May 30, 2023
@garypen
Try to stop OTLP controllers when Telemetry is dropped
72429b5 
We already have code to specifically drop tracers and we are adding some
additional logic to do the same thing with metrics exporters.

fixes: #3140
garypen added a commit that referenced this issue Aug 22, 2023
@garypen
Try to stop OTLP controllers when Telemetry is dropped (#3143)
5c8b82c 
We already have code to specifically drop tracers and we are adding some
additional logic to do the same thing with metrics exporters.

fixes: #3140 

<!-- start metadata -->

**Checklist**

Complete the checklist (and note appropriate exceptions) before a final
PR is raised.

- [x] Changes are compatible[^1]
- [ ] Documentation[^2] completed
- [ ] Performance impact assessed and acceptable
- Tests added and passing[^3]
    - [ ] Unit Tests
    - [ ] Integration Tests
    - [ ] Manual Tests

**Exceptions**

*Note any exceptions here*

**Notes**

[^1]. It may be appropriate to bring upcoming changes to the attention
of other (impacted) groups. Please endeavour to do this before seeking
PR approval. The mechanism for doing this will vary considerably, so use
your judgement as to how and when to do this.
[^2]. Configuration is an important part of many changes. Where
applicable please try to document configuration examples.
[^3]. Tick whichever testing boxes are applicable. If you are adding
Manual Tests:
- please document the manual testing (extensively) in the Exceptions.
- please raise a separate issue to automate the test and label it (or
ask for it to be labeled) as `manual test`
@BrynCooke BrynCooke mentioned this issue Aug 24, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@garypen garypen

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Try to stop OTLP controllers when Telemetry is dropped apollographql/router
1 participant
@garypen