sampling: add trace group sampling reservoirs #4182
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Introduce a type for maintaining a collection of trace groups. For each trace group we measure the observed ingest rate for the trace group (number of root traces per sampling interval), and maintain a weighted random sampling reservoir for recording trace IDs. Trace IDs are weighted by root transaction duration, which means slower traces will have a greater chance of being captured.
💚 Build Succeeded
Expand to view the summary
Build stats
Test stats 🧪
Steps errorsExpand to view the steps failures
|
axw
commented
Sep 9, 2020
axw
commented
Sep 9, 2020
- better name for decay factor - fix doc comments to reflect reality - fix criteria for removing trace groups
axw
commented
Sep 17, 2020
simitt
approved these changes
Sep 17, 2020
axw
added a commit
to axw/apm-server
that referenced
this pull request
Sep 18, 2020
* sampling: add trace group sampling reservoirs Introduce a type for maintaining a collection of trace groups. For each trace group we measure the observed ingest rate for the trace group (number of root traces per sampling interval), and maintain a weighted random sampling reservoir for recording trace IDs. Trace IDs are weighted by root transaction duration, which means slower traces will have a greater chance of being captured. * Address self-review comments - better name for decay factor - fix doc comments to reflect reality - fix criteria for removing trace groups * Update x-pack/apm-server/sampling/groups.go * Remove unnecessary variable assignment
axw
added a commit
that referenced
this pull request
Sep 18, 2020
* sampling: add trace group sampling reservoirs Introduce a type for maintaining a collection of trace groups. For each trace group we measure the observed ingest rate for the trace group (number of root traces per sampling interval), and maintain a weighted random sampling reservoir for recording trace IDs. Trace IDs are weighted by root transaction duration, which means slower traces will have a greater chance of being captured. * Address self-review comments - better name for decay factor - fix doc comments to reflect reality - fix criteria for removing trace groups * Update x-pack/apm-server/sampling/groups.go * Remove unnecessary variable assignment
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation/summary
Introduce a type for maintaining a collection of trace groups.
For each trace group we measure the observed ingest rate for
the trace group (number of root traces per sampling interval),
and maintain a weighted random sampling reservoir for recording
trace IDs. Trace IDs are weighted by root transaction duration,
which means slower traces will have a greater chance of being
captured.
Metrics/monitoring for this will come later.
Checklist
- [ ] I have updated CHANGELOG.asciidocI have considered changes for:
- [ ] documentation- [ ] logging (add log lines, choose appropriate log selector, etc.)- [ ] metrics and monitoring (create issue for Kibana team to add metrics to visualizations, e.g. Kibana#44001)- [ ] telemetry- [ ] Elasticsearch Service (https://cloud.elastic.co)- [ ] Elastic Cloud Enterprise (https://www.elastic.co/products/ece)- [ ] Elastic Cloud on Kubernetes (https://www.elastic.co/elastic-cloud-kubernetes)How to test these changes
make testRelated issues
None.