Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update stats collectors to use tagged metrics #13895

Merged
merged 13 commits into from
Apr 9, 2020
Merged

Conversation

yoomlam
Copy link
Contributor

@yoomlam yoomlam commented Apr 5, 2020

Connect #13769

Description

In order to easily chart related metrics, DataDog recommends using tags when submitting metrics.

Also add validation for metric and tag names.
https://docs.datadoghq.com/tagging/

Acceptance Criteria

  • Code compiles correctly and tests pass

Details

See:

In DataDog, click on a metric in Metrics Summary to see tags for a metric.

@yoomlam yoomlam added Type: Metrics or Reporting Team: Echo 🐬 Priority: Low Reported issue, not a blocker. "Last in" priority for new work. labels Apr 5, 2020
@yoomlam yoomlam self-assigned this Apr 5, 2020
@codeclimate
Copy link

codeclimate bot commented Apr 5, 2020

Code Climate has analyzed commit 6976df6 and detected 0 issues on this pull request.

View more on Code Climate.

return emit(hash[:metric], hash[:value], tags: hash.except(:metric, :value)) if hash[:metric] && hash[:value]

# when hash is { "metric_name" => 123 }
return emit(hash.first[0], hash.first[1]) if hash.size == 1
Copy link
Contributor Author

@yoomlam yoomlam Apr 5, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Handle metric without tag.

Comment on lines +12 to +14
[].tap do |stats|
stats.concat flatten_stats(TOTALS_METRIC_NAME_PREFIX, total_counts_hash)
stats.concat flatten_stats(INCREMENTS_METRIC_NAME_PREFIX, daily_counts)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Return array since metric names are no longer unique when using tags.

METRIC_NAME_PREFIX = "request_issues.unidentified_with_contention"
include Collectors::StatsCollector

METRIC_NAME_PREFIX = "req_issues.w_contentions"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shorten name metric name prefix for readability in DataDog.

Comment on lines +153 to +161
class ExampleTaggingCollector
def collect_stats
[
{ "ex_tagged_metric" => 9 },
{ metric: "ex_tagged_metric.status", value: 3, "status": "open" },
{ metric: "ex_tagged_metric.status", value: 5, "status": "closed" }
]
end
end
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test with collector that uses the status tag.

protected

def add_tags_to_group_counts(prefix, metric_name, group_counts)
tag_key = to_valid_tag_key(metric_name.split(".").last)
Copy link
Contributor Author

@yoomlam yoomlam Apr 5, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use that last part of the metric_name as the tag_key.
Example: If metric_name="hearing.disposition", then tag_key="disposition"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest passing in metric_name and doing the split inside of the to_valid_tag_key, that way you have all of the context of creating a valid tag key encapsulated in the function, instead of split up

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I considered this. The reason I did not encapsulate the split call into to_valid_tag_key is in case other collectors want to use it. I don't want to be doing too much manipulation to the name input argument. I could be convinced otherwise, but I'm leaning towards keeping it as it.

def group_key_to_name(key)
return "nil" unless key

return key.map(&:underscore).map(&:parameterize).join(".") if key.instance_of?(Array)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Example: when key=["EstablishClaim", "started"], return "establish_claim.started"


return key.underscore.parameterize if key.instance_of?(String)

key.to_s.underscore.parameterize
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For numbers and booleans.


return key.map(&:underscore).map(&:parameterize).join(".") if key.instance_of?(Array)

return key.underscore.parameterize if key.instance_of?(String)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

@lomky lomky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

couple comments; will finish the review in the AM!

app/jobs/stats_collector_job.rb Outdated Show resolved Hide resolved
protected

def add_tags_to_group_counts(prefix, metric_name, group_counts)
tag_key = to_valid_tag_key(metric_name.split(".").last)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest passing in metric_name and doing the split inside of the to_valid_tag_key, that way you have all of the context of creating a valid tag key encapsulated in the function, instead of split up

Copy link
Contributor

@lomky lomky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice refactor! 🎨 A couple optional suggestions. :shipit:

@yoomlam yoomlam added the Ready-to-Merge This PR is ready to be merged and will be picked up by va-bot to automatically merge to master label Apr 9, 2020
@va-bot va-bot merged commit 327745e into master Apr 9, 2020
@va-bot va-bot deleted the yoom/allow-tagged-metrics branch April 9, 2020 21:40
@yoomlam
Copy link
Contributor Author

yoomlam commented Apr 20, 2020

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Priority: Low Reported issue, not a blocker. "Last in" priority for new work. Ready-to-Merge This PR is ready to be merged and will be picked up by va-bot to automatically merge to master Team: Echo 🐬 Type: Metrics or Reporting
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants