Report RequestIssue stats to DataDog #13770
Conversation
yoomlam
added
Priority: Low
yoomlam
changed the title
|
Code Climate has analyzed commit 8285642 and detected 0 issues on this pull request. View more on Code Climate. |
|
For instructions see https://github.com/department-of-veterans-affairs/appeals-team/pull/1266/files When adding a DataDog widget, select the metric in step 2: |
| stats[METRIC_NAME_PREFIX] = req_issues.count | ||
|
|
||
| req_issues.group(:closed_status).count.each do |ben_type, count| | ||
| stats["#{METRIC_NAME_PREFIX}.st.#{ben_type || 'nil'}"] = count | ||
| end | ||
|
|
||
| req_issues.group(:benefit_type).count.each do |ben_type, count| | ||
| stats["#{METRIC_NAME_PREFIX}.ben.#{ben_type}"] = count | ||
| end | ||
|
|
||
| dr_counts_by_type = req_issues.group(:decision_review_type).count | ||
| dr_counts_by_type.each do |dr_type, count| | ||
| stats["#{METRIC_NAME_PREFIX}.dr.#{dr_type}"] = count | ||
| end | ||
|
|
||
| # Could use `req_issues.group(:veteran_participant_id).count.count` but there's a count discrepancy | ||
| stats["#{METRIC_NAME_PREFIX}.vet_count"] = req_issues.map(&:decision_review).map(&:veteran_file_number).uniq.count |
There was a problem hiding this comment.
Populate stats hash with desire metrics.
|
|
||
| # :reek:FeatureEnvy | ||
| def collect_stats | ||
| start_of_month = Time.zone.now.prev_month.beginning_of_month |
There was a problem hiding this comment.
This collector reports on statistics about last month.
| RequestIssue.where.not(contention_reference_id: nil) | ||
| .where(is_unidentified: true) | ||
| .where("created_at >= ?", start_date) | ||
| .where("created_at < ?", end_date) |
There was a problem hiding this comment.
All Rails queries have been tested in the production environment.
| def emit(name, value, attrs: {}) | ||
| DataDogService.emit_gauge( | ||
| metric_group: METRIC_GROUP_NAME, | ||
| metric_name: name, | ||
| metric_value: value, | ||
| app_name: APP_NAME, | ||
| attrs: attrs | ||
| ) | ||
| end |
There was a problem hiding this comment.
Note the attributes passed to DataDog:
app_name: "caseflow_job"- mapped to aapp:caseflow_jobtag in DataDogmetric_group: YOUR_JOB_NAME_jobmetric_name: YOUR_METRIC_NAME- can have multiplemetric_names per jobmetric_value: YOUR_METRIC_VALUE- value to log in DataDogattrs: { YOUR_ATTR_KEY: YOUR_ATTR_VALUE }- each hash entry is mapped to aYOUR_ATTR_KEY:YOUR_ATTR_VALUEtag in DataDog for selecting subsets of your metric or sets of different metrics
The metric label in DataDog will be dsva_appeals.YOUR_JOB_NAME_job.YOUR_METRIC_NAME as defined in DataDogService.get_stat_name
app/jobs/stats_collector_job.rb
Outdated
| DAILY_COLLECTORS = { | ||
| "unidentified_request_issues_with_contention" => Collectors::RequestIssuesStatsCollector | ||
| }.freeze | ||
| WEEKLY_COLLECTORS = { | ||
| }.freeze | ||
| MONTHLY_COLLECTORS = { | ||
| }.freeze |
There was a problem hiding this comment.
Add new collectors here.
|
|
||
| metric_name_prefix = described_class::METRIC_NAME_PREFIX | ||
| expect(stats[metric_name_prefix]).to eq(9) | ||
| expect(stats["#{metric_name_prefix}.vet_count"]).to be_between(1, 9) |
There was a problem hiding this comment.
There should be at least 1 unique veteran but no more than 9 (total number relevant request issues).
There was a problem hiding this comment.
I generally like this approach of a single job where we can register various collectors. It's similar to data integrity checker in that respect.
I would move the collector classes outside the app/jobs space since they are not jobs, but POROs.
yoomlam
added
Ready-to-Merge
yoomlam
added
the
Ready-to-Merge
Resolves #13769
Description
Provides initial capability to run a new job that reports RequestIssues stats to DataDog. This PR will be an example for future metrics, which can be added to this job (if it's related) or implemented as a new job.
A StatsCollectorJob has been implemented to simplify sending metrics to DataDog and to minimize the cost of running jobs as AWS Lambdas. StatsCollectorJob holds sets of Collectors that run daily, weekly, and monthly.
The StatsCollectorJob is scheduled to run in a Lambda in https://github.com/department-of-veterans-affairs/appeals-lambdas/pull/51.
Acceptance Criteria
Collectors.Testing Plan (after merge)
RequestIssuesStatsCollectorto run monthly.