Application Alerting for BIP

[meganhicks]

As a member of the VRO platform team responsible for supporting partner teams, I need to be alerted about any connectivity issues with BIP. This will allow me to proactively notify partner teams about potential interruptions in their services

Acceptance Criteria:

• When the monitor for BIP detects a disconnect or significant delay in service there is an alert that is triggered notifying the team.
• Share why the thresholds were chosen to the team in slack

What you need to know

Implementation / Developer Steps (to be filled out by ticket assignee)

linked ticket-https://app.zenhub.com/workspaces/vro-team-6557e67173391c000e1409f3/issues/gh/department-of-veterans-affairs/abd-vro/2932

[dfitchett]

Current Monitors can be viewed on this dashboard. They need to be re-evaluated for accuracy.

[dfitchett]

Datadog Dashboard

For this ticket, I did the following:

• Updated Monitor: VRO : BIP Elevated Average Latency
  • Lowered threshold for alert to 1500ms, removed threshold. High latency indicates issues with downstream BIP claims API.
• Updated Monitor: VRO : BIP non-2xx responses
  • Lowered threshold to 1 non-2xx response since any error should be looked into
• Added monitor overlay for VRO : BIP Elevated Average Latency to request duration chart in dashboard