[Synthetics] Synthetics UI Alerting - Initial

[paulb-elastic]

The very first implementation for Alerting in Synthetics UI will be based on what can be done today in Uptime, but from the Synthetics UI, with a few exceptions.

ACs:

• There will be new alert rule type in the app, though it will contain no configuration as such
• Instance if rule type will be created when we create a first monitor
• Each monitor will have enable/disable flag as part of config if alerting is enabled
• We will use the default connector types from settings or inform user on action if they haven't defined yet
• Just like disable monitor there will be a new action, enable alerting in overview card list.
• Alert criteria will be simple, if monitor goes down we fire an alert on default connector
• Recovery message will be sent only when we find up check after monitor has been down
• Deleting the monitor will resolve the alert and sends a message
• Disable a monitor will keept active alert open

[elasticmachine]

Pinging @elastic/uptime (Team:uptime)

[paulb-elastic]

Linking for reference:

• [Synthetics 1.0] Alerting Implementation #144639
• https://github.com/elastic/synthetics-dev/issues/145

[shahzad31]

i am going to start work on basic alerting for 1.0.

[dominiqueclarke]

Post FF testing on 8.7.0-SNAPSHOT, only tested with UI monitors so far. Will test tomorrow with project monitors.

cc: @drewpost @shahzad31 @paulb-elastic need feedback.

• There will be new alert rule type in the app, though it will contain no configuration as such
  The monitor rule does have the ability to configure the "check every" value. Is this desired?

Also raised #151806

• Instance if rule type will be created when we create a first monitor
  ✅
• Each monitor will have enable/disable flag as part of config if alerting is enabled
  ✅ The configuration is present
  ❌ disabling alerting in configuration doesn't actually stop alerts from firing [Synthetics] Alerting - alert still fires when alert enabled check is off #151809
• We will use the default connector types from settings or inform user on action if they haven't defined yet
  ✅ Using the default connectors, though there are some raised bugs: [Synthetics] Alert connectors - some alert connector context values seem to be mismatched #151796, [Synthetics] Alerting - recovery message does not populate in alert connectors #151795, [Connectors] Default cloud smtp email connector doesn't seem to work #151791
  ❌ There is no information provided when default connectors at not yet configured, as far as I can tell
• Just like disable monitor there will be a new action, enable alerting in overview card list.
  ✅
• Alert criteria will be simple, if monitor goes down we fire an alert on default connector
  ✅
• Recovery message will be sent only when we find up check after monitor has been down
  ✅ Recovery message is indeed sent
  ❌ Recovery message content is missing [Synthetics] Alerting - recovery message does not populate in alert connectors #151795
• Deleting the monitor will resolve the alert and sends a message
  ✅
• Disable a monitor will keep active alert open
  ✅

[drewpost]

@dominiqueclarke the check every value is NOT desired. We'd prefer it fired automatically when a test failed but there's a framework limitation so we're stuck with this.

I do, however, think it should not be displayed given that we are defaulting to that for a reason and exposing it lets folks get in trouble and potentially delay alerts without realising what they're doing by changing that value.

[shahzad31]

@drewpost
❌ There is no information provided when default connectors at not yet configured, as far as I can tell
do we want to warn/inform users? if they haven't configured any connectors?

[drewpost]

Yea, we should warn them. Perhaps with a link to the setting page for them to specify that value? Some sort of toast or blue banner?

[shahzad31]

Where do we warn, on homepage? or in monitor details page?

[drewpost]

Prefer the overview page, frankly and, if we can on monitor edit/config page as well

[paulb-elastic]

I've created the following to capture these requirements:

• [Synthetics] Remove check every option from Synthetics alert rules #152430
• [Synthetics] Warn if there is no default connector configured #152431

[paulb-elastic]

@dominiqueclarke with these raised, is there any additional Post FF Testing to do, or can it be moved to Done Done?

[dominiqueclarke]

This can be moved to Done Done.