[Actions] Bi-directional action types

[peterschretlen]

All of our built-in action types are "notification" style actions. Send a message, call a webhook, index a document - they are one-time, one-way communications.

Case management ( #50103 ) is an example of an action that is bi-directional. A case in an external system could be opened in Kibana, but Kibana should realize when that case has been closed.

There are likely other action types in #45023 that fall into this category as well.

This enhancement would extend the Actions system to allow for longer-lived action types. Rather than an action completing as soon as it executes successfully, a bi-directional action would remain open and continue communicating with the external service.

For example, a case management action might have a background task that periodically polls for the case status, and the action does not complete until the case is closed. Another example could be an action that initiates a conversation with a chatbot, and does not end until the conversation is complete.

[elasticmachine]

Pinging @elastic/kibana-stack-services (Team:Stack Services)

[pmuellr]

A case in an external system could be opened in Kibana, but Kibana should realize when that case has been closed.

My initial take is that you could do this via:

• creating a new alert type that polls the external system for "case changes"; the alert type would take config parameters used to find the cases to poll for (API keys, server hosts, case query params, etc)
• create a new action type (if required, perhaps the index action is good enough) to be used when a case changes - eg, perhaps updating an application-specific index with the case change
• create actions that will be used with the new alert
• create alerts with the new alert type, adding new actions (and/or reusing existing actions) as appropriate - eg a combination of slack/email and a custom action type to update an application-specific index.

So basically, do the polling in an alert, have actions that do any typical notification you might want as well as run application-specific actions on case changes.

There seems to be one wrinkle with this, in that for such a scenario, you will want to have "secrets" associated with the alert itself, for it's own HTTP calls in it's alertType execution function. But I don't think we support that today IIRC. Only actions have encrypted secrets.

This enhancement would extend the Actions system to allow for longer-lived action types.

This seems like it greatly complicates actions, and it's not clear to me that it's actually needed to provide the function requested.

[pmuellr]

There seems to be one wrinkle with this, in that for such a scenario, you will want to have "secrets" associated with the alert itself, for it's own HTTP calls in it's alertType execution function. But I don't think we support that today IIRC. Only actions have encrypted secrets.

Adding "secrets" to alerts might well be something we'll end up wanting to do anyway, but perhaps there's another answer.

Currently only actions have encrypted secrets available at the API level, so, you could imagine creating an action that does the query on the case management system, which would then be invoked by the alert execution function. So, you'd basically be setting that action id as a config parameter when creating the alert. The alert function would then execute the action and use the result to determine case changes.

It feels a bit cumbersome to do all that, but does reuse the parts of the system that do what they do well, without having to add new functionality - eg, reuse encrypted secrets in actions, doesn't require changing alerts to support encrypted secrets.

[peterschretlen]

This enhancement would extend the Actions system to allow for longer-lived action types.

This seems like it greatly complicates actions, and it's not clear to me that it's actually needed to provide the function requested.

I don't think it's strictly needed, but I do think it would be desirable to package the whole lifecycle ( start, monitor/interact, close ) into an action? While it could be split across an alert and one or more actions, I don't see that as less complicated, the complexity is just moved externally and exposed to the user.

I acknowledge this would add more complexity, but I think long-running actions with state and bi-directionality opens up the possibilities for the types of actions we can allow. Perhaps what's needed is more concrete examples of these potential actions to understand that the value warrants the effort & complexity involved.

[tobio]

The Cloud Vanguard team is looking at replacing our existing mix of alert sources with ES/Kibana Alerting. One aspect of the project is the ability for an alert to automatically trigger some remediation action (currently running on Rundeck).

Should that remediation script fail to address the alert we'd fall back to notifying a human. With the current API we'd have to have two alerts to cover this functionality:

• The first to trigger the remediation job, which then pushes it's result back to ES
• A second alert which fires should the job fail.

Having bi-directional actions would ideally allow us to setup an action pipeline, short circuiting on the first successful action. That way, no humans are notified should the remediation step succeed and the entire pipeline is configured as a single alert.

[henrikno]

A simple example of two-way interaction would be a Slack message with a button that can Ack the alert
https://api.slack.com/docs/message-buttons

[mikecote]

Adding a needs research label and merging with the now closed research issue: #56396. Worth looking at the comments in the other issue.

[mikecote]

Moving from 7.x - Candidates to 8.x - Candidates (Backlog) after the latest 7.x planning session.

[ymao1]

This seems like a functionality that will be handled by cases vs actions. Closing this issue but will support any alerting-related work by the cases team to add this functionality.