[Actions] Add support for system actions in Rules APIs #161726

cnasikas · 2023-07-12T07:43:27Z

Summary

In this PR we allow the usage of system actions through the Rules APIs. Major decisions:

The actions will have the type property only inside the methods of the rule client.
The APIs will not accept the type of the action. The framework will detect this automatically.
All routes will add the type to the actions before passing the request to the rule client.
The rule client methods will return actions with the type.
All routes will remove the type from the actions before returning the response to the client.
Users should not be able to set the frequency, the group, or the alerts filter for system actions.
The group is now optional because system actions do not support it.
The type of the action will not be persisted to ES.

To keep the size of the PR small, and because it will be merged into a feature branch, I decided to work in the Create Rule, Update Rule, and Bulk Edit Rules APIs. I also follow TS errors and failed tests and update some of the other routes/client methods. Another PR will follow to handle the rest of the APIs and the CI errors produced by these APIs and another PR will follow to handle the APIs of Security Solution.

Issue: #160367
Depends on: #165671

Checklist

Delete any items that are not applicable to this PR.

Unit or functional tests were updated or added to match the most common scenarios

For maintainers

This was checked for breaking API changes and was labeled appropriately

…ana into system_actions_registration

…ntext

cnasikas · 2023-09-18T08:18:07Z

x-pack/plugins/alerting/common/routes/rule/apis/bulk_edit/schemas/v1.ts

@@ -38,7 +38,7 @@ const ruleSnoozeScheduleSchemaWithValidation = schema.object(
 );

 const ruleActionSchema = schema.object({
-  group: schema.string(),
+  group: schema.maybe(schema.string()),


System actions do not have a group. Changing this to optional to allow system actions to be created without a group.

cnasikas · 2023-09-18T08:18:35Z

x-pack/plugins/alerting/common/rule.ts

+  type: typeof RuleActionTypes.SYSTEM;
+}
+
+export type RuleAction = RuleDefaultAction | RuleSystemAction;


This type is used internally by the rules client.

cnasikas · 2023-09-18T08:22:38Z

x-pack/plugins/alerting/public/alert_navigation_registry/types.ts

@@ -16,4 +16,4 @@ import { SanitizedRule } from '../../common';
 * originally registered to {@link PluginSetupContract.registerNavigation}.
 *
 */
-export type AlertNavigationHandler = (rule: SanitizedRule) => string;
+export type AlertNavigationHandler = (rule: SanitizedRuleResponse) => string;


The responses from the alerting APIs do not contain the type of action.

cnasikas · 2023-09-18T08:27:26Z

x-pack/plugins/alerting/server/application/rule/methods/bulk_edit/bulk_edit_rules.ts

@@ -115,8 +118,15 @@ export async function bulkEditRules<Params extends RuleParams>(
  context: RulesClientContext,
  options: BulkEditOptions<Params>
 ): Promise<BulkEditResult<Params>> {
+  try {
+    bulkEditOperationsSchema.validate(options.operations);


Validation of the schema was missing from the method. The schema requires at least one operation to be set.

cnasikas · 2023-09-18T08:29:33Z

x-pack/plugins/alerting/server/application/rule/methods/bulk_edit/bulk_edit_rules.ts

@@ -458,20 +472,15 @@ async function updateRuleAttributesAndParamsInMemory<Params extends RuleParams>(
      rule.references = migratedActions.resultedReferences;
    }

-    const ruleActions = injectReferencesIntoActions(


transformRuleAttributesToRuleDomain calls transformRawActionsToDomainActions which in turn injects the references into the actions. No need to do it twice.

cnasikas · 2023-09-18T08:34:01Z

...alerting/server/application/rule/methods/bulk_edit/schemas/bulk_edit_rules_option_schemas.ts

@@ -49,7 +59,7 @@ const bulkEditTagSchema = schema.object({
 const bulkEditActionsSchema = schema.object({
  operation: schema.oneOf([schema.literal('add'), schema.literal('set')]),
  field: schema.literal('actions'),
-  value: schema.arrayOf(bulkEditActionSchema),
+  value: schema.arrayOf(schema.oneOf([bulkEditDefaultActionSchema, bulkEditSystemActionSchema])),


Rule methods accept two types of actions.

cnasikas · 2023-09-18T08:34:39Z

...alerting/server/application/rule/methods/bulk_edit/schemas/bulk_edit_rules_option_schemas.ts

+  type: schema.literal(RuleActionTypes.DEFAULT),
+});
+
+export const bulkEditSystemActionSchema = schema.object({


System actions are not allowed to contain: group, alertsFilter, and frequency.

cnasikas · 2023-09-18T08:36:31Z

...ugins/alerting/server/application/rule/transforms/transform_raw_actions_to_domain_actions.ts

+  references?: SavedObjectReference[];
+}
+
+export const transformRawActionsToDomainActions = ({


This function adds the id from the references and the type in all actions. The type is not persisted in ES.

cnasikas · 2023-09-18T08:43:03Z

...gins/alerting/server/application/rule/transforms/transform_rule_attributes_to_rule_domain.ts

@@ -149,13 +149,13 @@ export const transformRuleAttributesToRuleDomain = <Params extends RuleParams =
      })?.toISOString()
    : null;

-  let actions = esRule.actions
-    ? injectReferencesIntoActions(id, esRule.actions as RawRule['actions'], references || [])


Logic moved inside transformRawActionsToDomainActions.

cnasikas · 2023-09-18T08:44:13Z

x-pack/plugins/alerting/server/connector_adapters/validate_rule_action_params.ts

+  }
+
+  const connectorAdapter = connectorAdapterRegistry.get(connectorTypeId);
+  const schema = connectorAdapter.ruleActionParamsSchema;


This is the schema of the system action params persisted in the rule.

cnasikas · 2023-09-18T08:46:13Z

x-pack/plugins/alerting/server/routes/legacy/create.ts

-            data: { ...alert, notifyWhen },
+            data: {
+              ...alert,
+              actions: rewriteActionsReq(alert.actions, (connectorId: string) =>


Adds the type to the actions.

cnasikas · 2023-09-18T08:46:58Z

x-pack/plugins/alerting/server/routes/legacy/create.ts

          return res.ok({
-            body: alertRes,
+            body: { ...alertRes, actions: rewriteActionsResLegacy(alertRes.actions) },


Removes the type from the actions.

cnasikas · 2023-09-18T09:17:43Z

x-pack/plugins/alerting/server/routes/lib/rewrite_actions.ts

+    return [];
+  }
+
+  return actions.map((action) => {


Adds the type to the actions.

cnasikas · 2023-09-18T09:18:08Z

x-pack/plugins/alerting/server/routes/lib/rewrite_actions.ts

-      : {}),
-  }));
+
+  return actions.map((action) => {


Removes the type from the actions.

kibana-ci · 2023-09-19T08:34:20Z

💔 Build Failed

Failed CI Steps

Test Failures

[job] [logs] x-pack/test/alerting_api_integration/security_and_spaces/group2/config_non_dedicated_task_runner.ts / alerting api integration security and spaces enabled - Group 2 Alerts alerts alerts superuser at space1 should schedule task, run alert and schedule actions when appropriate
[job] [logs] x-pack/test/alerting_api_integration/security_and_spaces/group2/config.ts / alerting api integration security and spaces enabled - Group 2 Alerts alerts alerts superuser at space1 should schedule task, run alert and schedule actions when appropriate
[job] [logs] x-pack/test/alerting_api_integration/security_and_spaces/group2/config_non_dedicated_task_runner.ts / alerting api integration security and spaces enabled - Group 2 Alerts alerts alerts superuser at space1 should schedule task, run alert and schedule actions when appropriate
[job] [logs] x-pack/test/alerting_api_integration/security_and_spaces/group2/config.ts / alerting api integration security and spaces enabled - Group 2 Alerts alerts alerts superuser at space1 should schedule task, run alert and schedule actions when appropriate
[job] [logs] FTR Configs #65 / Alerting create should store references correctly for actions
[job] [logs] FTR Configs #65 / Alerting create should store references correctly for actions
[job] [logs] FTR Configs #34 / Alerting update system actions should update a rule with a system action correctly
[job] [logs] FTR Configs #34 / Alerting update system actions should update a rule with a system action correctly
[job] [logs] Jest Tests #10 / bulkEdit() index pattern operations should skip operation when params modifiers does not modify index pattern array
[job] [logs] Jest Tests #10 / bulkEditRulesRoute actions adds the type of the actions correctly before passing the request to the rules client
[job] [logs] FTR Configs #9 / detection engine api security and spaces enabled - Group 1 add_actions adding actions should be able to create a new webhook action and attach it to a rule
[job] [logs] FTR Configs #9 / detection engine api security and spaces enabled - Group 1 add_actions adding actions should be able to create a new webhook action and attach it to a rule
[job] [logs] FTR Configs #23 / detection engine api security and spaces enabled - Group 10 import_rules importing rules with an index should migrate legacy actions in existing rule if overwrite is set to true
[job] [logs] FTR Configs #23 / detection engine api security and spaces enabled - Group 10 import_rules importing rules with an index should migrate legacy actions in existing rule if overwrite is set to true
[job] [logs] FTR Configs #23 / detection engine api security and spaces enabled - Group 4 Detection rule type telemetry Detection rule telemetry "kql" rule type should show "notifications_disabled" to be "1" for rule that has at least "1" action(s) and the alert is "disabled"/"in-active"
[job] [logs] FTR Configs #23 / detection engine api security and spaces enabled - Group 4 Detection rule type telemetry Detection rule telemetry "kql" rule type should show "notifications_disabled" to be "1" for rule that has at least "1" action(s) and the alert is "disabled"/"in-active"
[job] [logs] FTR Configs #62 / Monitoring app Cluster listing Alerts should show a toast when alerts are created successfully
[job] [logs] FTR Configs #62 / Monitoring app Cluster listing Alerts should show a toast when alerts are created successfully
[job] [logs] FTR Configs #57 / Observability Rules Synthetics SyntheticsRules creates rule when settings are configured
[job] [logs] FTR Configs #57 / Observability Rules Synthetics SyntheticsRules creates rule when settings are configured

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`alerting`	96	97	+1

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`alerting`	21.8KB	21.9KB	+112.0B

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @cnasikas

ersin-erdal · 2023-09-19T10:21:37Z

x-pack/plugins/alerting/public/lib/common_transformations.ts

@@ -27,6 +34,7 @@ function transformAction(input: AsApiContract<RuleAction>): RuleAction {
          },
        }
      : {}),
+    ...(alertsFilter && { alertsFilter }),


Thanks for fixing this.

ersin-erdal

Went through the code and left some comments.
I will run and test the PR locally then approve.

ersin-erdal · 2023-09-28T16:23:19Z

x-pack/plugins/actions/server/plugin.ts

@@ -586,6 +587,12 @@ export class ActionsPlugin implements Plugin<PluginSetupContract, PluginStartCon
      inMemoryConnectors: this.inMemoryConnectors,
      renderActionParameterTemplates: (...args) =>
        renderActionParameterTemplates(actionTypeRegistry, ...args),
+      isSystemActionConnector: (connectorId: string): boolean => {
+        return !!this.inMemoryConnectors.find(


Nit: inMemoryConnectors.some could be better here.

ersin-erdal · 2023-09-29T09:31:57Z

x-pack/plugins/actions/server/application/connector/methods/get_all/get_all.ts

+    throw error;
+  }
+
+  const systemActions = context.inMemoryConnectors.filter((connector) => connector.isSystemAction);


I think these are connectors not actions (I know isSystemAction is misleading :) )
Even the return type is FindConnectorResult

ersin-erdal · 2023-09-29T12:06:57Z

x-pack/plugins/alerting/server/task_runner/rule_loader.ts

@@ -25,8 +25,12 @@ import {
 import { MONITORING_HISTORY_LIMIT, RuleTypeParams } from '../../common';
 import { AlertingEventLogger } from '../lib/alerting_event_logger/alerting_event_logger';

-export interface RuleData<Params extends RuleTypeParams> extends LoadedIndirectParams<RawRule> {
-  indirectParams: RawRule;


I think indirectParams has been removed by mistake

cnasikas · 2023-10-06T13:31:25Z

Closing the PR in favor of smaller PRs to aid the review process

cnasikas and others added 30 commits June 28, 2023 10:51

Add isSystemAction to ActionResult type and default it to false

1352e52

Disallow registering system actions

09b82f6

Add test

dc1a2c8

Fix alerting types

bb7e9a5

Fix integration tests

774b389

Fix triggers_actions_ui types

83ee747

Fix cases types

d2b8e67

Fix cases tests

31f1f39

Fix integration tests

85ca6a4

Fix cases tests

b070bd9

Merge branch 'main' into system_actions_registration

a8e7143

Fix types from other plugins

b50a3e0

Merge branch 'system_actions_registration' of github.com:cnasikas/kib…

ee5df7b

…ana into system_actions_registration

Rename preconfiguredConnectors to inMemoryConnectors

1caf874

Create system actions

2aabeed

Prevent create/update/delete of system actions

3cf469a

Fix types

8c1246c

Merge branch 'main' into load_system_actions_in_memory

0d7aea1

Fix types

52ef0e5

Remove cases from basic

d763161

Convert preconfigured to in-memory

5c3245e

Merge branch 'main' into load_system_actions_in_memory

31bc750

Get in-memory connectors after they have been created in the route co…

d35a7f9

…ntext

Register test system action in integration tests

34ee2ab

Add integration tests

86b279c

Fix i18n

57b4d3a

Fix types

a162163

Fix integration test

2ea3c9c

Fixes

7362b97

Merge branch 'main' into load_system_actions_in_memory

cbaeb1f

cnasikas added 6 commits September 17, 2023 14:18

Add tests for get/resolve/clone routes

742a0d6

Remove triggers_actions_ui code

53aac58

Fix public APIs

88ce36f

Remove type from legacy objects

a3f33bf

Remove type from create/update legacy routes

70c84d9

Fix bug with actionsClient context (this)

25fca6e

cnasikas commented Sep 18, 2023

View reviewed changes

miltonhultgren approved these changes Sep 18, 2023

View reviewed changes

Remove type from legacy find

132a46d

cnasikas commented Sep 18, 2023

View reviewed changes

Fix rewriteBodyRes

b6e2578

cnasikas marked this pull request as ready for review September 18, 2023 17:10

Improvements and fixes

47ca6e4

ersin-erdal reviewed Sep 19, 2023

View reviewed changes

ersin-erdal reviewed Sep 29, 2023

View reviewed changes

cnasikas closed this Oct 6, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Actions] Add support for system actions in Rules APIs #161726

[Actions] Add support for system actions in Rules APIs #161726

cnasikas commented Jul 12, 2023 •

edited

Loading

cnasikas Sep 18, 2023

cnasikas Sep 18, 2023 •

edited

Loading

cnasikas Sep 18, 2023

cnasikas Sep 18, 2023 •

edited

Loading

cnasikas Sep 18, 2023

cnasikas Sep 18, 2023

cnasikas Sep 18, 2023

cnasikas Sep 18, 2023 •

edited

Loading

cnasikas Sep 18, 2023

cnasikas Sep 18, 2023

cnasikas Sep 18, 2023 •

edited

Loading

cnasikas Sep 18, 2023

cnasikas Sep 18, 2023

cnasikas Sep 18, 2023

kibana-ci commented Sep 19, 2023 •

edited

Loading

ersin-erdal Sep 19, 2023

ersin-erdal left a comment

ersin-erdal Sep 28, 2023

ersin-erdal Sep 29, 2023

ersin-erdal Sep 29, 2023

cnasikas commented Oct 6, 2023

[Actions] Add support for system actions in Rules APIs #161726

[Actions] Add support for system actions in Rules APIs #161726

Conversation

cnasikas commented Jul 12, 2023 • edited Loading

Summary

Checklist

For maintainers

Choose a reason for hiding this comment

cnasikas Sep 18, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cnasikas Sep 18, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cnasikas Sep 18, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cnasikas Sep 18, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kibana-ci commented Sep 19, 2023 • edited Loading

💔 Build Failed

Failed CI Steps

Test Failures

Metrics [docs]

Module Count

Page load bundle

History

Choose a reason for hiding this comment

ersin-erdal left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cnasikas commented Oct 6, 2023

cnasikas commented Jul 12, 2023 •

edited

Loading

cnasikas Sep 18, 2023 •

edited

Loading

cnasikas Sep 18, 2023 •

edited

Loading

cnasikas Sep 18, 2023 •

edited

Loading

cnasikas Sep 18, 2023 •

edited

Loading

kibana-ci commented Sep 19, 2023 •

edited

Loading