The max_signals field for rules can support up to 1000 alerts

[nastasha-solomon]

Description

Will need to update the max_signals field description in the Create rule API docs to indicate that the max supported value is 1000, even if the API allows users to set any value.

Related:

• [Security Solution] Detection Rules' max_signals parameter is not respected beyond 1000 kibana#157313

Notes/questions

• Only need to backport to 8.8.
• Need to verify if this should be doc'd as a known issue. If there are plans in the future to reduce the max value for the max_signals field (e.g., lower it back down to 100), this might need to be release-noted in 8.8.x.

cc: @peluja1012

[e40pud]

Adding the link to the discussion https://elastic.slack.com/archives/CN7AD3605/p1683741078780019

[rylnd]

How far should this change be backported? Seems like it might've been introduced in 8.8 with the conditional actions feature so we might need to only apply this change to the 8.8 docs.

I think 8.8+ is appropriate; this change came from the feature work around per-alert actions

Need to verify if this should be doc'd as a known issue.

1000 seems like a reasonable limit, and there are workarounds if you (for some reason) need to generate a TON of alerts (i.e. run more frequently), so I don't imagine this changing in the near future. I think it's best to document the limitation so as not to confuse/surprise users, but I defer to @peluja1012 here.

I wouldn't be opposed to adding API validation to enforce this (and serve as self-documentation) in the future, as well.

[peluja1012]

Need to verify if this should be doc'd as a known issue.

I think it's worth documenting it as a known issue to further highlight the change in behavior. Although my hope is that we don't have too many users with a max_signals setting higher than 1000.

[nastasha-solomon]

Hey, @e40pud - thanks for the additional information. I still have a few questions:

Here is the overview of these attributes and the difference between them:
max_signals is a Security Solution “create rule” API optional field. Users have access to it only via direct API call. We don’t expose this field in our UI. This field defines the maximum number of alerts the rule can create during a single execution. Security Solution won’t fail the rule if there are more alerts than the specified number. It will just not create alerts above this number.

So basically, this field determines the max number of alerts a Security detection rule can generate in a single rule execution cycle. Once the limit is met, no more alerts are created during the rule’s execution window but the rule continues to execute on its defined schedule.

xpack.alerting.rules.run.alerts.max is an Alerting Framework configuration which works similarly to the max_signals but used within the alerting framework. Alerting framework will fail the rule if we will try to create more alerts than this number.

Couple questions here:

• Are you saying that the xpack.alerting.rules.run.alerts.max field value determines the max alert limit per rule execution for all rules using the Kibana alerting framework (so Kibana, Observability, and Security rules)? I really don’t know the nuances of how Security rules fit into the Kibana alerting framework so apologies if this is an ignorant question.
• Am I understanding this correctly: If a user was creating a Security rule, and they set the max_signals value higher than the xpack.alerting.rules.run.alerts.max value, the rule would fail once it hit the value defined in the xpack.alerting.rules.run.alerts.max field.

Before conditional actions changes in 8.8, security solution would create only one alert via alerting framework where we would summarize all the generated signals (alerts). And we would never reach alerting framework xpack.alerting.rules.run.alerts.max limit.

Does the last sentence still apply if the user selected the On each rule execution action frequency for a rule action created in 8.7 or ealier? Or does it for

After conditional actions changes in 8.8 (to support “for each alert” notifications), security solution creates as many alerts via alerting framework as many were generated during the rule execution. Meaning, that if max_signals > xpack.alerting.rules.run.alerts.max we will reach alerting framework limit and it will throw an exception and fail the rule.
That’s why, we need to put in documentation the note to the user that the security solution max_signals cannot be bigger than the alerting framework xpack.alerting.rules.run.alerts.max.

Should the message be “Do not set a max_signals value that’s higher than the xpack.alerting.rules.run.alerts.max value.” Or, do we want to tell user that the max value they can set for the max_signals field is 1000 and if they want to raise that limit, they’ll also need to increase the xpack.alerting.rules.run.alerts.max value?

[e40pud]

@nastasha-solomon here are the answers:

So basically, this field determines the max number of alerts a Security detection rule can generate in a single rule execution cycle. Once the limit is met, no more alerts are created during the rule’s execution window but the rule continues to execute on its defined schedule.

Correct

Are you saying that the xpack.alerting.rules.run.alerts.max field value determines the max alert limit per rule execution for all rules using the Kibana alerting framework (so Kibana, Observability, and Security rules)? I really don’t know the nuances of how Security rules fit into the Kibana alerting framework so apologies if this is an ignorant question.

Yes, xpack.alerting.rules.run.alerts.max is an alerting framework configuration which is used to limit number of alerts created within the framework per rule run. Alerting framework is used by all other applications like Security Solution or Observability.

Am I understanding this correctly: If a user was creating a Security rule, and they set the max_signals value higher than the xpack.alerting.rules.run.alerts.max value, the rule would fail once it hit the value defined in the xpack.alerting.rules.run.alerts.max field.

Correct. Alerting framework does its own alerts count check against the xpack.alerting.rules.run.alerts.max and throws an exception if created alerts count is bigger.

Does the last sentence still apply if the user selected the On each rule execution action frequency for a rule action created in 8.7 or ealier? Or does it for

I think there might be a confusion here. It does not really matter if it is a "on each rule execution" or custom frequency (hourly, daily, weekly). "On each rule execution" means we would notify about all the alerts generated during last rule execution and timeframe will be the rule schedule interval in this case.

Should the message be “Do not set a max_signals value that’s higher than the xpack.alerting.rules.run.alerts.max value.” Or, do we want to tell user that the max value they can set for the max_signals field is 1000 and if they want to raise that limit, they’ll also need to increase the xpack.alerting.rules.run.alerts.max value?

I would say something like: "max_signals cannot be higher than xpack.alerting.rules.run.alerts.max (link to this config in alerting framework documentation) otherwise it will lead to a rule failure.". Maybe we can present it as a known issue. @peluja1012 what do you think?

[nastasha-solomon]

@rylnd if you do end up adding API validation in a future release that prevents users from setting the max_signals value higher than the xpack.alerting.rules.run.alerts.max value, let me know so I can remove this note in the docs. Thank you! 🙏🏼

[nastasha-solomon]

@e40pud @peluja1012 updated docs for the max_signals field in #3293 and release-noted the new behavior as a known issue in 8.8 here.