[UX] Add quick create for log type in Create/Edit detection rule flow

[xeniatup]

#827

Problem:

The detection rules in Security Analytics scan log data to produce security findings representing potential threats. Additionally, user can create new detection rules and customize them for the custom log sources.
Creating a log type to represent a custom log source is an experience performed on a dedicated page. Having to switch to a different page in order to manage custom log types takes user out of the context of the creating or editing detection rules.

Current experience:

In “Create detection rule” flow within Security Analytics plugin user provides parameters to define rule metadata and detection fields. One of the mandatory parameters of the rule is Log type, which is internal abstraction to categorize objects and write queries for Security Analytics. The concept of Log type originates from Sigma rules framework where logsource: fields define how to onboard the log data source correctly.

A log type as an object has Name, Description and a log type category it belongs to in OpenSearch. User can create as many custom log types as they like and categorize them in the pre-existing categories.

When user creates a custom detection rule, they might need to create a new log type to proceed with the flow. We provide a “Manage” button next to the Log type select field, which opens a Security Analytics → Detection rules -> Log types page listing all available log types.

Proposal:

Given the simplicity of the “Create log type” flow we propose to allow user to create a log type without loosing context (in a OuiFlyout ) to be able to then proceed with creating or editing a detection rule.

We provide a “Create log type” button next to the “Log type” select field, which opens “Create log type” flow in a flyout. We add a backdrop to block the parent page from editing and avoid confusion for the main flow CTAs. After the form is submitted, and a log type is created, a success toast message is shown on closing the flyout. The “Log type” select is updated without reloading the page, and the newly created log type gets selected (as we assume user’s intention).

Additional considerations:

Security Analytics experiences that include “Log type” selection

We have other experiences that include selecting a log type, e.g. “Create threat detector” and “Create correlation rule”. I would not recommend to include contextual creation of log types in those flows. If user creates a new log type within “Create threat detector” flow, it doesn’t provide them with the detection rules for the new log type, hence the change doesn't improve or simplify achieving the main goal of the flow (to apply a set of detection rules to a data source). User should take care of the detection rules first, and then create a detector. Creating a log type is an intermediary step.
For “Create correlation rule” flow we ask for a log type as a part of the query to search the existing findings, and creating a new log type doesn’t streamline the overall workflow.

Using the pattern for other experiences in OpenSearch

It might be beneficial to consider the pattern to create objects in a quick contextual flow, get the user feedback on it, and expand to other experiences within the product. A good candidate being “Create notification channel” under "Notigications" flow, when performed contextually can unblock user in more complex flows like “Create threat detector” (Security Analytics) and “Create alerting monitor”(Alerting).