Skip to content

Commit

Permalink
[Security Solution] Per-field diffs test plan (#176474)
Browse files Browse the repository at this point in the history
## Summary

Resolves #176473

This PR introduces a test plan for the per-field diff preview. This
preview is displayed in the upgrade prebuilt rule flyout under the
`Updates` tab.

<img width="1538" alt="Screenshot 2024-02-08 at 1 35 05 AM"
src="https://github.com/elastic/kibana/assets/56367316/37e25106-f4a5-47e6-9d93-beab8d0f90a4">
  • Loading branch information
dplumlee authored Feb 12, 2024
1 parent 956fa0a commit e93add0
Showing 1 changed file with 98 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,11 @@ Status: `in progress`. The current test plan matches `Milestone 2` of the [Rule
- [**Scenario: Properties with semantically equal values should not be shown as modified**](#scenario-properties-with-semantically-equal-values-should-not-be-shown-as-modified)
- [**Scenario: Unchanged sections of a rule should be hidden by default**](#scenario-unchanged-sections-of-a-rule-should-be-hidden-by-default)
- [**Scenario: Properties should be sorted alphabetically**](#scenario-properties-should-be-sorted-alphabetically)
- [Rule upgrade workflow: viewing rule changes in per-field diff view](#rule-upgrade-workflow-viewing-rule-changes-in-per-field-diff-view)
- [**Scenario: User can see changes in a side-by-side per-field diff view**](#scenario-user-can-see-changes-in-a-side-by-side-per-field-diff-view)
- [**Scenario: Field groupings should be rendered together in the same accordion panel**](#scenario-field-groupings-should-be-rendered-together-in-the-same-accordion-panel)
- [**Scenario: Undefined values are displayed with empty diffs**](#scenario-undefined-values-are-displayed-with-empty-diffs)
- [**Scenario: Field diff components have the same grouping and order as in rule details overview**](#scenario-field-diff-components-have-the-same-grouping-and-order-as-in-rule-details-overview)
- [Rule upgrade workflow: preserving rule bound data](#rule-upgrade-workflow-preserving-rule-bound-data)
- [**Scenario: Rule bound data is preserved after upgrading a rule to a newer version with the same rule type**](#scenario-rule-bound-data-is-preserved-after-upgrading-a-rule-to-a-newer-version-with-the-same-rule-type)
- [**Scenario: Rule bound data is preserved after upgrading a rule to a newer version with a different rule type**](#scenario-rule-bound-data-is-preserved-after-upgrading-a-rule-to-a-newer-version-with-a-different-rule-type)
Expand Down Expand Up @@ -953,6 +958,99 @@ When a user expands all hidden sections
Then all properties of the rule should be sorted alphabetically
```

### Rule upgrade workflow: viewing rule changes in per-field diff view

#### **Scenario: User can see changes in a side-by-side per-field diff view**

**Automation**: 1 e2e test

```Gherkin
Given X prebuilt rules are installed in Kibana
And for Y of these rules new versions are available
When user opens the Rule Updates table and selects a rule
Then the per-field upgrade preview should open
And rule changes should be displayed in a two-column diff view with each field in its own accordion component
And all field diff accordions should be open by default
And correct rule version numbers should be displayed in their respective columns
When the user selects another rule without closing the preview
Then the preview should display the changes for the newly selected rule
```

#### **Scenario: User can see changes when updated rule is a different rule type**

**Automation**: 1 UI integration test

```Gherkin
Given a prebuilt rule is installed in Kibana
And this rule has an update available that changes the rule type
When user opens the upgrade preview
Then the rule type changes should be displayed in grouped field diffs with corresponding query fields
And a tooltip is displayed with information about changing rule types
```

#### **Scenario: Field groupings should be rendered together in the same accordion panel**

**Automation**: 1 UI integration test

```Gherkin
Given a prebuilt rule is installed in Kibana
And this rule contains one or more <field> values
When user opens the upgrade preview
The <field> diff accordion panel should display its grouped rule properties
And each property should have its name displayed inside the panel above its value
Examples:
| field |
| data_source |
| kql_query |
| eql_query |
| esql_query |
| threat_query |
| rule_schedule |
| rule_name_override |
| timestamp_override |
| timeline_template |
| building_block |
| threshold |
```

#### **Scenario: Undefined values are displayed with empty diffs**

**Automation**: 1 UI integration test

```Gherkin
Given a prebuilt rule is installed in Kibana
And this rule has field in the <version_one> version that didn't exist in the <version_two> version
When a user opens the upgrade preview
Then the preview should open
And the old/new field should render an empty panel
Examples:
| version_one | version_two |
| target | current |
| current | target |
```

#### **Scenario: Field diff components have the same grouping and order as in rule details overview**

**Automation**: 1 UI integration test

```Gherkin
Given a prebuilt rule is installed in Kibana
And this rule has multiple fields that are different between the current and target version
When a user opens the upgrade preview
Then the multiple field diff accordions should be sorted in the same order as on the rule details overview tab
And the field diff accordions should be grouped inside its corresponding <section> accordion
And any <section> accordion that doesn't have fields inside it shouldn't be displayed
Examples:
| section |
| About |
| Definition |
| Schedule |
| Setup Guide |
```

### Rule upgrade workflow: preserving rule bound data

#### **Scenario: Rule bound data is preserved after upgrading a rule to a newer version with the same rule type**
Expand Down

0 comments on commit e93add0

Please sign in to comment.