Skip to content

Commit

Permalink
[Security Solution] Array of scalar values diff algorithm test plan (#…
Browse files Browse the repository at this point in the history 
…186325)

## Summary

Related ticket: #180162

Adds test plan for diff algorithm for arrays of scalar values
implemented here: #186323

### For maintainers

- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
  • Loading branch information
@dplumlee
dplumlee authored Jul 9, 2024
1 parent 47178a7 commit 0b405a0
Show file tree
Hide file tree
Showing 2 changed files with 246 additions and 146 deletions.
146 changes: 0 additions & 146 deletions ...esting/test_plans/detection_response/prebuilt_rules/installation_and_upgrade.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -55,14 +55,6 @@ Status: `in progress`. The current test plan matches `Milestone 2` of the [Rule
- [**Scenario: User can see correct rule information in preview before upgrading**](#scenario-user-can-see-correct-rule-information-in-preview-before-upgrading)
- [**Scenario: Tabs and sections without content should be hidden in preview before upgrading**](#scenario-tabs-and-sections-without-content-should-be-hidden-in-preview-before-upgrading)
- [Rule upgrade workflow: filtering, sorting, pagination](#rule-upgrade-workflow-filtering-sorting-pagination)
- [Rule upgrade workflow: simple diff algorithm](#rule-upgrade-workflow-simple-diff-algorithm)
- [**Scenario: Rule field doesn't have an update and has no custom value**](#scenario-rule-field-doesnt-have-an-update-and-has-no-custom-value)
- [**Scenario: Rule field doesn't have an update but has a custom value**](#scenario-rule-field-doesnt-have-an-update-but-has-a-custom-value)
- [**Scenario: Rule field has an update and doesn't have a custom value**](#scenario-rule-field-has-an-update-and-doesnt-have-a-custom-value)
- [**Scenario: Rule field has an update and a custom value that are the same**](#scenario-rule-field-has-an-update-and-a-custom-value-that-are-the-same)
- [**Scenario: Rule field has an update and a custom value that are NOT the same**](#scenario-rule-field-has-an-update-and-a-custom-value-that-are-not-the-same)
- [**Scenario: Rule field has an update and a custom value that are NOT the same and the rule base version doesn't exist**](#scenario-rule-field-has-an-update-and-a-custom-value-that-are-not-the-same-and-the-rule-base-version-doesnt-exist)
- [**Scenario: Rule field has an update and a custom value that are the same and the rule base version doesn't exist**](#scenario-rule-field-has-an-update-and-a-custom-value-that-are-the-same-and-the-rule-base-version-doesnt-exist)
- [Rule upgrade workflow: viewing rule changes in JSON diff view](#rule-upgrade-workflow-viewing-rule-changes-in-json-diff-view)
- [**Scenario: User can see changes in a side-by-side JSON diff view**](#scenario-user-can-see-changes-in-a-side-by-side-json-diff-view)
- [**Scenario: User can see precisely how property values would change after upgrade**](#scenario-user-can-see-precisely-how-property-values-would-change-after-upgrade)
Expand Down Expand Up @@ -823,144 +815,6 @@ And the Investigation Guide tab should NOT be displayed

TODO: add scenarios https://github.com/elastic/kibana/issues/166215

### Rule upgrade workflow: simple diff algorithm

#### **Scenario: Rule field doesn't have an update and has no custom value**

**Automation**: 2 integration tests with mock rules + a set of unit tests for the algorithm

```Gherkin
Given at least 1 prebuilt rule is installed in Kibana
And for this rule there is a new version available
And <field_name> field is not customized by the user (current version == base version)
And <field_name> field is not updated by Elastic in this upgrade (target version == base version)
Then for <field_name> field the diff algorithm should output the current version as the merged one without a conflict
And <field_name> field should not be returned from the `upgrade/_review` API endpoint
And <field_name> field should not be shown in the upgrade preview UI
Examples:
| field_name | base_version | current_version | target_version |
| name | "A" | "A" | "A" |
| risk_score | 1 | 1 | 1 |
```

#### **Scenario: Rule field doesn't have an update but has a custom value**

**Automation**: 2 integration tests with mock rules + a set of unit tests for the algorithm

```Gherkin
Given at least 1 prebuilt rule is installed in Kibana
And for this rule there is a new version available
And <field_name> field is customized by the user (current version != base version)
And <field_name> field is not updated by Elastic in this upgrade (target version == base version)
Then for <field_name> field the diff algorithm should output the current version as the merged one without a conflict
And <field_name> field should be returned from the `upgrade/_review` API endpoint
And <field_name> field should be shown in the upgrade preview UI
Examples:
| field_name | base_version | current_version | target_version |
| name | "A" | "B" | "A" |
| risk_score | 1 | 2 | 1 |
```

#### **Scenario: Rule field has an update and doesn't have a custom value**

**Automation**: 2 integration tests with mock rules + a set of unit tests for the algorithm

```Gherkin
Given at least 1 prebuilt rule is installed in Kibana
And for this rule there is a new version available
And <field_name> field is not customized by the user (current version == base version)
And <field_name> field is updated by Elastic in this upgrade (target version != base version)
Then for <field_name> field the diff algorithm should output the target version as the merged one without a conflict
And <field_name> field should be returned from the `upgrade/_review` API endpoint
And <field_name> field should be shown in the upgrade preview UI
Examples:
| field_name | base_version | current_version | target_version |
| name | "A" | "A" | "B" |
| risk_score | 1 | 1 | 2 |
```

#### **Scenario: Rule field has an update and a custom value that are the same**

**Automation**: 2 integration tests with mock rules + a set of unit tests for the algorithm

```Gherkin
Given at least 1 prebuilt rule is installed in Kibana
And for this rule there is a new version available
And <field_name> field is customized by the user (current version != base version)
And <field_name> field is updated by Elastic in this upgrade (target version != base version)
And customized <field_name> field is the same as the Elastic update in this upgrade (current version == target version)
Then for <field_name> field the diff algorithm should output the current version as the merged one without a conflict
And <field_name> field should be returned from the `upgrade/_review` API endpoint
And <field_name> field should be shown in the upgrade preview UI
Examples:
| field_name | base_version | current_version | target_version |
| name | "A" | "B" | "B" |
| risk_score | 1 | 2 | 2 |
```

#### **Scenario: Rule field has an update and a custom value that are NOT the same**

**Automation**: 2 integration tests with mock rules + a set of unit tests for the algorithm

```Gherkin
Given at least 1 prebuilt rule is installed in Kibana
And for this rule there is a new version available
And <field_name> field is customized by the user (current version != base version)
And <field_name> field is updated by Elastic in this upgrade (target version != base version)
And customized <field_name> field is different than the Elastic update in this upgrade (current version != target version)
Then for <field_name> field the diff algorithm should output the current version as the merged one with a conflict
And <field_name> field should be returned from the `upgrade/_review` API endpoint
And <field_name> field should be shown in the upgrade preview UI
Examples:
| field_name | base_version | current_version | target_version |
| name | "A" | "B" | "C" |
| risk_score | 1 | 2 | 3 |
```

#### **Scenario: Rule field has an update and a custom value that are NOT the same and the rule base version doesn't exist**

**Automation**: 2 integration tests with mock rules + a set of unit tests for the algorithm

```Gherkin
Given at least 1 prebuilt rule is installed in Kibana
And for this rule there is a new version available
And the base version of the rule cannot be determined
And customized <field_name> field is different than the Elastic update in this upgrade (current version != target version)
Then for <field_name> field the diff algorithm should output the target version as the merged one with a conflict
And <field_name> field should be returned from the `upgrade/_review` API endpoint
And <field_name> field should be shown in the upgrade preview UI
Examples:
| field_name | base_version | current_version | target_version |
| name | N/A | "B" | "C" |
| risk_score | N/A | 2 | 3 |
```

#### **Scenario: Rule field has an update and a custom value that are the same and the rule base version doesn't exist**

**Automation**: 2 integration tests with mock rules + a set of unit tests for the algorithm

```Gherkin
Given at least 1 prebuilt rule is installed in Kibana
And for this rule there is a new version available
And the base version of the rule cannot be determined
And customized <field_name> field is the same as the Elastic update in this upgrade (current version == target version)
Then for <field_name> field the diff algorithm should output the current version as the merged one without a conflict
And <field_name> field should not be returned from the `upgrade/_review` API endpoint
And <field_name> field should not be shown in the upgrade preview UI
Examples:
| field_name | base_version | current_version | target_version |
| name | N/A | "A" | "A" |
| risk_score | N/A | 1 | 1 |
```

### Rule upgrade workflow: viewing rule changes in JSON diff view

#### **Scenario: User can see changes in a side-by-side JSON diff view**
Expand Down
Loading

0 comments on commit 0b405a0

Please sign in to comment.