[Security Solution] Array of scalar values diff algorithm test plan

[dplumlee]

Summary

Related ticket: #180162

Adds test plan for diff algorithm for arrays of scalar values implemented here: #186323

For maintainers

• This was checked for breaking API changes and was labeled appropriately

[elasticmachine]

Pinging @elastic/security-detections-response (Team:Detections and Resp)

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[elasticmachine]

Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management)

[dplumlee]

Trying to find a better name for all these scenarios that would be unique enough to use in a TOC, any suggestions?

[nikitaindik]

While not perfect, I think adding a three-letter type should work.

### Rule field doesn't have an update and has no custom value - `AAA`
#### **Scenario: AAA - Rule field is any type**

Yep, some duplication, but it's not too bad IMO.

[nikitaindik]

Does this mean that it's only applicable to some fields and for other fields order doesn't matter?

[dplumlee]

Not sure if I understand the question, things that aren't array fields wouldn't have an order per say. This is basically saying that any fields that are arrays should perform the same no matter the order (e.g. an array of ["A", "B"] would be treated the same by the algorithm as ["B", "A"]).

[nikitaindik]

Oh, I should have phrased it better. I meant "do we have arrays where order matters and arrays where it doesn't?" I have just checked the code and seen that we don't ever expect the order to matter.

Then I am confused by the meaning of "CASE" keyword. I understand it as "note that you should also test this extra case in addition to a default one". As if by default the order is important, but there is also a case where it isn't.

But here we actually want to convey: "note that all arrays are order-agnostic".

IMO, if we had something like NOTE instead of CASE it wouldn't be confusing.

NOTE: array fields should work the same agnostic of order

I can't find CASE in the Gherkin spec. Is it something that we have introduced ourselves? What's your opinion on using NOTE vs CASE?

[nikitaindik]

Thanks @dplumlee! I have reviewed and left a few questions and suggestions.

[jpdjere]

Taking a look at the feedback Georgii left on #186323, I think we should reflect those test cases in the test plan. So I think it still makes sense to have a general test case scenario for "any type", but then for scalar arrays, we need a different section not only on scenarios ABC and -BC but in all. We should reflect cases in which, in the change from base to current and target:

• an element can be added to either/both of them
• an element can be removed from either/both of them
• an element can stay unchanged in both of them
• elements can be reordered in either/both of them in the same way or different ways
• elements can be duplicated at any given index in an array

Also, for scalar arrays, its especially important to describe the behaviour for how we handle:

• duplicated values
• reordering of values
• case-sensitivity
• empty arrays

as discussed in the PR linked above.

Also, let's change our examples to always use at least 3 elements as default, of course, except when needing to make an example out of adding or removing them.

[jpdjere]

correct to sensitive

[jpdjere]

We ended up changing the logic for this, as explained in this thread. The diff should be case sensitive.

I would replace the last example for one that uses index patterns. In that case, it makes much more clear why it should be case-sensitive, as explained in the threat I linked above. We could have something like:

| algorithm        | field_name | base_version            | current_version         | target_version            | merged_version                 |
| array of scalars | index       | ["logs-*", "endgame-*", "endpoint-*"]   | ["Logs-*", "endgame-*"]          | ["logs-*", "endgame-*", "new-*"]         | ["Logs-*", "endgame-*", "new-*"]               |
| array of scalars | index       | ["logs-*"]   | ["logs-*", "Logs-*"]          | ["logs-*", "new-*"]         | ["logs-*", "Logs-*", "new-*"]               |

In the first case, "Logs-" would be preserved since it was added by the user, and "logs-" was removed by the user, so it does not appear in the final version.

In the second example, both are preserved, since they do not represent the same value to the algorithm.

[dplumlee]

I can put it in the example, but index won't technically be covered by this algorithm as it's under the grouped data source field

[jpdjere]

This looks much more cleaner and complete now 👍

But one more thing pending to update: our algorithm for scalar values should be case-sensitive, not insensitive. I left a comment with a link to the discussion we had about that.

[jpdjere]

Thanks for addressing the changes! LGTM ✅

[kibanamachine]

💚 All backports created successfully

Status	Branch	Result
✅	8.15

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation