[Security Solution] Grouping - fix null group missing

[stephmilovic]

Adjusts the grouping query to include a "missing" group in the groupByFields bucket results. The "missing" field in the terms aggregation needs to match the field's es type. A helper function, getFieldTypeMissingValue, assigns this value according to the es type. In order to avoid leaving out events that are equivalent with the default value, we are checking 2 default values. This looks like:

    groupByFields: {
      multi_terms: {
        terms: [
          {
            field: groupByField,
            missing: getFieldTypeMissingValues(selectedGroupEsTypes)[0],
          },
          {
            field: groupByField,
            missing: getFieldTypeMissingValues(selectedGroupEsTypes)[1],
          },
        ],
        size: MAX_QUERY_SIZE,
      }
        ...

Once we get the response, we use parseGroupingQuery . For each group, it checks if the elements of the key property are equal. If they are, it sets the key/key_as_string properties to an array with the first element of key, the grouping field, and sets the isNullGroup property to false. If they are not equal, it sets the key to -, and isNullGroup to true. This is so that we have a flag in the UI for the "missing" group.

This is what it looks like in the UI:

user.name

source.ip

[stephmilovic]

this is an unrelated change, but a follow up from the last PR.

[elasticmachine]

Pinging @elastic/security-threat-hunting (Team:Threat Hunting)

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[YulNaumenko]

LGTM. Have a minor comment about nullGroupMessage

[stephmilovic]

I changed this to be a host query since its more realistic for testing, rule name will not have a null group.

[stephmilovic]

@elasticmachine merge upstream

[semd]

This looks awesome!! Thanks @stephmilovic
I wrote a couple of minor suggestions, feel free to apply them.
LGTM! 🚀

[kibana-ci]

💚 Build Succeeded

• Buildkite Build
• Commit: 33b4153
• Storybooks Preview

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
securitySolution	3859	3860	+1

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
securitySolution	9.1MB	9.1MB	+2.7KB

Unknown metric groups

ESLint disabled line counts

id	before	after	diff
enterpriseSearch	17	19	+2
securitySolution	399	402	+3
total			+5

References to deprecated APIs

id	before	after	diff
securitySolution	546	547	+1

Total ESLint disabled count

id	before	after	diff
enterpriseSearch	18	20	+2
securitySolution	479	482	+3
total			+5

History

• 💔 Build #123979 failed 0d345e0
• 💔 Build #123951 failed db83fe8
• 💔 Build #123935 failed 7f97490
• 💚 Build #123674 succeeded 1547a49
• 💔 Build #123658 failed f7a90c9

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

[kibanamachine]

💚 All backports created successfully

Status	Branch	Result
✅	8.8

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation