Migrate Host risk and User risk UI to ECS schema

[machadoum]

Links:
issue
PR for transforms and dashboards
proposed ECS risk fields.

Summary

Migrate Host risk and User risk indices to the ECS schema

Affected components:

• User risk score tab
• User detail overview
• User detail risk score tab
• Host detail overview
• Host risk score tab
• Host detail risk score tab
• Host flyout
• User flyout
• Entity analytics Host table
• Entity analytics User table
• Host overview
• All hosts tab (risk column)

Add mocked data

node ./scripts/es_archiver load ./x-pack/test/security_solution_cypress/es_archives/risky_users --config ./x-pack/test/security_solution_cypress/config.ts --es-url https://admin:<PASSWORD>@<ENVIRONMENT> --kibana-url http://admin:<PASSWORD>@localhost:5601/

node ./scripts/es_archiver load ./x-pack/test/security_solution_cypress/es_archives/risky_hosts --config ./x-pack/test/security_solution_cypress/config.ts --es-url https://admin:<PASSWORD>@<ENVIRONMENT> --kibana-url http://admin:<PASSWORD>@localhost:5601/

TODO

• Test if it works with Migrate Host and User Risk Score transforms and dashboards to ECS #139865

[elasticmachine]

Pinging @elastic/security-threat-hunting (Team:Threat Hunting)

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[angorayc]

Suggested change

	title: host.name,
	title: get(item, RiskScoreFields.hostName),

[machadoum]

get isn't type-safe.

[angorayc]

Suggested change

	count: host.risk.calculated_score_norm,
	count: get(item, RiskScoreFields.hostRiskScore),

[angorayc]

Suggested change

	copy: host.risk.calculated_level,
	copy: get(item, RiskScoreFields.hostRisk),

[angorayc]

Suggested change

	? Math.round(userRiskData.user.risk.calculated_score_norm)
	? Math.round(get(userRiskData, RiskScoreFields.userRiskScore))

[angorayc]

Suggested change

	severity={userRiskData.user.risk.calculated_level as RiskSeverity}
	severity={get(userRiskData, RiskScoreFields.userRisk)}

[angorayc]

Suggested change

	[hit._source?.host.name ?? '']: hit._source?.host.risk.calculated_level,
	[hit._source?.host.name ?? '']: get(hit, `_source.${RiskScoreFields.hostRisk}`),

[kibana-ci]

💚 Build Succeeded

• Buildkite Build
• Commit: 48842e7

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
securitySolution	6.4MB	6.4MB	+355.0B

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
securitySolution	259.9KB	260.1KB	+222.0B

History

• 💔 Build #70411 failed 9b774a806c8e60b5940712a1b4d8fc9c6e242e79
• 💛 Build #70322 was flaky 47a0b7d
• 💔 Build #69965 failed 2c54e53359f2e4363fc48f5a73cb7c72c9418815
• 💔 Build #69955 failed f8bce6cfd0c6382076cf53d1384cf094c0e8b5eb
• 💔 Build #69710 failed fad3f1d566d72fea6c8993b8409bb8d3410e3021
• 💔 Build #69684 failed 7eb59926aa762bdc017d40b02812a4f0dea924a6

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @machadoum

[MadameSheema]

security-engineering-productivity changes LGTM

[logeekal]

@machadoum , Since we are already aggregating by a field such as RiskScoreFields.userRisk or RiskScoreFields.hostRisk, wouldn't cardinality be always 1 (i.e. unique_entries) for that particular field? Is it intensional? See screenshot below.

Do you think below was the intention? Apologies if I have missed something.

 {
  "aggs": {
    "risk": {
      "terms": {
        "field": "user.risk.calculated_score_norm"
      }
    },
    "unique_entries": {
      "cardinality": {
        "field": "user.risk.calculated_score_norm"
      }
    }
  }
}

[machadoum]

There is a mistake in the code above, unique_entries.cardinality.field should be user.name or host.name. If the fields of the term aggregation and cardinality are the same, the result would always be 1, but that is not the case. Here is a sample of the query that runs:

    "aggs": {
      "risk": {
        "terms": {
          "field": "host.risk.calculated_level"
        },
        "aggs": {
          "unique_entries": {
            "cardinality": {
              "field": "host.name"
            }
          }
        }
    }

The terms aggregation groups all entries by risk severity, and cardinality counts how many users or hosts exist for each severity.

[logeekal]

Okay... this makes sense. My bad 😰

[logeekal]

PR checked-out and desk tested. User risk fields are populating without any issue. LGTM