[Infra][ObsUX] Hosts & Container Logs only overview #202992

Bluefinger · 2024-12-04T17:37:11Z

Summary

Enables a logs only overview for hosts & containers. Disables the metrics tab as there's no data incoming for metrics, and provides Logs charts on the overview page detailing the Log Rate (all logs generated) and Log Error Rate (all recorded errors).

Recording.2024-12-04.183310.mp4

Closes #201752

How to test

This is tested using synthtrace data, loading the scenario below:

node scripts/synthtrace traces_logs_entities.ts --live

This loads a logs-only host, though if other scenarios contain logs only containers, feel free to use those as well.

Go to Inventory page. Click on a host or container.
If it is a logs only host/container, no metrics tab should be shown. Metrics KPI charts should be replaced with Logs KPI charts (Log Rate and Log Error Rate).
If the host/container contains metrics, the metrics tab should be visible and the normal Metrics KPI charts should be present.

elasticmachine · 2024-12-04T17:37:14Z

Pinging @elastic/obs-ux-infra_services-team (Team:obs-ux-infra_services)

github-actions · 2024-12-04T17:37:28Z

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

/oblt-deploy : Deploy a Kibana instance using the Observability test environments.
run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

cauemarcondes · 2024-12-05T09:34:17Z

The Log charts colour don't match the design.

...gins/observability_solution/infra/public/components/asset_details/tabs/overview/overview.tsx

...ugins/observability_solution/infra/public/components/asset_details/hooks/use_page_header.tsx

...rvability_solution/infra/public/components/asset_details/components/kpis/logs_kpi_charts.tsx

cauemarcondes

LGTM, just some nits

...plugins/observability_solution/infra/public/components/asset_details/hooks/use_log_charts.ts

...ugins/observability_solution/infra/public/components/asset_details/hooks/use_page_header.tsx

Bluefinger · 2024-12-05T14:22:18Z

@elasticmachine merge upstream

Bluefinger · 2024-12-05T18:06:30Z

@elasticmachine merge upstream

crespocarlos · 2024-12-10T09:18:09Z

...plugins/observability_solution/infra/public/components/asset_details/hooks/use_log_charts.ts

+const LOG_RATE = i18n.translate('xpack.infra.assetDetails.charts.logRate', {
+  defaultMessage: 'Log Rate',
+});
+
+const LOG_ERROR_RATE = i18n.translate('xpack.infra.assetDetails.charts.logErrorRate', {
+  defaultMessage: 'Log Error Rate',
+});
+
+const logMetric: LensConfig & { id: string } = {
+  id: 'logMetric',
+  chartType: 'metric',
+  title: LOG_RATE,
+  label: LOG_RATE,
+  trendLine: true,
+  value: 'count()',
+  format: 'number',
+  decimals: 1,
+  normalizeByUnit: 's',
+};
+
+const logErrorMetric: LensConfig & { id: string } = {
+  id: 'logErrorMetric',
+  chartType: 'metric',
+  title: LOG_ERROR_RATE,
+  label: LOG_ERROR_RATE,
+  trendLine: true,
+  value:
+    'count(kql=\'log.level: "error" OR log.level: "ERROR" OR error.log.level: "error" OR error.log.level: "ERROR"\')',
+  format: 'number',
+  decimals: 1,
+  normalizeByUnit: 's',
+};


I'd rather centralize these formulas here x-pack/plugins/observability_solution/metrics_data_access/common/inventory_models/**/formulas but I understand it'd be a relatively big change because the structure is not prepared to support logs

Maybe as a follow-up PR? That way, we don't pollute this one with too many changes and then this change can be evaluated as its own thing?

...plugins/observability_solution/infra/public/components/asset_details/hooks/use_log_charts.ts

...ugins/observability_solution/infra/public/components/asset_details/hooks/use_page_header.tsx

...lity_solution/infra/public/components/asset_details/components/kpis/container_kpi_charts.tsx

elasticmachine · 2024-12-10T14:44:37Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: 60973ef
Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-202992-60973efe23af

Failed CI Steps

Jest Tests #20

Test Failures

[job] [logs] Jest Tests #20 / AgentPolicyListPage Uninstall command flyout should not render "Uninstall agents on this policy" menu item for managed Agent

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`infra`	1539	1542	+3

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`infra`	1.8MB	1.8MB	+1.9KB

History

💚 Build #258631 succeeded ac3ee18
💚 Build #258392 succeeded 77379d6
💔 Build #258154 failed 20537a0
💔 Build #257608 failed a56a28e
💛 Build #257551 was flaky 710c089

kibanamachine · 2024-12-10T16:38:39Z

Starting backport for target branches: 8.17, 8.x

https://github.com/elastic/kibana/actions/runs/12260624734

## Summary Enables a logs only overview for hosts & containers. Disables the metrics tab as there's no data incoming for metrics, and provides Logs charts on the overview page detailing the Log Rate (all logs generated) and Log Error Rate (all recorded errors). https://github.com/user-attachments/assets/ced14b6d-dd08-4514-9066-6c02c62d5ff8 Closes elastic#201752 ## How to test This is tested using synthtrace data, loading the scenario below: ``` node scripts/synthtrace traces_logs_entities.ts --live ``` This loads a logs-only host, though if other scenarios contain logs only containers, feel free to use those as well. * Go to Inventory page. Click on a host or container. * If it is a logs only host/container, no metrics tab should be shown. Metrics KPI charts should be replaced with Logs KPI charts (Log Rate and Log Error Rate). * If the host/container contains metrics, the metrics tab should be visible and the normal Metrics KPI charts should be present. --------- Co-authored-by: Elastic Machine <[email protected]> (cherry picked from commit 305bb1b)

kibanamachine · 2024-12-10T16:47:55Z

💔 Some backports could not be created

Status	Branch	Result
❌	8.17	Backport failed because of merge conflicts
✅	8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Manual backport

To create the backport manually run:

node scripts/backport --pr 202992

Questions ?

Please refer to the Backport tool documentation

## Summary Enables a logs only overview for hosts & containers. Disables the metrics tab as there's no data incoming for metrics, and provides Logs charts on the overview page detailing the Log Rate (all logs generated) and Log Error Rate (all recorded errors). https://github.com/user-attachments/assets/ced14b6d-dd08-4514-9066-6c02c62d5ff8 Closes elastic#201752 ## How to test This is tested using synthtrace data, loading the scenario below: ``` node scripts/synthtrace traces_logs_entities.ts --live ``` This loads a logs-only host, though if other scenarios contain logs only containers, feel free to use those as well. * Go to Inventory page. Click on a host or container. * If it is a logs only host/container, no metrics tab should be shown. Metrics KPI charts should be replaced with Logs KPI charts (Log Rate and Log Error Rate). * If the host/container contains metrics, the metrics tab should be visible and the normal Metrics KPI charts should be present. --------- Co-authored-by: Elastic Machine <[email protected]> (cherry picked from commit 305bb1b) # Conflicts: # x-pack/plugins/observability_solution/infra/public/components/asset_details/tabs/overview/overview.tsx

Bluefinger · 2024-12-11T15:03:30Z

💚 All backports created successfully

Status	Branch	Result
✅	8.17

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

## Summary Enables a logs only overview for hosts & containers. Disables the metrics tab as there's no data incoming for metrics, and provides Logs charts on the overview page detailing the Log Rate (all logs generated) and Log Error Rate (all recorded errors). https://github.com/user-attachments/assets/ced14b6d-dd08-4514-9066-6c02c62d5ff8 Closes elastic#201752 ## How to test This is tested using synthtrace data, loading the scenario below: ``` node scripts/synthtrace traces_logs_entities.ts --live ``` This loads a logs-only host, though if other scenarios contain logs only containers, feel free to use those as well. * Go to Inventory page. Click on a host or container. * If it is a logs only host/container, no metrics tab should be shown. Metrics KPI charts should be replaced with Logs KPI charts (Log Rate and Log Error Rate). * If the host/container contains metrics, the metrics tab should be visible and the normal Metrics KPI charts should be present. --------- Co-authored-by: Elastic Machine <[email protected]>

kibanamachine · 2024-12-12T16:48:47Z

Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync.

kibanamachine · 2024-12-13T16:48:54Z

Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync.

kibanamachine · 2024-12-16T17:48:51Z

Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync.

[Infra][ObsUX] Hosts & Container Logs only overview

b32320d

Bluefinger added release_note:skip Skip the PR/issue when compiling release notes v9.0.0 backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) Team:obs-ux-infra_services Observability Infrastructure & Services User Experience Team labels Dec 4, 2024

Bluefinger requested a review from a team as a code owner December 4, 2024 17:37

botelastic bot added the ci:project-deploy-observability Create an Observability project label Dec 4, 2024

cauemarcondes reviewed Dec 5, 2024

View reviewed changes

Refactor logs content, simplify logs stream checks, better theming

56179d4

cauemarcondes approved these changes Dec 5, 2024

View reviewed changes

elasticmachine and others added 3 commits December 5, 2024 14:22

Merge branch 'main' into logs-only-hosts-container-views

a13acf4

Fix export and naming nits

0893ca4

Fix showing metrics grid at the same time as callout

710c089

Merge branch 'main' into logs-only-hosts-container-views

a56a28e

Bluefinger enabled auto-merge (squash) December 5, 2024 18:10

Merge branch 'main' into logs-only-hosts-container-views

20537a0

Bluefinger disabled auto-merge December 9, 2024 14:01

Remove test assertion that no longer applies

77379d6

Bluefinger added the v8.17.0 label Dec 10, 2024

crespocarlos self-requested a review December 10, 2024 09:16

crespocarlos reviewed Dec 10, 2024

View reviewed changes

Bluefinger added 2 commits December 10, 2024 11:34

Merge branch 'main' into logs-only-hosts-container-views

ac3ee18

Refactor series color resolution, fix nits

60973ef

Bluefinger requested a review from crespocarlos December 10, 2024 13:42

crespocarlos approved these changes Dec 10, 2024

View reviewed changes

Bluefinger merged commit 305bb1b into elastic:main Dec 10, 2024
8 checks passed

kibanamachine mentioned this pull request Dec 10, 2024

[8.x] [Infra][ObsUX] Hosts & Container Logs only overview (#202992) #203644

Open

kibanamachine mentioned this pull request Dec 10, 2024

[Security Solution] Fixes exception item comment validation on newline chars \n #202063

Merged

5 tasks

Bluefinger mentioned this pull request Dec 11, 2024

[8.17] [Infra][ObsUX] Hosts & Container Logs only overview (#202992) #203815

Open

kibanamachine added the backport missing Added to PRs automatically when the are determined to be missing a backport. label Dec 12, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Infra][ObsUX] Hosts & Container Logs only overview #202992

[Infra][ObsUX] Hosts & Container Logs only overview #202992

Bluefinger commented Dec 4, 2024 •

edited by kibanamachine

Loading

elasticmachine commented Dec 4, 2024

github-actions bot commented Dec 4, 2024

cauemarcondes commented Dec 5, 2024

cauemarcondes left a comment •

edited

Loading

Bluefinger commented Dec 5, 2024

Bluefinger commented Dec 5, 2024

crespocarlos Dec 10, 2024

Bluefinger Dec 10, 2024

elasticmachine commented Dec 10, 2024 •

edited

Loading

kibanamachine commented Dec 10, 2024

kibanamachine commented Dec 10, 2024

Bluefinger commented Dec 11, 2024

kibanamachine commented Dec 12, 2024

kibanamachine commented Dec 13, 2024

kibanamachine commented Dec 16, 2024

[Infra][ObsUX] Hosts & Container Logs only overview #202992

[Infra][ObsUX] Hosts & Container Logs only overview #202992

Conversation

Bluefinger commented Dec 4, 2024 • edited by kibanamachine Loading

Summary

How to test

elasticmachine commented Dec 4, 2024

github-actions bot commented Dec 4, 2024

🤖 GitHub comments

cauemarcondes commented Dec 5, 2024

cauemarcondes left a comment • edited Loading

Choose a reason for hiding this comment

Bluefinger commented Dec 5, 2024

Bluefinger commented Dec 5, 2024

crespocarlos Dec 10, 2024

Choose a reason for hiding this comment

Bluefinger Dec 10, 2024

Choose a reason for hiding this comment

elasticmachine commented Dec 10, 2024 • edited Loading

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

Metrics [docs]

Module Count

Async chunks

History

kibanamachine commented Dec 10, 2024

kibanamachine commented Dec 10, 2024

💔 Some backports could not be created

Manual backport

Questions ?

Bluefinger commented Dec 11, 2024

💚 All backports created successfully

Questions ?

kibanamachine commented Dec 12, 2024

kibanamachine commented Dec 13, 2024

kibanamachine commented Dec 16, 2024

Bluefinger commented Dec 4, 2024 •

edited by kibanamachine

Loading

cauemarcondes left a comment •

edited

Loading

elasticmachine commented Dec 10, 2024 •

edited

Loading