Pinging @elastic/infra-monitoring-ui (Team:Infra Monitoring UI)

Paired with @miltonhultgren on this today aiming at refining what the query would look like initially and what data we can/want to extract out of the documents returned.

Observations:

• targeted metrics-*,metricbeat-* indices
• there are not a lot of hosts (<10) available in edge-oblt
• the query will collapse on host.hostname. We'll likely want to surface documents that have more metadata included (cloud.*, kubernetes.*..), how can we do that ? For example, if a hostname is represented by two documents and only one of them have cloud.service.name, can collapse favors it instead of the other ?
• how can we rule out false positive (ie host.hostname is wrongly filled by apm php) ? We won't at first, but one potential solution is to build a blocklist of offending sources
• some hosts may not have the metadata necessary to set the correct type (ie gce/ec2), what asset.type and asset.ean should we persist ? Should we ignore those for the moment or can we persist these values as a generic host:<id>. What if this asset is stored with the correct type from another source ? We won't be able to dedup based on ean. Note that this will also prevent us from routing the asset document to the right asset-{type} stream
• Host data can surface containers and/or k8s pod informations. Should we also create assets out of them ? Yes, we can take the same approach when looking at services in apm data. For situations where it will create duplicate and two way references, the API accounts for that
• how do we categorize hosts with cloud.provider (eg aws) but no cloud.service.name (eg ec2) ? Can we safely fallback to a default service ?

how can we rule out false positive (ie host.hostname is wrongly filled by apm php) ? We won't at first, but one potential solution is to build a blocklist of offending sources

Just to note also that the OpenTelemetry agent seems to do the same thing (the pod is reported as host.name).