[Discover] Implement log document context resolution

[davismcphee]

📓 Summary

The first document profile supported by One Discover will be "log". This issue covers the initial implementation of a log DocumentProfileProvider, primarily its resolve method. The aim should be to identify a log as a document type by inspecting the current DataTableRecord. Associated extension point implementations will be added later under separate issues.

Some ideas of how we might implement this:

• In ES|QL mode, records may not have all fields when using KEEP, but we should be ok initially to have those resolve to default, and instead focus on situations where we have the fields we need.

• In the current example implementation, we're just checking that the field data_stream.type === logs.

• We don't need to rely on single heuristic for the resolution process. For example, we may continue to use data_stream.type to check if a record is a log, but we can additionally use other fields or combinations of fields to identify a log as well.

• 🔗 related to: [Discover] Implement logs data source context resolution #184079

✔️ Acceptance criteria

• Define a set of heuristics to identify a record as a log.
• Create and register a log DocumentProfileProvider with a resolve method based on the defined heuristics.

[elasticmachine]

Pinging @elastic/kibana-data-discovery (Team:DataDiscovery)

[elasticmachine]

Pinging @elastic/obs-ux-logs-team (Team:obs-ux-logs)

[weltenwort]

💭 Thoughts on a log document detection heuristic

The description already mentions the most obvious field value criterion:

• data_stream.type === 'logs'

Furthermore, even documents from a mixed source still have their individual _index, which we could inspect to determine whether it matches a well-known name like

• _index starts with logstash- or filebeat-

It would also be possible to also use certain combinations of event.dataset and agent.type, such as

• event.dataset === 'apache.error'
• agent.type === 'filebeat'

In addition to the values of specific fields the mere presence of some fields would also be feasible criteria:

• @timestamp
• message
• error.message
• log.*

But except for data_stream.type they all have the risk of misclassifying the document, which happens sometimes in the old Logs UI. To me the safest criteria to start with would be

• to recognize documents in modern data streams:
  • data_stream.type === 'logs'
• to recognize documents in legacy indices
  • logs.* exists OR
  • _index starts with 'logstash-', 'filebeat-', 'winlogbeat-' or 'auditbeat-'