[Logs UI] Multi-value fields only render the first value in the log stream and details flyout

[weltenwort]

Problem description

Since #76919 migrated the data fetching to using the field parameter, the APIs for log stream entries as well as for the log entry details flyout only return the first value and drops any value beyond that.

Background

Previously, the values were read from the _source of a document, which left no room for ambiguity about whether a value is a single value or an array of values. Internally, though, Elasticsearch treats all fields as multi-value fields, even if the value array only contains one item. This is mirrored in the response obtained via the field parameter. In order to avoid changing the API the field refactoring in #76919 always extracts the first value of the field value array from the ES response. This leads to correct results for single-value fields, but ignores every value beyond the first for multi-value fields.

Possible solutions

The correct ™️ handling of single or multiple values depends on the type of the field in the mapping. For the purposes of solving this issue, it could be decomposed into two separate problems:

1. Which representation to choose in the HTTP API when transferring the log entries to the browser?
2. How to render the field values in the stream and in the log entry detail flyout?

The first problem should be easily solvable by exposing the underlying ES field semantics through the log entry API and therefore always sending an array of values.

The solution for the second problem must include a visual representation of the field values that looks reasonable for value arrays of a length of one as well as higher lengths.

Special care needs to be taken for values of "compound" fields types such as object, nested, flattened, Geo-point, Geo-shape and so on. An unsolved question so far is the flattening algorithm applied, which could broadly be one of:

1. Fully flatten all object hierarchies, thereby disregarding the semantics of the mapping.
2. Partially flatten the object hierarchy depending on the mapping type (i.e. treat object and nested types differently).

There is a related discussion taking place as part of the "Fields" project. And ultimately the Logs UI should probably follow the consensus once it is achieved.

Related links

• Project: Fields #78315
• [Data Services] Support fields parameter in high level search apis #77241
• Show runtime field values in Discover #77107
• [Discover] Experimental usage of ES fields API  #75407

[elasticmachine]

Pinging @elastic/logs-metrics-ui (Team:logs-metrics-ui)