Kibana feature : Allow users to display a string with newlines inside the datatable

[AditiKhalatkar]

Describe the feature:
If the value of a field is stored as a string having newlines in Elasticsearch, the feature may enable the user to display the string which parses "\n" as a newline in the string. This feature would be specifically useful for someone who wants newlines in a datatable, however other visualizations may also benefit from this. Kindly consider.

Describe a specific use case for the feature:
For instance, there is a field like address, having value = "X Street \n Y Apt \n Z State", the user may want to represent this value inside the datatable as:

"X Street
Y Apt
Z State"

[elasticmachine]

Pinging @elastic/kibana-app (Team:KibanaApp)

[piehld]

I'd also like this feature, as we are currently performing the multiline consolidation process at the rsyslog stage, so incoming messages will contain the '\n' new line character.

EDIT: Please disregard my comment below--I realized the issue was due to the escaping of the new line character in the JSON message field.
In viewing the JSON formatted value of the message field (e.g., in Discover, clicking on a document -> JSON), the field I wanted to display as multi-line (in my case, 'message') contained \\n instead of just \n. So, I added a 'gsub' processor to my ingest pipeline to perform the conversion and it displays properly now!

For those who might benefit from this experience, this is what my message field looked like with problematic escaped new line characters (viewing it in JSON view):
"[2020-10-02 08:56:11 -0700] [25076] [INFO] [descriptorMatch.matchGetQuery] Got <DescriptorType.inchi: 'InChI'> 'RHFUOMFWUGWKKO-XVFCMESISA-N' 'graph-relaxed'\\nWarning: OEParseInChI failed on 'RHFUOMFWUGWKKO-XVFCMESISA-N'"

Confusingly however, the representation of the new line character in the original log message during ingestion was \\\\n, so I converted it to \n using the following the gsub processor:
{ "gsub": { "field": "message", "pattern": "\\\\n", "replacement": "\n" } }

OLD COMMENT - DISREGARD
I was surprised to find out that ingesting these lines into Elasticsearch (via Filebeat) did not render them as new lines in Kibana, as the use of the 'multiline' option in Filebeat (for a separate application) DOES render the new line characters into new lines in discover. So, I'm curious to know what additional action the 'multiline' option in Filebeat is doing to enable those new line characters to be rendered in Kibana, versus the case of ingesting a string containing new line characters.

[piehld]

@AditiKhalatkar I'm not sure if my experience is the same as yours, but might be worth checking if you haven't already.

[srothkin]

We need this feature too.

[ruudkenter]

We need this feature as well

[flash1293]

@stratoula Will this be possible once dynamic row height lands?

[stratoula]

I am not 100% sure, I want to test it first. We definitely need to add the dynamic heights to support this. I am not sure if it will be supported without any change from our side. I will check it and update the issue :)

[stratoula]

@flash1293 with the new auto-height we can have this result

but with double \n. For example the above screenshot is generated for this text: X Street \n\n Y Apt \n\n Z State

With a single \nI get them in the same line:

[elasticmachine]

Pinging @elastic/kibana-data-discovery (Team:DataDiscovery)

[ghudgins]

whatever we come up with for this in vis editors we should also do for Discover as well. Legacy table has this feature CC @VijayDoshi

[VijayDoshi]

Do we have a path forward on this? Seems like we are all in agreement that we should respect line breaks.

[flash1293]

@kertal @stratoula It seems to me like we can simply add this behavior if the user has set an "auto" or multi line row height by replacing newlines by breaks for both Discover and Lens, what do you think?

[stratoula]

Yes I was thinking the same ++

[kertal]

Sure thing

[flash1293]

Do this automatically if row auto height is enabled (not supported for headers, only data rows)

[flavono123]

Where is the config in Kibana or Helm values?
Cannot find in Stack Management > Advanced Settings.

[stratoula]

@flavono123 they are on Lens editor,

[flavono123]

@stratoula So speedy !
May I configure on the "Discover", not Lens?

Kibana version is 7.13.1 and cannot find similar one from there.
I just upgraded the version to 7.17.3, stil cannot find, however.

[flavono123]

Oh, newlines work in Discover, never mind.

I did not selected, clicking the + button of the field("log").

Thank you for your very very fast response btw! @stratoula

[stratoula]

Nice! It also exists in Discover btw but I am not pretty sure in which version it was added.