[Kibana] Track TSDB migration

[constanca-m]

This issue tracks the TSDB migration for Kibana integration. For more information about TSDB, please refer to this page.

• [kibana] only allow single value in service.address beats#37167
• Set fields as dimensions: [Kibana] Set dimension fields for TSDB migration #7025
  • There are no documents being overwritten
• Set metric type for metric fields:
  • Dashboards work as expected
• Enable TSDB by default:

[mlunadia]

@lalit-satapathy This package has no description of any of the fields therefore it cannot be migrated and is also blocked by elastic/beats#37167
@klacabane we are now winding down the TSDB migration project which means this package might have to stay without migrations as we have been waiting for a while to get the descriptions in place and for elastic/beats#37167 to be solved.
Can you please confirm if this is happening soon or whether this package is likely to be left out?

[klacabane]

we have been waiting for a while to get the descriptions in place

Can you help me understand what's required for that ?
I will take care of elastic/beats#37167 with 8.12.0 as target release

[mlunadia]

IIUC the fields on the datastream have no descriptions which blocks us from making an informed migration to TSDB. @constanca-m can you keep me honest on this one?

[constanca-m]

For the dimensions issue, indeed we need to fix this issue first, otherwise we will keep loosing data if we enable TSDB.

For the metric type issue, if I take this data stream as an example, the metrics have no description. As an outsider, I cannot really understand if I am working with counter or gauge. That is the main problem for setting the metrics.