support multiple indices for ilm

[simitt]

Introduce a top level key indices that combines information for template, ilm, index name and index condition.

Deprecate usage of
output.elasticsearch.index and output.elasticsearch.indices.*
setup.template.*

Remove usage of (as it was only in beta):
ilm.*

This requires allowing one template per index entry (reuse code from #9247).

#9616 refactors current ILM handling and introduces multiple indices for ilm handling, but nested within the current config yml structure. Reuse code where possible.

Open Points to address:

• How do we deal with Kibana index patterns? Do we need one Kibana index pattern per ES template?
• Clarify how to retrieve the fields per template. Should it be allowed to set a module name for which the fields are retrieved? Ensure current field creation procedure allows creating multiple fields files and merges common_fields properly.
• Check if Central Config Management can interfere with indices settings and ILM.
• Make <beatname> setup work if output is not Elasticsearch
• Adapt beats to use new config options by default and remove deprecated options from reference files.

[nerddelphi]

Is there any update?

[simitt]

There are no immediate plans for supporting this directly in beats. Some internal refactoring allowed for implementing multi-index-support for ILM required by APM directly in the apm-server code base. Closing this for now to avoid confusion.

[TomaszKlosinski]

Any news on this? Replacing output.elasticsearch.indices with templating conditions would be really useful in ILM.