Automatic Index alias and template creation on ES output for ISM usage

[tingeltangelthomas]

We are using EKS + elasticsearch 7.10 running on AWS opensearch.
We would like to use a more flexible way to create our indices automatically by fluentbit.
Based on the value of a kubernetes label an index (<label-value>-000001), an index alias (<label-value>) and an index template (with <label-value> as rollover alias) should be created.
We do not want to add any date information to the index name. The indices will be rolled over via ISM.
Fluentbit should send log data to the created index alias (<label-value>), which always points to the current write-index.

This should be a fully automated process.

I found a possibility to do that with fluentd, but not with fluentbit:

        # ${tag} = <label-value>

	rollover_index true
	deflector_alias "${tag}"
	application_name index

	index_name "${tag}"
	index_date_pattern ""

	template_name      "fluentd-template-${tag}"
	template_file      /etc/fluent/config.d/index-template.json
	customize_template {"<<TAG>>":"${tag}"}
	template_overwrite true

I saw couple of issues but not a real automated solution.
e. g. #1670 (comment)

Is there any way to accomplish this with fluentbit?

Thank you for your help!

Regards
Thomas

[taleb1994]

No answer yet?