add data_stream_template_use_index_patterns_wildcard in elasticsearch_data_stream

[bgruszka]

Add data_stream_template_use_index_patterns_wildcard config param in elasticsearch_data_stream

Specify whether index patterns should include a wildcard (*) when creating an index template. This is particularly useful to prevent errors in scenarios where index templates are generated automatically, and multiple services with distinct suffixes are in use.

Default value is true.

Consider the following JSON error response when index patterns clash due to wildcard usage:

{
  "error": {
    "root_cause": [
      {
        "type": "illegal_argument_exception",
        "reason": "index template [eks-kube-apiserver] has index patterns [eks-kube-apiserver*] matching patterns from existing templates [eks-kube-apiserver-audit] with patterns (eks-kube-apiserver-audit => [eks-kube-apiserver-audit*]) that have the same priority [0], multiple index templates may not match during index creation, please use a different priority"
      }
    ],
    "type": "illegal_argument_exception",
    "reason": "index template [eks-kube-apiserver] has index patterns [eks-kube-apiserver*] matching patterns from existing templates [eks-kube-apiserver-audit] with patterns (eks-kube-apiserver-audit => [eks-kube-apiserver-audit*]) that have the same priority [0], multiple index templates may not match during index creation, please use a different priority"
  },
  "status": 400
}

Usage Examples

When data_stream_template_use_index_patterns_wildcard is set to true (default):

data_stream_name: foo
data_stream_template_use_index_patterns_wildcard: true

In this case, the resulting index patterns will be: ["foo*"]

When data_stream_template_use_index_patterns_wildcard is set to false:

data_stream_name: foo
data_stream_template_use_index_patterns_wildcard: false

The resulting index patterns will be: ["foo"]

(check all that apply)

• tests added
• tests passing
• README updated (if needed)
• README Table of Contents updated (if needed)
• History.md and version in gemspec are untouched
• backward compatible
• feature works in elasticsearch_dynamic (not required but recommended)

[cosmo0920]

I read your patch and understand that this PR is useful for some of the use cases.
And thank you for adding the documentation. It's quite useful for users 😄

[bgruszka]

@cosmo0920 thank you :) can you release a new tag and new gem version? :)

[cosmo0920]

@kenhys Do you have a cycle to handle this?

[bgruszka]

@cosmo0920 @kenhys ping :)

[bgruszka]

@cosmo0920 @kenhys Can someone release this? We really need it in our company, but unfortunately, I don't have the necessary permissions. If I had them, I would have already taken care of it...

[bgruszka]

ping

[cosmo0920]

This patch is released as v5.4.3. Sorry for delaying to publish.

[bgruszka]

thank you 🙂