-
Notifications
You must be signed in to change notification settings - Fork 24.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow multiple field names/patterns for (path_)(un)match #66364
Comments
Pinging @elastic/es-search (Team:Search) |
I got here looking for a solution for the "unmatch" scenario, that is the pattern should match everything besides two or more patterns.
|
Would be great to get this moving in the context of elastic/integrations#5055. Here the specific can be found: elastic/elastic-package#1093 Having support for arrays on |
Arrays of patterns are now allowed for dynamic_templates in the match, unmatch, path_match and path_unmatch fields. DynamicTemplate has been modified to support List<String> for these fields. The patterns can be either simple wildcards or regex. As with previous functionality, mixing of wildcards and regex will not throw an error, but will not work as expected at mapping time. One new error pathway was added: if a user specifies a list of non-strings for one of these pattern fields (e.g., "match": [10, false]) a MapperParserException will be thrown. Closes elastic#66364.
…5558) * Allow multiple field names/patterns for (path_)(un)match (#66364) Arrays of patterns are now allowed for dynamic_templates in the match, unmatch, path_match and path_unmatch fields. DynamicTemplate has been modified to support List<String> for these fields. The patterns can be either simple wildcards or regex. As with previous functionality, when match_pattern="regex", simple wildcards will be flagged with an error, but when match_pattern="simple", using regular expressions in the match will not throw an error. One new error pathway was added: if a user specifies a list of non-strings for one of these pattern fields (e.g., "match": [10, false]) a MapperParserException will be thrown. A dynamic_template yamlRestTest was added. This is a BWC change, so the REST test that uses arrays of patterns is limited to v8.9 and above. Closes #66364.
Great to see this landed in Elasticsearch. @P1llus This should make it possible to simplify the ECS templates but not sure how you will deal with it as it is only available in newer versions of Elasticsearch. |
@ruflin the template that is going to be bundled with ES for logs-* should use this then, and we can handle some sort of reference in fleet, that way it can still use the template bundled with an integration for old releases. |
Use Case
I'd like to have a dynamic mapping, for different fields, that can't all be described in one pattern.
The most straightforward way of doing so would be to provide an array of fieldnames to the
match
-argument in a dynamic template, see the example.And if we're going to change this, I think it would be logical to do the same for
unmatch
andpath_match
/path_unmatch
Desired behaviour
I'd like to be be able to specify:
And have the resulting mappings for
testindex
:Current behaviour
Right now, the dynamic template is ignored. Under water, the array is converted to a string-representation, so the dynamic template actually becomes
Which will, of course, not match on any of my fields.
Workaround
It is of course possible to specify each pattern seperately, like below, but this gets large when there are more patterns.
Alternatively, a regex could match all field names (
^one.*|.*two$
), but this get also get cumbersome.The text was updated successfully, but these errors were encountered: