Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merge custom and core multi_fields array #982
Merge custom and core multi_fields array #982
Changes from 4 commits
eae6bf5
9898651
f0746ae
d1953cf
2ca5584
23cc12c
cb0e5b9
6eb01c0
File filter
Filter by extension
Conversations
Jump to
There are no files selected for viewing
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Minor issue I stumbled across while testing this out. Not sure it would be a blocker to merging, but worth noting the behavior.
The union will remove exact duplicate items:
But if the sets are not exact duplicates, it could lead to duplicate field names:
schema include file:
Resulting intermediate state:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh good catch, what do we think the expected behavior should be in this scenario? I could put in a check to ensure that two of the same
name
fields don't exist in the resulting set and throw an error if they do? Or maybe just have core override?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
IMO we should dedupe on
name
and take the most recent definition in the case of dupes (this would allow for overrides).There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@webmat do you have any thoughts? I recall back in #864, logic was removed from the tooling to allow
--include
supplied custom fields to be more permissive:Perhaps we simply make sure to note that users need to be aware of introducing such duplicates fields?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree with @madirey. We should keep it simple and only ensure we have unique multi-field names.
The
--include
option is meant to override, so the ideal behaviour is for a custom multi-field definition to replace or be merged with an entry of the same name. I'm on the fence on whether to merge/replace an entry of the same name, though. Happy to be convinced either way.But to take a concrete example, let's say someone has tuned a normalizer that works well for user agent strings, I want them to be able to replace the default
user_agent.original.text
multi-field with such a custom definition:I think I have a preference with merging the pre-existing multi-field definitions of the same name, as this is more in line with how everything else is handled with custom fields. And it has the bonus of allowing a more terse custom definition: