Cherry-pick #20170 to 7.x: [Filebeat] Fix fileset field namespacing #20199
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
When the fields.yml file is constructed it is done by appending files together and adding some indenting. In the case of Filebeat, a fileset's fields.yml is appended with an indent of 8 spaces to the module's fields.yml. This generally allows for all of the filesets fields to become children of the module. The problem we had was that the new filesets added in elastic#19713 expected that their fields would be root fields (not children to the module namespace). In cases where the module already existed and had declared a module namespace field in its fields.yml this resulted in unexpectedly namespaced fieldset fields (e.g. microsoft.rsa.* instead of rsa.*). The size of the x-pack/filebeat index-pattern is still large (915885 bytes), but not so large that it goes beyond the Kibana request payload limit. Fixes elastic#19965 (cherry picked from commit ea7c05f)
botelastic
bot
added
the
needs_team
|
Pinging @elastic/siem (Team:SIEM) |
botelastic
bot
removed
the
needs_team
💔 Tests Failed
Expand to view the summary
Build stats
Test stats 🧪
Test errorsExpand to view the tests failures
--------------------- >> end captured stdout << ----------------------
--------------------- >> end captured stdout << ----------------------
--------------------- >> end captured stdout << ----------------------
--------------------- >> end captured stdout << ----------------------
--------------------- >> end captured stdout << ----------------------
--------------------- >> end captured stdout << ----------------------
--------------------- >> end captured stdout << ----------------------
--------------------- >> end captured stdout << ----------------------
--------------------- >> end captured stdout << ----------------------
--------------------- >> end captured stdout << ---------------------- Steps errorsExpand to view the steps failures
Log outputExpand to view the last 100 lines of log output
|
adriansr
approved these changes
Jul 23, 2020
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Cherry-pick of PR #20170 to 7.x branch. Original message:
What does this PR do?
When the fields.yml file is constructed it is done by appending files together and adding some indenting.
In the case of Filebeat, a fileset's fields.yml is appended with an indent of 8 spaces to the module's fields.yml.
This generally allows for all of the filesets fields to become children of the module.
The problem we had was that the new filesets added in #19713 expected that their fields would be root fields
(not children to the module namespace). In cases where the module already existed and had declared
a module namespace field in its fields.yml this resulted in unexpectedly namespaced fieldset fields
(e.g. microsoft.rsa.* instead of rsa.*).
The size of the x-pack/filebeat index-pattern is still large (915885 bytes), but not so large that it goes beyond the Kibana
request payload limit.
Why is it important?
Reduces the number of fields in the Filebeat mapping and allows Kibana to generate a smaller index pattern.
Checklist
CHANGELOG.next.asciidocorCHANGELOG-developer.next.asciidoc.How to test this PR locally
filebeat setupRelated issues