-
Notifications
You must be signed in to change notification settings - Fork 24.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Better out-of-the-box mappings for logs, metrics and synthetics #64978
Merged
Merged
Changes from all commits
Commits
Show all changes
24 commits
Select commit
Hold shift + click to select a range
1100b95
Proposal: Default templates: Path match for ip and message
ruflin dae84d6
Update x-pack/plugin/core/src/main/resources/logs-mappings.json
ruflin 90ca4b1
update mapping with discussion
ruflin 35d0727
Update x-pack/plugin/core/src/main/resources/logs-mappings.json
ruflin ce72eef
Update x-pack/plugin/core/src/main/resources/logs-mappings.json
ruflin 2e130a5
Put the most specific dynamic templates first.
jpountz 7497f29
Add tests for dynamic templates.
jpountz ce711a7
add mappings to metrics and synthetics too
ruflin 30e97b3
Merge branch 'master' into path-match-ip-message
ruflin 74cb397
add index renaming
ruflin 67bed57
Merge branch 'master' into path-match-ip-message
jpountz 863bae7
Increment registry version.
jpountz 5dc16be
Trim message field from metrics and synthetics templates.
jpountz 9ea0a35
Fix tests.
jpountz 4707606
Factor conventions that are the same for all types into a single file.
jpountz d6eb0af
Merge branch 'master' into path-match-ip-message
jpountz 4c04815
Merge remote-tracking branch 'origin/master' into path-match-ip-message
jpountz c9249b5
Fix data_stream type.
jpountz db58ab9
Simplify host and observer mappings.
jpountz 9950d8b
Fix indentation.
jpountz c4cb7fd
Fix test failure.
jpountz f7376e5
Move `message` dynamic template to the shared mapping template.
jpountz 42c9f58
Remove `observer` field.
jpountz 89eb06d
Merge branch 'master' into path-match-ip-message
jpountz File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
67 changes: 67 additions & 0 deletions
67
x-pack/plugin/core/src/main/resources/data-streams-mappings.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,67 @@ | ||
{ | ||
"template": { | ||
"mappings": { | ||
"dynamic_templates": [ | ||
{ | ||
"match_ip": { | ||
"match_mapping_type": "string", | ||
"match": "ip", | ||
"mapping": { | ||
"type": "ip" | ||
} | ||
} | ||
}, | ||
{ | ||
"match_message": { | ||
"match_mapping_type": "string", | ||
"match": "message", | ||
"mapping": { | ||
"type": "match_only_text" | ||
} | ||
} | ||
}, | ||
{ | ||
"strings_as_keyword": { | ||
"mapping": { | ||
"ignore_above": 1024, | ||
"type": "keyword" | ||
}, | ||
"match_mapping_type": "string" | ||
} | ||
} | ||
], | ||
"date_detection": false, | ||
"properties": { | ||
"@timestamp": { | ||
"type": "date" | ||
}, | ||
"data_stream": { | ||
"properties": { | ||
"dataset": { | ||
"type": "constant_keyword" | ||
}, | ||
"namespace": { | ||
"type": "constant_keyword" | ||
} | ||
} | ||
}, | ||
"ecs": { | ||
"properties": { | ||
"version": { | ||
"ignore_above": 1024, | ||
"type": "keyword" | ||
} | ||
} | ||
}, | ||
"host": { | ||
"type": "object" | ||
} | ||
} | ||
} | ||
}, | ||
"_meta": { | ||
"description": "general mapping conventions for data streams", | ||
"managed": true | ||
}, | ||
"version": ${xpack.stack.template.version} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
36 changes: 0 additions & 36 deletions
36
x-pack/plugin/core/src/main/resources/metrics-mappings.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change | ||||||||
---|---|---|---|---|---|---|---|---|---|---|
|
@@ -4,6 +4,7 @@ | |||||||||
"data_stream": {}, | ||||||||||
"composed_of": [ | ||||||||||
"metrics-mappings", | ||||||||||
"data-streams-mappings", | ||||||||||
Comment on lines
6
to
+7
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Same here with:
Suggested change
|
||||||||||
"metrics-settings" | ||||||||||
], | ||||||||||
"allow_auto_create": true, | ||||||||||
|
43 changes: 0 additions & 43 deletions
43
x-pack/plugin/core/src/main/resources/synthetics-mappings.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change | ||||||||
---|---|---|---|---|---|---|---|---|---|---|
|
@@ -4,6 +4,7 @@ | |||||||||
"data_stream": {}, | ||||||||||
"composed_of": [ | ||||||||||
"synthetics-mappings", | ||||||||||
"data-streams-mappings", | ||||||||||
Comment on lines
6
to
+7
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. And same here with:
Suggested change
|
||||||||||
"synthetics-settings" | ||||||||||
], | ||||||||||
"allow_auto_create": true, | ||||||||||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should apply the
data-streams-mappings
settings first, so that any changes made to thelogs-mappings
component template always take precedence over the generic data stream mappings.What do you think?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually this would break tests because then the dynamic template that maps strings as keywords would take precedence over the dynamic template that maps message fields as
match_only_text
. In order to change the order, we would also need to configureunmatch:message
on the default dynamic template that maps strings as keywords. What is your preference?