Add ECS schema for user-agent ingest processor (#37727) #37984
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This switches the format of the user agent processor to use the schema from [ECS](https://github.com/elastic/ecs). So rather than something like this: ``` { "patch" : "3538", "major" : "70", "minor" : "0", "os" : "Mac OS X 10.14.1", "os_minor" : "14", "os_major" : "10", "name" : "Chrome", "os_name" : "Mac OS X", "device" : "Other" } ``` The structure is now like this: ``` { "name" : "Chrome", "original" : "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36", "os" : { "name" : "Mac OS X", "version" : "10.14.1", "full" : "Mac OS X 10.14.1" }, "device" : "Other", "version" : "70.0.3538.102" } ``` This is now the default for 7.0. The deprecated `ecs` setting in 6.x is not supported. Resolves elastic#37329
dakrone
added
>breaking
:Data Management/Ingest Node
|
Pinging @elastic/es-core-features |
|
jenkins retest this please |
jasontedor
added a commit
to jasontedor/elasticsearch
that referenced
this pull request
Jan 30, 2019
* master: Remove types from watcher docs (elastic#38002) Add test coverage for Painless general casting of boolean and Boolean (elastic#37780) Fixed test bug, lastFollowTime is null if there are no follower indices. Add ECS schema for user-agent ingest processor (elastic#37727) (elastic#37984) Extract TransportRequestDeduplication from ShardStateAction (elastic#37870) Expose retention leases in shard stats (elastic#37991)
5 tasks
jasontedor
added a commit
to dnhatn/elasticsearch
that referenced
this pull request
Jan 31, 2019
* elastic/master: ILM setPriority corrections for a 0 value (elastic#38001) Temporarily disable BWC for retention lease stats (elastic#38049) Skip Shrink when numberOfShards not changed (elastic#37953) Add dispatching to `HandledTransportAction` (elastic#38050) Update httpclient for JDK 11 TLS engine (elastic#37994) Reduce flaxiness of ccr recovery timeouts test (elastic#38035) Fix ILM status to allow unknown fields (elastic#38043) Fix ILM Lifecycle Policy to allow unknown fields (elastic#38041) Update verify repository to allow unknown fields (elastic#37619) [ML] Datafeed deprecation checks (elastic#38026) Deprecate minimum_master_nodes (elastic#37868) Remove types from watcher docs (elastic#38002) Add test coverage for Painless general casting of boolean and Boolean (elastic#37780) Fixed test bug, lastFollowTime is null if there are no follower indices. Add ECS schema for user-agent ingest processor (elastic#37727) (elastic#37984) Extract TransportRequestDeduplication from ShardStateAction (elastic#37870)
|
I just tried to use the changed user-agent ingest processor and realized that the |
|
Oh jeez, I made the mistake in field naming, in my initial issue 🤦♂️Sorry about that! The nesting of device.name is there to allow for other fields about the device eventually. Like device.category (tablet, mobile, etc). So the nesting is kind of important, even if there's nothing else under |
|
I created issue #38094 for this |
jakelandis
added a commit
to jakelandis/elasticsearch
that referenced
this pull request
Feb 12, 2019
elastic#37984)" This reverts commit cac6b8e. Related: elastic/beats#10650 Will replace this commit with the 6.7 version
jakelandis
added a commit
that referenced
this pull request
Feb 13, 2019
This change reverts the initial 7.0 commits and replaces them with the 6.7 variant that still allows for the ecs flag. This commit differs from the 6.7 variants in that ecs flag will now default to true. 6.7: `ecs` : default `false` 7.x: `ecs` : default `true` 8.0: no option, but behaves as `true` * Revert "Ingest node - user agent, move device to an object (#38115)" This reverts commit 5b008a3. * Revert "Add ECS schema for user-agent ingest processor (#37727) (#37984)" This reverts commit cac6b8e. * cherry-pick 5dfe193 Add ECS schema for user-agent ingest processor (#37727) * cherry-pick ec8ddc8 Ingest node - user agent, move device to an object (#38115) (#38121) * cherry-pick f63cbdb (with manual merge fixes) Dep. check for ECS changes to User Agent processor (#38362) * make true the default for the ecs option, and update 7.0 references and tests
jakelandis
added a commit
to jakelandis/elasticsearch
that referenced
this pull request
Feb 13, 2019
This change reverts the initial 7.0 commits and replaces them with the 6.7 variant that still allows for the ecs flag. This commit differs from the 6.7 variants in that ecs flag will now default to true. 6.7: `ecs` : default `false` 7.x: `ecs` : default `true` 8.0: no option, but behaves as `true` * Revert "Ingest node - user agent, move device to an object (elastic#38115)" This reverts commit 5b008a3. * Revert "Add ECS schema for user-agent ingest processor (elastic#37727) (elastic#37984)" This reverts commit cac6b8e. * cherry-pick 5dfe193 Add ECS schema for user-agent ingest processor (elastic#37727) * cherry-pick ec8ddc8 Ingest node - user agent, move device to an object (elastic#38115) (elastic#38121) * cherry-pick f63cbdb (with manual merge fixes) Dep. check for ECS changes to User Agent processor (elastic#38362) * make true the default for the ecs option, and update 7.0 references and tests
jakelandis
added a commit
that referenced
this pull request
Feb 13, 2019
Forward port of #38757 This change reverts the initial 7.0 commits and replaces them with the 6.7 variant that still allows for the ecs flag. This commit differs from the 6.7 variants in that ecs flag will now default to true. 6.7: `ecs` : default `false` 7.x: `ecs` : default `true` 8.0: no option, but behaves as `true` * Revert "Ingest node - user agent, move device to an object (#38115)" This reverts commit 5b008a3. * Revert "Add ECS schema for user-agent ingest processor (#37727) (#37984)" This reverts commit cac6b8e. * cherry-pick 5dfe193 Add ECS schema for user-agent ingest processor (#37727) * cherry-pick ec8ddc8 Ingest node - user agent, move device to an object (#38115) (#38121) * cherry-pick f63cbdb (with manual merge fixes) Dep. check for ECS changes to User Agent processor (#38362) * make true the default for the ecs option, and update 7.0 references and tests
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
(This is a forward-port with changes of #37727)
This switches the format of the user agent processor to use the schema from ECS.
So rather than something like this:
The structure is now like this:
This is now the default for 7.0. The deprecated
ecssetting in 6.x is notsupported.
Resolves #37329