-
Notifications
You must be signed in to change notification settings - Fork 8.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SIEM] Add apm-*-transcation* to siemDefaultIndex Kibana Advanced Setting #52297
Comments
Pinging @elastic/siem (Team:SIEM) |
cc @elastic/apm |
fyi: Users can configure which index apm is written to. There are config options in kibana.dev.yml for this: apm_oss.errorIndices: apm-*-error*
apm_oss.transactionIndices: apm-*-transaction*
apm_oss.spanIndices: apm-*-span*
apm_oss.metricsIndices: apm-*-metric* We've recently introduced the possibility of overwriting these settings via the UI and thus stored in saved objects - which could make this a little more complicated for you. LMK if you need further details. |
Thanks for the details @sqren -- the question of additional apm indices came up when we were discussing this, so I'll touch base with the team and detail feedback here before moving forward. |
The question is if we add |
Sounds good! Will move forward with adding In effort to ensure the SIEM Map displays APM data without any additional configuration (since it requires a matching named Kibana Index Pattern), I'm going to make a slight enhancement to how the map layers are generated. When the map is checking for matching Kibana Index Patterns, it'll match The maps config generation is fairly well abstracted, so this'll be a pretty simple change and something we can expand upon further if/when we decide to add any other indices. |
## Summary Resolves #52297, #52565 To improve the display of APM data within SIEM (specifically the `HTTP Table` and `Network Map`), this PR adds `apm-*-transcation*` to `siem:defaultIndex`, and additional support for showing `client`/`server` layers on the `Network Map` when a matching `apm-*` index pattern is present. The map now supports pattern matching when checking for available Kibana Index Patterns, and so matches `apm-*-transcation*` -> `apm-*` (if exists). Additionally, the map config was updated to generate layers for client/server geo fields (instead of the usual source/dest) since these are the fields Transactions use. ![image](https://user-images.githubusercontent.com/2946766/72573225-2a038880-3882-11ea-9590-a545d726dbf9.png) <img width="1214" alt="Screen Shot 2020-01-14 at 18 22 11" src="https://user-images.githubusercontent.com/2946766/72407120-bcd5e300-371b-11ea-90cc-a0714320a59c.png"> ### Checklist Use ~~strikethroughs~~ to remove checklist items you don't feel are applicable to this PR. - [ ] ~This was checked for cross-browser compatibility, [including a check against IE11](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility)~ - [X] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md) - [ ] [Documentation](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#writing-documentation) was added for features that require explanation or tutorials - Will work with @benskelker on updating the maps docs - [X] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios - [ ] ~This was checked for [keyboard-only and screenreader accessibility](https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Accessibility#Accessibility_testing_checklist)~ ### For maintainers - [ ] ~This was checked for breaking API changes and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~ - [ ] ~This includes a feature addition or change that requires a release note and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~
## Summary Resolves elastic#52297, elastic#52565 To improve the display of APM data within SIEM (specifically the `HTTP Table` and `Network Map`), this PR adds `apm-*-transcation*` to `siem:defaultIndex`, and additional support for showing `client`/`server` layers on the `Network Map` when a matching `apm-*` index pattern is present. The map now supports pattern matching when checking for available Kibana Index Patterns, and so matches `apm-*-transcation*` -> `apm-*` (if exists). Additionally, the map config was updated to generate layers for client/server geo fields (instead of the usual source/dest) since these are the fields Transactions use. ![image](https://user-images.githubusercontent.com/2946766/72573225-2a038880-3882-11ea-9590-a545d726dbf9.png) <img width="1214" alt="Screen Shot 2020-01-14 at 18 22 11" src="https://user-images.githubusercontent.com/2946766/72407120-bcd5e300-371b-11ea-90cc-a0714320a59c.png"> ### Checklist Use ~~strikethroughs~~ to remove checklist items you don't feel are applicable to this PR. - [ ] ~This was checked for cross-browser compatibility, [including a check against IE11](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility)~ - [X] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md) - [ ] [Documentation](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#writing-documentation) was added for features that require explanation or tutorials - Will work with @benskelker on updating the maps docs - [X] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios - [ ] ~This was checked for [keyboard-only and screenreader accessibility](https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Accessibility#Accessibility_testing_checklist)~ ### For maintainers - [ ] ~This was checked for breaking API changes and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~ - [ ] ~This includes a feature addition or change that requires a release note and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~
## Summary Resolves elastic#52297, elastic#52565 To improve the display of APM data within SIEM (specifically the `HTTP Table` and `Network Map`), this PR adds `apm-*-transcation*` to `siem:defaultIndex`, and additional support for showing `client`/`server` layers on the `Network Map` when a matching `apm-*` index pattern is present. The map now supports pattern matching when checking for available Kibana Index Patterns, and so matches `apm-*-transcation*` -> `apm-*` (if exists). Additionally, the map config was updated to generate layers for client/server geo fields (instead of the usual source/dest) since these are the fields Transactions use. ![image](https://user-images.githubusercontent.com/2946766/72573225-2a038880-3882-11ea-9590-a545d726dbf9.png) <img width="1214" alt="Screen Shot 2020-01-14 at 18 22 11" src="https://user-images.githubusercontent.com/2946766/72407120-bcd5e300-371b-11ea-90cc-a0714320a59c.png"> ### Checklist Use ~~strikethroughs~~ to remove checklist items you don't feel are applicable to this PR. - [ ] ~This was checked for cross-browser compatibility, [including a check against IE11](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility)~ - [X] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md) - [ ] [Documentation](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#writing-documentation) was added for features that require explanation or tutorials - Will work with @benskelker on updating the maps docs - [X] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios - [ ] ~This was checked for [keyboard-only and screenreader accessibility](https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Accessibility#Accessibility_testing_checklist)~ ### For maintainers - [ ] ~This was checked for breaking API changes and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~ - [ ] ~This includes a feature addition or change that requires a release note and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~
## Summary Resolves #52297, #52565 To improve the display of APM data within SIEM (specifically the `HTTP Table` and `Network Map`), this PR adds `apm-*-transcation*` to `siem:defaultIndex`, and additional support for showing `client`/`server` layers on the `Network Map` when a matching `apm-*` index pattern is present. The map now supports pattern matching when checking for available Kibana Index Patterns, and so matches `apm-*-transcation*` -> `apm-*` (if exists). Additionally, the map config was updated to generate layers for client/server geo fields (instead of the usual source/dest) since these are the fields Transactions use. ![image](https://user-images.githubusercontent.com/2946766/72573225-2a038880-3882-11ea-9590-a545d726dbf9.png) <img width="1214" alt="Screen Shot 2020-01-14 at 18 22 11" src="https://user-images.githubusercontent.com/2946766/72407120-bcd5e300-371b-11ea-90cc-a0714320a59c.png"> ### Checklist Use ~~strikethroughs~~ to remove checklist items you don't feel are applicable to this PR. - [ ] ~This was checked for cross-browser compatibility, [including a check against IE11](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility)~ - [X] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md) - [ ] [Documentation](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#writing-documentation) was added for features that require explanation or tutorials - Will work with @benskelker on updating the maps docs - [X] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios - [ ] ~This was checked for [keyboard-only and screenreader accessibility](https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Accessibility#Accessibility_testing_checklist)~ ### For maintainers - [ ] ~This was checked for breaking API changes and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~ - [ ] ~This includes a feature addition or change that requires a release note and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~
## Summary Resolves #52297, #52565 To improve the display of APM data within SIEM (specifically the `HTTP Table` and `Network Map`), this PR adds `apm-*-transcation*` to `siem:defaultIndex`, and additional support for showing `client`/`server` layers on the `Network Map` when a matching `apm-*` index pattern is present. The map now supports pattern matching when checking for available Kibana Index Patterns, and so matches `apm-*-transcation*` -> `apm-*` (if exists). Additionally, the map config was updated to generate layers for client/server geo fields (instead of the usual source/dest) since these are the fields Transactions use. ![image](https://user-images.githubusercontent.com/2946766/72573225-2a038880-3882-11ea-9590-a545d726dbf9.png) <img width="1214" alt="Screen Shot 2020-01-14 at 18 22 11" src="https://user-images.githubusercontent.com/2946766/72407120-bcd5e300-371b-11ea-90cc-a0714320a59c.png"> ### Checklist Use ~~strikethroughs~~ to remove checklist items you don't feel are applicable to this PR. - [ ] ~This was checked for cross-browser compatibility, [including a check against IE11](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility)~ - [X] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md) - [ ] [Documentation](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#writing-documentation) was added for features that require explanation or tutorials - Will work with @benskelker on updating the maps docs - [X] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios - [ ] ~This was checked for [keyboard-only and screenreader accessibility](https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Accessibility#Accessibility_testing_checklist)~ ### For maintainers - [ ] ~This was checked for breaking API changes and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~ - [ ] ~This includes a feature addition or change that requires a release note and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~
## Summary Resolves elastic#52297, elastic#52565 To improve the display of APM data within SIEM (specifically the `HTTP Table` and `Network Map`), this PR adds `apm-*-transcation*` to `siem:defaultIndex`, and additional support for showing `client`/`server` layers on the `Network Map` when a matching `apm-*` index pattern is present. The map now supports pattern matching when checking for available Kibana Index Patterns, and so matches `apm-*-transcation*` -> `apm-*` (if exists). Additionally, the map config was updated to generate layers for client/server geo fields (instead of the usual source/dest) since these are the fields Transactions use. ![image](https://user-images.githubusercontent.com/2946766/72573225-2a038880-3882-11ea-9590-a545d726dbf9.png) <img width="1214" alt="Screen Shot 2020-01-14 at 18 22 11" src="https://user-images.githubusercontent.com/2946766/72407120-bcd5e300-371b-11ea-90cc-a0714320a59c.png"> ### Checklist Use ~~strikethroughs~~ to remove checklist items you don't feel are applicable to this PR. - [ ] ~This was checked for cross-browser compatibility, [including a check against IE11](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility)~ - [X] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md) - [ ] [Documentation](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#writing-documentation) was added for features that require explanation or tutorials - Will work with @benskelker on updating the maps docs - [X] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios - [ ] ~This was checked for [keyboard-only and screenreader accessibility](https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Accessibility#Accessibility_testing_checklist)~ ### For maintainers - [ ] ~This was checked for breaking API changes and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~ - [ ] ~This includes a feature addition or change that requires a release note and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~
In effort to augment the SIEM UI with APM data we'll need to include
apm-*
in thesiem:defaultIndex
Kibana Advanced Setting.10-DEC-2019: Editing title to be
apm-*-transcation*
instead ofapm-*
The text was updated successfully, but these errors were encountered: