Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

User/Group Management Dashboards #15236

Merged
merged 3 commits into from
Feb 5, 2020
Merged

User/Group Management Dashboards #15236

merged 3 commits into from
Feb 5, 2020

Conversation

janniten
Copy link
Contributor

User and Group Management Events Dashboards

Screenshot_2019-12-20  winlogbeat  User Management Events - Kibana
Screenshot_2019-12-20  winlogbeat  Group Management Events - Kibana

@janniten janniten requested a review from a team as a code owner December 20, 2019 12:44
@elasticmachine
Copy link
Collaborator

Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually?

1 similar comment
@elasticmachine
Copy link
Collaborator

Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually?

@janniten
Copy link
Contributor Author

@leehinman I have created a new PR only for dashboards of user and group events

@leehinman leehinman self-assigned this Dec 20, 2019
@elasticmachine
Copy link
Collaborator

Pinging @elastic/siem (Team:SIEM)

Add dashboards to the Winlogbeat Security module for visualizing User Management events
and Group Management events. There are two versions of each dashboard - one with and without
TSVB (time series visual builder) visualizations.

This updates the Winlogbeat build to include the dashboards from the module directories.

Add it adds winlog.event_data.MemberName to the fields.yml because it's used in the user
management dashboard and should be in the fields.yml so the dashboards load without error.
@andrewkroh
Copy link
Member

jenkins, test this

@andrewkroh
Copy link
Member

I fixed up the dashboards. The only issue with them was that the format was slightly different than what is expected for the dashboards that are checked into the repo. The ones in the repo get decoded to make diffing a little easier, then are encoded again when released in packages. I think there is a --decode flag than can be set to do this like .\winlogbeat.exe export dashboard --decode --id=<uuid>.

@andrewkroh
Copy link
Member

Looks like we hit a few linter issues. I'll try to fix them:

>> check: Checking source code for common problems
>> Dashboard format - module/security/_meta/kibana/7/dashboard/01c54730-fee6-11e9-8405-516218e3d268.json:
   expected title with format '[Winlogbeat Module] Some title', found '[Winlogbeat] Group Management Events - Simple Metrics': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Group Management Events - Description': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Group Management Events - Event Actions - Donut': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Group Management Events - Event Actions - Table': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Group Management Events - Groups vs Actions - Heatmap': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Groups Created - Table': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Group Changes - Table': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Groups Deleted - Table': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Users Added - Table': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Users Removed from Group - Table': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found '5_Group_Enum': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Users Added - Simple Metric': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Groups Deleted- Simple Metric': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Groups Created - Simple Metric': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Groups Changes - Simple Metric': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Users Removed from Group  - Simple Metric': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Group Membership Enumeration  - Simple Metric': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Group Management Action Distribution over Time': title doesn't match pattern
>> Dashboard format - module/security/_meta/kibana/7/dashboard/71f720f0-ff18-11e9-8405-516218e3d268.json:
   expected title with format '[Winlogbeat Module] Some title', found '[Winlogbeat] User Management Events': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'User  Management Events - Description': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'User Management Events - Event Actions - Donut': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Users Created - Table': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'User Management Events - Event Actions - Table': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Users Enabled - Table': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Users Disabled - Table': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Users Deleted - Table': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'User Management Events - Affected Users vs Actions - Heatmap': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Users Password Changes - Table': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Unlocked Users - Table': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Users Changes  Table': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Users Locked Out - Table': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found '4_Event_Distribution in time': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Users Disabled - VB Metric': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Users Enabled - VB Metric': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Users Deleted - VB Metric': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Users Created - VB Metric': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Users Unlocks - VB Metric': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Users Password Changes - VB Metric': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Users locked Out - VB Metric': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Users Changes  VB Metric': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Users Renamed VB Metric': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Users Renamed - Table': title doesn't match pattern
>> Dashboard format - module/security/_meta/kibana/7/dashboard/8223bed0-b9e9-11e9-b6a2-c9b4015c4baf.json:
   expected title with format '[Winlogbeat Module] Some title', found '[Winlogbeat] User Management Events - Simple Metric': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'User  Management Events - Description': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'User Management Events - Event Actions - Donut': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Users Created - Table': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'User Management Events - Event Actions - Table': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Users Enabled - Table': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Users Disabled - Table': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Users Deleted - Table': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'User Management Events - Affected Users vs Actions - Heatmap': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Users Password Changes - Table': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Users Created - Simple Metric': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Users Enabled - Simple Metric': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Users Deleted - Simple Metric': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Users Disabled - Simple Metric': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Users Password Reset / Changes': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Unlocked Users - Table': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Users Changes  Table': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Users Unlocks - Simple Metric': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Unlocked Users - Simple Metric': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Users Changes - Simple Metric': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Users Locked Out - Table': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found '4_Event_Distribution in time': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Users Renamed - Simple Metric': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Users Renamed - Table': title doesn't match pattern
>> Dashboard format - module/security/_meta/kibana/7/dashboard/bb858830-f412-11e9-8405-516218e3d268.json:
   expected title with format '[Winlogbeat Module] Some title', found '[Winlogbeat] Group Management Events': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Group Management Events - Description': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Group Management Events - Event Actions - Donut': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Group Management Events - Event Actions - Table': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Users Added - Metric': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Groups Deleted - Metric': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Groups Created - Metric': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Groups Changed - Metric': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Users Removed - Table': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Groups Enumeration - Metric': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Group Management Events - Groups vs Actions - Heatmap': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Groups Created - Table': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Group Changes - Table': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Groups Deleted - Table': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Users Added - Table': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Users Removed from Group - Table': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found '5_Group_Enum': title doesn't match pattern
   expected title with format 'Some title [Winlogbeat Module]', found 'Group Management Action Distribution over Time': title doesn't match pattern
Error: there are format errors in dashboards

@andrewkroh
Copy link
Member

jenkins, test this

@andrewkroh andrewkroh merged commit d866824 into elastic:master Feb 5, 2020
@andrewkroh andrewkroh added the needs_backport PR is waiting to be backported to other branches. label Feb 5, 2020
andrewkroh pushed a commit to andrewkroh/beats that referenced this pull request Mar 11, 2020
Add dashboards to the Winlogbeat Security module for visualizing User Management events
and Group Management events. There are two versions of each dashboard - one with and without TSVB (time series visual builder) visualizations.

This updates the Winlogbeat build to include the dashboards from the module directories.

Add it adds winlog.event_data.MemberName to the fields.yml because it's used in the user
management dashboard and should be in the fields.yml so the dashboards load without error.

Co-authored-by: Andrew Kroh <[email protected]>
(cherry picked from commit d866824)
@andrewkroh andrewkroh added v7.7.0 and removed needs_backport PR is waiting to be backported to other branches. labels Mar 11, 2020
andrewkroh added a commit that referenced this pull request Mar 13, 2020
)

Add dashboards to the Winlogbeat Security module for visualizing User Management events
and Group Management events. There are two versions of each dashboard - one with and without TSVB (time series visual builder) visualizations.

This updates the Winlogbeat build to include the dashboards from the module directories.

Add it adds winlog.event_data.MemberName to the fields.yml because it's used in the user
management dashboard and should be in the fields.yml so the dashboards load without error.

Co-authored-by: Andrew Kroh <[email protected]>
(cherry picked from commit d866824)

Co-authored-by: Anabella Cristaldi <[email protected]>
"id": "4",
"params": {
"customLabel": "Performer LogonID",
"field": "winlog.logon.id",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It appears a user from the community (Using a fresh 7.8.0 stack) noticed that winlog.logon.id does not exist in the winlogbeat-* index pattern from the setup which yields in an error message "Could not locate that index-pattern-field (id: winlog.logon.id)" when trying to edit/look at some of the new visualizations such as "Users Deleted - Table [Winlogbeat Security]"

Not sure if I follow but it looks like the index pattern is "generated" from the fields.yml and if that is true I can see why this field does not get added to the winlogbeat-* index pattern.

The fields.common.yml is missing winlog.logon.id so maybe that is why the default index pattern that gets generated may not have this field?

Slack thread here: https://elasticstack.slack.com/archives/CNEDGGJQ3/p1595052755138500

Maybe this is a case of Dashboards/Visualizations using a custom index pattern that isn't fully compliant with the default winlogbeat-* index pattern created from the setup?

If users don't get all of the event logs that would contain these fields then the index pattern would never contain this field, so maybe it is worth adding into the fields.common.yml? I just wanted to make a note of this analysis.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @nicpenning,
I don't know exactly how the winlogbeat index pattern is generated, but probably I'll need to add the field winlog.logon.id in the security module specific fields.yml file (beats\x-pack\winlogbeat\module\security_meta\fields.yml)
@andrewkroh, what do you think? Should I add this field here or in any other place?
In addition. Which is the criteria in order to add a field in the yaml? The winlog.logon.id is not the only field I've added; I can add the necessary fields.
Thank you!
Regards

efd6 added a commit to efd6/integrations that referenced this pull request Mar 20, 2023
The dashboards were imported together from beats where they co-existed
after the second sets addition. The changes that added them to beats
appear to have been:

- elastic/beats#18775
- elastic/beats#15236
efd6 added a commit to elastic/integrations that referenced this pull request Mar 20, 2023
The dashboards were imported together from beats where they co-existed
after the second sets addition. The changes that added them to beats
appear to have been:

- elastic/beats#18775
- elastic/beats#15236
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants