-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add fileset for parsing linux auditd logs #3750
Merged
Merged
Changes from 2 commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,20 +1,28 @@ | ||
== System module | ||
|
||
This module collects and parses logs created by system logging server of common Unix/Linux based | ||
distributions. | ||
This module collects and parses logs created by system logging server of common | ||
Unix/Linux based distributions. | ||
|
||
[float] | ||
=== Compatibility | ||
|
||
This module was tested with logs from OSes like Ubuntu 12.04, Centos 7, macOS Sierra, and others. | ||
This module was tested with logs from OSes like Ubuntu 12.04, Centos 7, macOS | ||
Sierra, and others. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The "and others" might come back to us as an issue in support, maybe it's better to leave it out. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I agree, probably a good idea to drop that. Will do. |
||
|
||
This module is not available for Windows. | ||
|
||
[float] | ||
=== Dashboard | ||
|
||
This module comes with a sample dashboard. | ||
This module comes with sample dashboards. The first is a more generic syslog | ||
dashboard that shows syslog data. | ||
|
||
image::./images/kibana-system.png[] | ||
|
||
And second is the audit dashboard that shows audit log data. | ||
|
||
image::./images/kibana-system-audit.png[] | ||
|
||
[float] | ||
=== Syslog fileset settings | ||
|
||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,60 @@ | ||
- name: audit | ||
type: group | ||
description: > | ||
Fields from the Linux audit logs. Not all fields are documented here because | ||
they are dynamic and vary by audit event type. | ||
fields: | ||
- name: type | ||
description: > | ||
The audit event type. | ||
- name: epoch | ||
description: > | ||
The unix epoch timestamp from the audit log. | ||
- name: counter | ||
type: long | ||
description: > | ||
The audit event counter. | ||
- name: pid | ||
type: long | ||
description: > | ||
The ID of the process. | ||
- name: ppid | ||
type: long | ||
description: > | ||
The ID of the process. | ||
- name: items | ||
type: long | ||
description: > | ||
The number of items in an event. | ||
- name: item | ||
type: long | ||
description: > | ||
The item field indicates which item out of the total number of items. | ||
This number is zero-based; a value of 0 means it is the first item. | ||
- name: geoip | ||
type: group | ||
description: > | ||
Contains GeoIP information gathered based on the `system.audit.addr` | ||
field. Only present if the GeoIP Elasticsearch plugin is available and | ||
used. | ||
fields: | ||
- name: continent_name | ||
type: keyword | ||
description: > | ||
The name of the continent. | ||
- name: city_name | ||
type: keyword | ||
description: > | ||
The name of the city. | ||
- name: region_name | ||
type: keyword | ||
description: > | ||
The name of the region. | ||
- name: country_iso_code | ||
type: keyword | ||
description: > | ||
Country ISO code. | ||
- name: location | ||
type: geo_point | ||
description: > | ||
The longitude and latitude. |
13 changes: 13 additions & 0 deletions
13
...beat/module/system/audit/_meta/kibana/dashboard/dfbb49f0-0a0f-11e7-8a62-2d05eaaac5cb.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
{ | ||
"hits": 0, | ||
"timeRestore": false, | ||
"description": "", | ||
"title": "Filebeat System Audit", | ||
"uiStateJSON": "{\"P-2\":{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}}}", | ||
"panelsJSON": "[{\"col\":1,\"id\":\"6295bdd0-0a0e-11e7-825f-6748cda7d858\",\"panelIndex\":1,\"row\":1,\"size_x\":4,\"size_y\":4,\"type\":\"visualization\"},{\"col\":9,\"id\":\"5ebdbe50-0a0f-11e7-825f-6748cda7d858\",\"panelIndex\":2,\"row\":1,\"size_x\":4,\"size_y\":4,\"type\":\"visualization\"},{\"col\":1,\"id\":\"2bb0fa70-0a11-11e7-9e84-43da493ad0c7\",\"panelIndex\":3,\"row\":5,\"size_x\":6,\"size_y\":3,\"type\":\"visualization\"},{\"col\":1,\"columns\":[\"message\"],\"id\":\"4ac0a370-0a11-11e7-8b04-eb22a5669f27\",\"panelIndex\":4,\"row\":8,\"size_x\":12,\"size_y\":3,\"sort\":[\"@timestamp\",\"desc\"],\"type\":\"search\"},{\"col\":7,\"id\":\"d1726930-0a7f-11e7-8b04-eb22a5669f27\",\"panelIndex\":5,\"row\":5,\"size_x\":6,\"size_y\":3,\"type\":\"visualization\"},{\"size_x\":4,\"size_y\":4,\"panelIndex\":6,\"type\":\"visualization\",\"id\":\"c5411910-0a87-11e7-8b04-eb22a5669f27\",\"col\":5,\"row\":1}]", | ||
"optionsJSON": "{\"darkTheme\":false}", | ||
"version": 1, | ||
"kibanaSavedObjectMeta": { | ||
"searchSourceJSON": "{\"filter\":[{\"query\":{\"query_string\":{\"analyze_wildcard\":true,\"query\":\"*\"}}}],\"highlightAll\":true,\"version\":true}" | ||
} | ||
} |
16 changes: 16 additions & 0 deletions
16
filebeat/module/system/audit/_meta/kibana/search/4ac0a370-0a11-11e7-8b04-eb22a5669f27.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
{ | ||
"sort": [ | ||
"@timestamp", | ||
"desc" | ||
], | ||
"hits": 0, | ||
"description": "", | ||
"title": "Audit Events", | ||
"version": 1, | ||
"kibanaSavedObjectMeta": { | ||
"searchSourceJSON": "{\"index\":\"filebeat-*\",\"highlightAll\":true,\"version\":true,\"query\":{\"query_string\":{\"query\":\"_exists_:system.audit\",\"analyze_wildcard\":true}},\"filter\":[]}" | ||
}, | ||
"columns": [ | ||
"message" | ||
] | ||
} |
10 changes: 10 additions & 0 deletions
10
.../module/system/audit/_meta/kibana/visualization/2bb0fa70-0a11-11e7-9e84-43da493ad0c7.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
{ | ||
"visState": "{\"type\":\"timelion\",\"title\":\"Audit Event Results\",\"params\":{\"expression\":\".es(q=\\\"_exists_:system.audit NOT system.audit.res:failure\\\").label(\\\"Success\\\") .es(q=\\\"system.audit.res:failed\\\").label(\\\"Failure\\\").title(\\\"Audit Event Results\\\")\",\"interval\":\"auto\"}}", | ||
"description": "", | ||
"title": "Audit Event Results", | ||
"uiStateJSON": "{}", | ||
"version": 1, | ||
"kibanaSavedObjectMeta": { | ||
"searchSourceJSON": "{}" | ||
} | ||
} |
10 changes: 10 additions & 0 deletions
10
.../module/system/audit/_meta/kibana/visualization/5ebdbe50-0a0f-11e7-825f-6748cda7d858.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
{ | ||
"visState": "{\"title\":\"Audit Top Exec Commands\",\"type\":\"table\",\"params\":{\"perPage\":10,\"showPartialRows\":false,\"showMeticsAtAllLevels\":false,\"sort\":{\"columnIndex\":null,\"direction\":null},\"showTotal\":false,\"totalFunc\":\"sum\"},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"schema\":\"metric\",\"params\":{}},{\"id\":\"2\",\"enabled\":true,\"type\":\"terms\",\"schema\":\"bucket\",\"params\":{\"field\":\"system.audit.a0\",\"size\":30,\"order\":\"desc\",\"orderBy\":\"1\",\"customLabel\":\"Command (arg 0)\"}}],\"listeners\":{}}", | ||
"description": "", | ||
"title": "Audit Top Exec Commands", | ||
"uiStateJSON": "{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}}", | ||
"version": 1, | ||
"kibanaSavedObjectMeta": { | ||
"searchSourceJSON": "{\"index\":\"filebeat-*\",\"query\":{\"query_string\":{\"query\":\"system.audit.type:EXECVE\",\"analyze_wildcard\":true}},\"filter\":[]}" | ||
} | ||
} |
10 changes: 10 additions & 0 deletions
10
.../module/system/audit/_meta/kibana/visualization/6295bdd0-0a0e-11e7-825f-6748cda7d858.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
{ | ||
"visState": "{\"title\":\"Audit Event Types\",\"type\":\"pie\",\"params\":{\"addTooltip\":true,\"addLegend\":true,\"legendPosition\":\"right\",\"isDonut\":true},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"schema\":\"metric\",\"params\":{}},{\"id\":\"2\",\"enabled\":true,\"type\":\"terms\",\"schema\":\"segment\",\"params\":{\"field\":\"system.audit.type\",\"size\":50,\"order\":\"desc\",\"orderBy\":\"1\"}}],\"listeners\":{}}", | ||
"description": "", | ||
"title": "Audit Event Types", | ||
"uiStateJSON": "{}", | ||
"version": 1, | ||
"kibanaSavedObjectMeta": { | ||
"searchSourceJSON": "{\"index\":\"filebeat-*\",\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":[]}" | ||
} | ||
} |
10 changes: 10 additions & 0 deletions
10
.../module/system/audit/_meta/kibana/visualization/c5411910-0a87-11e7-8b04-eb22a5669f27.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
{ | ||
"visState": "{\"title\":\"Audit Event Account Tag Cloud\",\"type\":\"tagcloud\",\"params\":{\"scale\":\"linear\",\"orientation\":\"single\",\"minFontSize\":15,\"maxFontSize\":42,\"hideLabel\":false},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"schema\":\"metric\",\"params\":{}},{\"id\":\"2\",\"enabled\":true,\"type\":\"terms\",\"schema\":\"segment\",\"params\":{\"field\":\"system.audit.acct\",\"size\":15,\"order\":\"desc\",\"orderBy\":\"1\"}}],\"listeners\":{}}", | ||
"description": "", | ||
"title": "Audit Event Account Tag Cloud", | ||
"uiStateJSON": "{}", | ||
"version": 1, | ||
"kibanaSavedObjectMeta": { | ||
"searchSourceJSON": "{\"index\":\"filebeat-*\",\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":[]}" | ||
} | ||
} |
10 changes: 10 additions & 0 deletions
10
.../module/system/audit/_meta/kibana/visualization/d1726930-0a7f-11e7-8b04-eb22a5669f27.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
{ | ||
"visState": "{\"title\":\"Audit Event Address Geo Location\",\"type\":\"tile_map\",\"params\":{\"mapType\":\"Scaled Circle Markers\",\"isDesaturated\":true,\"addTooltip\":true,\"heatMaxZoom\":16,\"heatMinOpacity\":0.1,\"heatRadius\":25,\"heatBlur\":15,\"heatNormalizeData\":true,\"legendPosition\":\"bottomright\",\"mapZoom\":2,\"mapCenter\":[15,5],\"wms\":{\"enabled\":false,\"url\":\"https://basemap.nationalmap.gov/arcgis/services/USGSTopo/MapServer/WMSServer\",\"options\":{\"version\":\"1.3.0\",\"layers\":\"0\",\"format\":\"image/png\",\"transparent\":true,\"attribution\":\"Maps provided by USGS\",\"styles\":\"\"}}},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"schema\":\"metric\",\"params\":{}},{\"id\":\"2\",\"enabled\":true,\"type\":\"geohash_grid\",\"schema\":\"segment\",\"params\":{\"field\":\"system.audit.geoip.location\",\"autoPrecision\":true,\"precision\":2}}],\"listeners\":{}}", | ||
"description": "", | ||
"title": "Audit Event Address Geo Location", | ||
"uiStateJSON": "{}", | ||
"version": 1, | ||
"kibanaSavedObjectMeta": { | ||
"searchSourceJSON": "{\"index\":\"filebeat-*\",\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":[]}" | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
input_type: log | ||
paths: | ||
{{ range $i, $path := .paths }} | ||
- {{$path}} | ||
{{ end }} | ||
exclude_files: [".gz$"] |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a specific reason this was moved?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This applies the overrides first, which allows an override to disable a fileset. It allows a CLI config like:
-M system.audit.enabled=false