Skip to content

Commit

Permalink
Add fileset for parsing linux auditd logs (elastic#3750) (elastic#3923)…
Browse files Browse the repository at this point in the history
… (elastic#3941) (elastic#3962)

The PR adds an auditd module for parsing logs from the auditd daemon. There is a sample dashboard for viewing this data.

Features

- Parses audit event type, unix epoch time, audit event counter, and the arbitrary key/value pairs that follow.
- Applies geoip lookup to the `auditd.log.addr` field which is present for some remote login events.
- Decodes hex encoded ascii values that are sometimes used for the `auditd.log.exe` and `auditd.log.cmd` fields.
- Remove key/value pairs where the value is `?`.

Missing Features

- Decoder for `auditd.log.saddr` field present in SOCKADDR audit events. A recipe is described [here](https://unix.stackexchange.com/questions/102926/how-to-interpret-the-saddr-field-of-an-audit-log) and could probably be ported to painless. Sample value:
  `type=SOCKADDR msg=audit(1481078693.491:873): saddr=01002F7661722F72756E2F6E7363642F736F636B657400008983B330BE7F0000000000000000000070830130BE7F000004000000000000001B00000000000000D128B7ED000000008B8BB330BE7F00003B00000000000000403E4BF7FD7F0000447F0130BE7F000080150230BE7F`
  • Loading branch information
andrewkroh committed Apr 10, 2017
1 parent f59240e commit 529f983
Show file tree
Hide file tree
Showing 33 changed files with 3,390 additions and 17 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -153,6 +153,7 @@ https://github.com/elastic/beats/compare/v5.2.2...v5.3.0[View commits]
- The `symlinks` and `harverster_limit` settings are now GA, instead of experimental. {pull}3525[3525]
- close_timeout is also applied when the output is blocking. {pull}3511[3511]
- Improve handling of different path variants on Windows. {pull}3781[3781]
- Add auditd module for reading audit logs on Linux. {pull}3750[3750] {pull}3941[3941]
*Metricbeat*
Expand Down
149 changes: 149 additions & 0 deletions filebeat/docs/fields.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ This document describes the fields that are exported by Filebeat. They are
grouped in the following categories:
* <<exported-fields-apache2>>
* <<exported-fields-auditd>>
* <<exported-fields-beat>>
* <<exported-fields-cloud>>
* <<exported-fields-log>>
Expand Down Expand Up @@ -285,6 +286,154 @@ type: keyword
The module producing the logged message.
[[exported-fields-auditd]]
== Auditd Fields
Module for parsing auditd logs.
[float]
== auditd Fields
Fields from the auditd logs.
[float]
== log Fields
Fields from the Linux audit log. Not all fields are documented here because they are dynamic and vary by audit event type.
[float]
=== auditd.log.record_type
The audit event type.
[float]
=== auditd.log.old_auid
For login events this is the old audit ID used for the user prior to this login.
[float]
=== auditd.log.new_auid
For login events this is the new audit ID. The audit ID can be used to trace future events to the user even if their identity changes (like becoming root).
[float]
=== auditd.log.old_ses
For login events this is the old session ID used for the user prior to this login.
[float]
=== auditd.log.new_ses
For login events this is the new session ID. It can be used to tie a user to future events by session ID.
[float]
=== auditd.log.sequence
type: long
The audit event sequence number.
[float]
=== auditd.log.acct
The user account name associated with the event.
[float]
=== auditd.log.pid
The ID of the process.
[float]
=== auditd.log.ppid
The ID of the process.
[float]
=== auditd.log.items
The number of items in an event.
[float]
=== auditd.log.item
The item field indicates which item out of the total number of items. This number is zero-based; a value of 0 means it is the first item.
[float]
=== auditd.log.a0
The first argument to the system call.
[float]
=== auditd.log.res
The result of the system call (success or failure).
[float]
== geoip Fields
Contains GeoIP information gathered based on the `auditd.log.addr` field. Only present if the GeoIP Elasticsearch plugin is available and used.
[float]
=== auditd.log.geoip.continent_name
type: keyword
The name of the continent.
[float]
=== auditd.log.geoip.city_name
type: keyword
The name of the city.
[float]
=== auditd.log.geoip.region_name
type: keyword
The name of the region.
[float]
=== auditd.log.geoip.country_iso_code
type: keyword
Country ISO code.
[float]
=== auditd.log.geoip.location
type: geo_point
The longitude and latitude.
[[exported-fields-beat]]
== Beat Fields
Expand Down
Binary file added filebeat/docs/images/kibana-audit-auditd.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
41 changes: 41 additions & 0 deletions filebeat/docs/modules/auditd.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
////
This file is generated! See scripts/docs_collector.py
////

[[filebeat-module-auditd]]
== Auditd module

This module collects and parses logs from the audit daemon (`auditd`).

[float]
=== Compatibility

This module was tested with logs from `auditd` on OSes like CentOS 6 and
CentOS 7.

This module is not available for Windows.

[float]
=== Dashboard

This module comes with a sample dashboard showing an overview of the audit log
data. You can build more specific dashboards that are tailored to the audit
rules that you use on your systems.

image::./images/kibana-audit-auditd.png[]

[float]
=== Syslog fileset settings

[float]
==== var.paths

An array of paths where to look for the log files. If left empty, Filebeat
will choose the paths depending on your operating systems.


=== Fields

For a description of each field in the metricset, see the
<<exported-fields-auditd,exported fields>> section.

11 changes: 7 additions & 4 deletions filebeat/docs/modules/system.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -5,18 +5,21 @@ This file is generated! See scripts/docs_collector.py
[[filebeat-module-system]]
== System module

This module collects and parses logs created by system logging server of common Unix/Linux based
distributions.
This module collects and parses logs created by system logging server of common
Unix/Linux based distributions.

[float]
=== Compatibility

This module was tested with logs from OSes like Ubuntu 12.04, Centos 7, macOS Sierra, and others.
This module was tested with logs from OSes like Ubuntu 12.04, Centos 7, and
macOS Sierra.

This module is not available for Windows.

[float]
=== Dashboard

This module comes with a sample dashboard.
This module comes with a sample dashboard showing syslog data.

image::./images/kibana-system.png[]

Expand Down
2 changes: 2 additions & 0 deletions filebeat/docs/modules_list.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ This file is generated! See scripts/docs_collector.py

* <<filebeat-modules-overview>>
* <<filebeat-module-apache2>>
* <<filebeat-module-auditd>>
* <<filebeat-module-mysql>>
* <<filebeat-module-nginx>>
* <<filebeat-module-system>>
Expand All @@ -14,6 +15,7 @@ This file is generated! See scripts/docs_collector.py

include::modules-overview.asciidoc[]
include::modules/apache2.asciidoc[]
include::modules/auditd.asciidoc[]
include::modules/mysql.asciidoc[]
include::modules/nginx.asciidoc[]
include::modules/system.asciidoc[]
Expand Down
13 changes: 13 additions & 0 deletions filebeat/filebeat.full.yml
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,19 @@ filebeat.modules:
# can be added under this section.
#prospector:

#------------------------------- Auditd Module -------------------------------
#- module: auditd
#log:
#enabled: true

# Set custom paths for the log files. If left empty,
# Filebeat will choose the paths depending on your OS.
#var.paths:

# Prospector configuration (advanced). Any prospector configuration option
# can be added under this section.
#prospector:

#-------------------------------- MySQL Module -------------------------------
#- module: mysql
# Error logs
Expand Down
98 changes: 98 additions & 0 deletions filebeat/filebeat.template-es2x.json
Original file line number Diff line number Diff line change
Expand Up @@ -169,6 +169,104 @@
}
}
},
"auditd": {
"properties": {
"log": {
"properties": {
"a0": {
"ignore_above": 1024,
"index": "not_analyzed",
"type": "string"
},
"acct": {
"ignore_above": 1024,
"index": "not_analyzed",
"type": "string"
},
"geoip": {
"properties": {
"city_name": {
"ignore_above": 1024,
"index": "not_analyzed",
"type": "string"
},
"continent_name": {
"ignore_above": 1024,
"index": "not_analyzed",
"type": "string"
},
"country_iso_code": {
"ignore_above": 1024,
"index": "not_analyzed",
"type": "string"
},
"location": {
"type": "geo_point"
},
"region_name": {
"ignore_above": 1024,
"index": "not_analyzed",
"type": "string"
}
}
},
"item": {
"ignore_above": 1024,
"index": "not_analyzed",
"type": "string"
},
"items": {
"ignore_above": 1024,
"index": "not_analyzed",
"type": "string"
},
"new_auid": {
"ignore_above": 1024,
"index": "not_analyzed",
"type": "string"
},
"new_ses": {
"ignore_above": 1024,
"index": "not_analyzed",
"type": "string"
},
"old_auid": {
"ignore_above": 1024,
"index": "not_analyzed",
"type": "string"
},
"old_ses": {
"ignore_above": 1024,
"index": "not_analyzed",
"type": "string"
},
"pid": {
"ignore_above": 1024,
"index": "not_analyzed",
"type": "string"
},
"ppid": {
"ignore_above": 1024,
"index": "not_analyzed",
"type": "string"
},
"record_type": {
"ignore_above": 1024,
"index": "not_analyzed",
"type": "string"
},
"res": {
"ignore_above": 1024,
"index": "not_analyzed",
"type": "string"
},
"sequence": {
"type": "long"
}
}
}
}
},
"beat": {
"properties": {
"hostname": {
Expand Down
Loading

0 comments on commit 529f983

Please sign in to comment.