From 211d2524463694fa700de090d9466ffd154d619d Mon Sep 17 00:00:00 2001 From: Andrew Kroh Date: Thu, 13 Jul 2017 05:56:27 -0400 Subject: [PATCH] Auditbeat file module improvements (#4636) Unify the behavior across operating systems. - Add tests. - Resolve UID/GID/SID into owner/group. - Log warning if a file watch cannot be added. Previously it would fail at startup. - Remove usage of fsevents on macOS and use kqueues instead. The data provided was inconsistent with the fsnotify. It was valuable data but the signal needs some refinement to make it consistent. The biggest issue is that multiple events are coalesced into one. Another issue is that the watches are recursive (this is great, but it's inconsistent) and we will need to allow a glob to be specified then match the incoming events against the glob (future enhancement). - Document hash fields - Initialize watches on Start() rather than at construction time. - Update data.json to include file owner and group names. - Add file type and symlink target path to the events. - Filter fsnotify events with empty paths (saw this once on Windows). * Add auditbeat file dashboard - Change mode format to %#04o to always get a leading 0 and a minimum of 4 digits. - Change Stat of Lstat in assertSameFile - Reduce code duplication between linux and bsd platforms. --- Vagrantfile | 21 ++ auditbeat/docs/fields.asciidoc | 44 ++- .../auditbeat-file-integrity-dashboard.png | Bin 0 -> 218137 bytes .../default/dashboard/Auditbeat-audit.json | 132 --------- .../dashboard/auditbeat-file-integrity.json | 188 +++++++++++++ auditbeat/module/audit/file/_meta/data.json | 15 +- auditbeat/module/audit/file/_meta/fields.yml | 22 +- auditbeat/module/audit/file/event.go | 109 +++++--- .../module/audit/file/eventreader_darwin.go | 86 ------ .../module/audit/file/eventreader_fsnotify.go | 119 ++++++++ .../module/audit/file/eventreader_other.go | 83 ------ auditbeat/module/audit/file/fileinfo_bsd.go | 39 +-- auditbeat/module/audit/file/fileinfo_linux.go | 41 +-- .../module/audit/file/fileinfo_windows.go | 87 ++++-- auditbeat/module/audit/file/filepath_posix.go | 61 +++++ auditbeat/module/audit/file/metricset.go | 13 +- auditbeat/module/audit/file/metricset_test.go | 259 ++++++++++++++++++ .../module/audit/file/security_windows.go | 56 ++++ .../audit/file/security_windows_test.go | 40 +++ .../module/audit/file/zsecurity_windows.go | 30 ++ 20 files changed, 999 insertions(+), 446 deletions(-) create mode 100644 auditbeat/docs/images/auditbeat-file-integrity-dashboard.png delete mode 100644 auditbeat/module/audit/_meta/kibana/default/dashboard/Auditbeat-audit.json create mode 100644 auditbeat/module/audit/_meta/kibana/default/dashboard/auditbeat-file-integrity.json delete mode 100644 auditbeat/module/audit/file/eventreader_darwin.go create mode 100644 auditbeat/module/audit/file/eventreader_fsnotify.go delete mode 100644 auditbeat/module/audit/file/eventreader_other.go create mode 100644 auditbeat/module/audit/file/filepath_posix.go create mode 100644 auditbeat/module/audit/file/security_windows.go create mode 100644 auditbeat/module/audit/file/security_windows_test.go create mode 100644 auditbeat/module/audit/file/zsecurity_windows.go diff --git a/Vagrantfile b/Vagrantfile index 47857ef61c0..86da498e6c0 100644 --- a/Vagrantfile +++ b/Vagrantfile @@ -55,6 +55,15 @@ cd ~/go/src/github.com/elastic if [ -d "/vagrant" ]; then ln -s /vagrant beats; fi SCRIPT +# Linux GVM +$linuxGvmProvision = <