Merge pull request #199 from freedomofpress/add_ossec_rules

Adds details about OSSEC rule addition

docs/development/updating_ossec.rst

@@ -23,6 +23,8 @@ suggesting a rule for it. Each additional alert that admins must read and/or
 respond to takes time. Alerts that are unimportant or otherwise require no action
 can lead to alert fatigue and thus to critical alerts being ignored.
 
+.. _using_ossec_logtest :
+
 Using ``ossec-logtest``
 -----------------------
 
@@ -57,6 +59,8 @@ level, you can simply pass the event to ``ossec-logtest``:
 
 This is the utility we use in automated tests of OSSEC.
 
+.. _writing_automated_tests_for_ossec_rules :
+
 Writing Automated Tests for OSSEC Rules
 ---------------------------------------
 
@@ -82,6 +86,123 @@ a failing test which you then can make pass with a patch to the OSSEC rules:
 .. _syscheck: https://ossec-docs.readthedocs.io/en/latest/docs/manual/syscheck/index.html
 .. _2134: https://github.com/freedomofpress/securedrop/issues/2134
 
+
+How to add a new OSSEC rule?
+=============================
+
+OSSEC processes events in two steps:
+
+1. `Decoders <https://ossec-documentation.readthedocs.io/en/latest/manual/lids/decoders.html>`_
+   parse and filter log events that meet certain criteria for subsequent processing.
+   SecureDrop's custom rules are defined in
+   ``install_files/securedrop-ossec-server/var/ossec/rules/local_rules.xml``.
+
+2. `Rules <https://ossec-documentation.readthedocs.io/en/latest/manual/lids/rules.html>`_
+   check decoded events against conditions and optionally yield alerts.
+   SecureDrop's custom rules are defined in
+   ``install_files/securedrop-ossec-server/var/ossec/etc/local_decoder.xml``.
+
+A basic decoder filters log events by ``program_name`` (e.g., ``fwupd``).
+If a decoder is already defined for the program of interest, you can go straight
+to :ref:`defining a new rule <the_rules>` unless you have a reason to add additional
+:ref:`decoders <the_decoder_file>` for further filtering.
+
+
+.. _the_decoder_file:
+
+The decoder file
+-----------------
+
+For example, to add a decoder for log events from ``fwupd``, you can add to
+``local_decoder.xml``:
+
+::
+
+    <!--
+      The default fwupd tries to auto-update and generates error.
+    -->
+    <decoder name="fwupd">
+      <program_name>fwupd</program_name>
+    </decoder>
+
+You can find this ``program_name`` value using the :ref:`"ossec-logtest" command
+<using_ossec_logtest>`.  Copy-paste the log event as input to this command, and
+it will give you some parsed output:
+
+::
+
+    $ echo "Mar  1 13:22:53 app fwupd[133921]: 13:22:53:0883 FuPluginUefi         Error opening directory “/sys/firmware/efi/esrt/entries�: No such file or directory" | sudo /var/ossec/bin/ossec-logtest
+    [...]
+    **Phase 1: Completed pre-decoding.
+        full event: 'Mar  1 13:22:53 app fwupd[133921]: 13:22:53:0883 FuPluginUefi         Error opening directory “/sys/firmware/efi/esrt/entries�: No such file or directory'
+        hostname: 'app'
+        program_name: 'fwupd'
+        log: '13:22:53:0883 FuPluginUefi         Error opening directory “/sys/firmware/efi/esrt/entries�: No such file or directory'
+
+    **Phase 2: Completed decoding.
+        No decoder matched.
+
+    **Phase 3: Completed filtering (rules).
+        Rule id: '1002'
+        Level: '2'
+        Description: 'Unknown problem somewhere in the system.'
+    **Alert to be generated.
+
+.. _the_rules:
+
+The rules
+---------
+
+Next, you can add one or more rules corresponding to the new decoder, making
+sure that the rules have proper unique `id` numbers and are written in the
+correct (sorted) place in the ``local_rules.xml`` file.
+
+
+::
+
+    <group name="fwupd">
+    <rule id="100111" level="0">
+        <decoded_as>fwupd</decoded_as>
+        <match>Error opening directory</match>
+        <description>fwupd error</description>
+        <options>no_email_alert</options>
+    </rule>
+    <rule id="100112" level="0">
+        <decoded_as>fwupd</decoded_as>
+        <match>Failed to load SMBIOS</match>
+        <description>fwupd error for auto updates</description>
+        <options>no_email_alert</options>
+    </rule>
+    </group>
+
+
+Verify the new OSSEC rule
+-------------------------
+
+On the monitor server you can use the following command as `root` to verify
+the new rule:
+
+::
+
+    /var/ossec/bin/ossec-analysisd -t
+
+``ossec-analysisd`` will receive log messages and compare them to our rules,
+including the new rule you just added. Then it creates alerts when a log message
+matches an applicable rule.
+
+
+Adding an automated test for staging
+-------------------------------------
+
+You can then add tests in the ``molecule/testinfra/mon/test_ossec_ruleset.py``
+file. Here the test loops over the entries in the
+``log_events_with_ossec_alerts`` and ``log_events_without_ossec_alerts``
+variables in ``molecule/testinfra/vars/staging.yml`` and makes sure that the
+``rule_id`` and ``level`` match.  See :ref:`writing_automated_tests_for_ossec_rules`
+for details.
+
+
+
 Deployment
 ----------