Add OSSEC rule to indicate to admins what ansible-apt-key alert means

docs/development/testing_configuration_tests.rst

@@ -88,6 +88,9 @@ relevant location for the host you plan to test: ::
 In the example above, to add a new test for the ``app-staging`` host,
 add a new file to the ``testinfra/spec/app-staging`` directory.
 
+.. tip:: Read :ref:`updating_ossec_rules` to learn how to write tests for the
+         OSSEC rules.
+
 Config test layout
 ------------------
 
@@ -111,9 +114,9 @@ Config testing strategy
 -----------------------
 
 The config tests currently emphasize testing implementation rather than
-functionality. This is a temporary measure to increase the current testing
-baseline for validating the Ansible provisioning flow, to aid in migrating
-to a current version of Ansible (v2+). After the Ansible version is current,
+functionality. This was a temporary measure to increase the testing
+baseline for validating the Ansible provisioning flow, which aided in migrating
+to a current version of Ansible (v2+). Now that the Ansible version is current,
 the config tests can be improved to validate behavior, such as confirming
 ports are blocked via external network calls, rather than simply checking
 that the iptables rules are formatted as expected.

docs/development/updating_ossec.rst

@@ -0,0 +1,95 @@
+.. _updating_ossec_rules:
+
+Updating OSSEC Rules
+====================
+
+SecureDrop uses the OSSEC open source host-based intrusion detection system
+(IDS) for log analysis, file integrity checking, policy monitoring, rootkit
+detection and real-time alerting. Refer to our :ref:`OSSEC guide <ossec_guide>`
+to learn more about how SecureDrop admins set up and monitor OSSEC alerts.
+
+Alerting Strategy
+-----------------
+
+The goals of the OSSEC alerts in SecureDrop is to notify administrators of:
+
+1. Suspicious security events
+2. Changes that require some kind of administrator action
+3. Other important notifications regarding system state.
+
+If an alert is purely informational and there is no realistic action an
+admin is expected to take, you should think carefully before
+suggesting a rule for it. Each additional alert that admins must read and/or
+respond to takes time. Alerts that are unimportant or otherwise require no action
+can lead to alert fatigue and thus to critical alerts being ignored.
+
+Using ``ossec-logtest``
+-----------------------
+
+Development on the OSSEC rules should be done from the staging environment.
+
+On ``mon-staging``, there is a utility installed as part of OSSEC called
+``ossec-logtest`` that you can use to test log events. In order to evaluate
+whether an alert will be produced, and if so, what rule triggered it and its
+level, you can simply pass the event to ``ossec-logtest``:
+
+.. code:: sh
+
+  root@mon-staging:/home/vagrant# sudo echo "Feb 10 23:34:40 app-prod kernel: [  124.188641] grsec: denied RWX mmap of <anonymous mapping> by /usr/sbin/apache2[apache2:1328] uid/euid:33/33 gid/egid:33/33, parent /usr/sbin/apache2[apache2:1309] uid/euid:0/0 gid/egid:0/0" | /var/ossec/bin/ossec-logtest
+  2017/08/16 22:28:25 ossec-testrule: INFO: Reading local decoder file.
+  2017/08/16 22:28:25 ossec-testrule: INFO: Started (pid: 18973).
+  ossec-testrule: Type one log per line.
+
+  **Phase 1: Completed pre-decoding.
+         full event: 'Feb 10 23:34:40 app-prod kernel: [  124.188641] grsec: denied RWX mmap of <anonymous mapping> by /usr/sbin/apache2[apache2:1328] uid/euid:33/33 gid/egid:33/33, parent /usr/sbin/apache2[apache2:1309] uid/euid:0/0 gid/egid:0/0'
+         hostname: 'app-prod'
+         program_name: 'kernel'
+         log: '[  124.188641] grsec: denied RWX mmap of <anonymous mapping> by /usr/sbin/apache2[apache2:1328] uid/euid:33/33 gid/egid:33/33, parent /usr/sbin/apache2[apache2:1309] uid/euid:0/0 gid/egid:0/0'
+
+  **Phase 2: Completed decoding.
+         decoder: 'iptables'
+
+  **Phase 3: Completed filtering (rules).
+         Rule id: '100101'
+         Level: '7'
+         Description: 'grsec error was detected'
+  **Alert to be generated.
+
+This is the utility we use in automated tests of OSSEC.
+
+Writing Automated Tests for OSSEC Rules
+---------------------------------------
+
+We strongly recommend before making changes to OSSEC rules to attempt to write
+a failing test which you then can make pass with a patch to the OSSEC rules:
+
+1. Identify a log event you can use to trigger the alert.
+
+.. warning:: Be sure to use only log events from test SecureDrop instances or
+             those you have verified do not contain any sensitive data.
+
+2. Write a Testinfra_ test to verify that the log event does or does not trigger
+   an alert.
+3. Apply your patch to the OSSEC rule on the relevant VM (likely ``app``).
+4. Restart the service via ``sudo service ossec restart`` on ``mon``.
+
+.. note:: Currently we only have automated tests for alerts triggered due to
+          log events (for example not for `syscheck`_, OSSEC's integrity
+          checking process). If you have ideas for additional automated test
+          coverage of alerts, please suggest them in ticket `2134`_ on GitHub.
+
+.. _Testinfra: https://testinfra.readthedocs.io/en/latest/
+.. _syscheck: https://ossec-docs.readthedocs.io/en/latest/manual/syscheck/
+.. _2134: https://github.com/freedomofpress/securedrop/issues/2134
+
+Deployment
+----------
+
+The OSSEC rules and associated configuration files are distributed via Debian
+packages maintained by Freedom of the Press Foundation. Any changes made to OSSEC
+configuration files will land on production SecureDrop monitoring servers as
+part of each SecureDrop release. This upgrade will occur automatically.
+
+.. note:: The use of automatic upgrades for release deployment means that any
+          changes made locally by administrators to their OSSEC rules will not
+          persist after a SecureDrop update.

docs/index.rst

@@ -95,5 +95,6 @@ anonymous sources.
    development/testing_configuration_tests
    development/testing_continuous_integration
    development/apt_repo
+   development/updating_ossec
    development/apparmor_profiles
    development/threat_model

docs/ossec_alerts.rst

@@ -1,3 +1,5 @@
+.. _ossec_guide:
+
 OSSEC Guide
 ===========
 

install_files/securedrop-ossec-server/var/ossec/rules/local_rules.xml

@@ -155,3 +155,18 @@
     <options>no_email_alert</options>
   </rule>
 </group>
+
+<!--
+  Alert on Ansible playbook runs on both servers. Note that administrators may
+  not realize what an "Ansible playbook" is, so we should be clear that this
+  means that securedrop-admin was executed with one of the options that involves
+  an Ansible playbook run.
+-->
+<group name="Ansible playbooks">
+  <rule id="400001" level="13">
+    <if_sid>1003</if_sid>
+    <program_name>ansible-apt_key</program_name>
+    <match>Invoked with file=None keyserver=None url=None</match>
+    <description>Ansible playbook run on server (securedrop-admin install, backup, or restore).</description>
+  </rule>
+</group>

testinfra/mon/test_ossec_ruleset.py

@@ -1,61 +1,29 @@
+import pytest
 import re
 
 
 alert_level_regex = re.compile(r"Level: '(\d+)'")
+rule_id_regex = re.compile(r"Rule id: '(\d+)'")
+sdvars = pytest.securedrop_test_vars
 
 
-def test_grsec_denied_rwx_mapping_produces_alert(Command, Sudo):
-    """Check that a denied RWX mmaping produces an OSSEC alert"""
-    test_alert = ("Feb 10 23:34:40 app kernel: [  124.188641] grsec: denied "
-                  "RWX mmap of <anonymous mapping> by /usr/sbin/apache2"
-                  "[apache2:1328] uid/euid:33/33 gid/egid:33/33, parent "
-                  "/usr/sbin/apache2[apache2:1309] uid/euid:0/0 gid/egid:0/0")
-
+@pytest.mark.parametrize('log_event',
+                         sdvars.log_events_without_ossec_alerts)
+def test_ossec_false_positives_suppressed(Command, Sudo, log_event):
     with Sudo():
         c = Command('echo "{}" | /var/ossec/bin/ossec-logtest'.format(
-                test_alert))
-
-        # Level 7 alert should be triggered by rule 100101
-        assert "Alert to be generated" in c.stderr
-        alert_level = alert_level_regex.findall(c.stderr)[0]
-        assert alert_level == "7"
-
-
-def test_overloaded_tor_guard_does_not_produce_alert(Command, Sudo):
-    """Check that using an overloaded guard does not produce an OSSEC alert"""
-    test_alert = ("Aug 16 21:54:44 app-staging Tor[26695]: [warn] Your Guard "
-                  "<name> (<fingerprint>) is failing a very large amount of "
-                  "circuits. Most likely this means the Tor network is "
-                  "overloaded, but it could also mean an attack against you "
-                  "or potentially the guard itself.")
-
-    with Sudo():
-        c = Command('echo "{}" | /var/ossec/bin/ossec-logtest'.format(
-                test_alert))
+                log_event["alert"]))
         assert "Alert to be generated" not in c.stderr
 
 
-def test_ossec_keep_alive_mark_does_not_produce_alert(Command, Sudo):
-    """Check that OSSEC keep alive messages sent to the OSSEC manager
-    do not produce OSSEC alerts.
-
-    For more information see:
-    https://github.com/ossec/ossec-hids/issues/466
-    http://ossec-docs.readthedocs.io/en/latest/faq/alerts.html
-    """
-
-    # Example alert from:
-    # https://groups.google.com/forum/#!msg/ossec-list/dE3klm84JMU/kGZkRdSl3ZkJ
-    test_alert = ("Dec 02 09:48:40 app-staging ossec-keepalive: --MARK--: "
-                  "&pQSW__BPa5S?%tyDTJ3-iCG2lz2dU))r(F%6tjp8wqpf=]IKFT%ND2k"
-                  "P]ua/W)3-6'eHduX$;$Axqq7Vr.dVZ1SUDSaH)4xTXCIieaEKv47LD-b"
-                  "U)SXMnXO/jPGKn3.!NGBR_5]jD2UoSV9)h%z8G%7.xhI;s)267.rV214"
-                  "O@t2#w)Z(k'UQp9]MyDERrOrG[-,e?iS@B3Rg/kGiR[g6mc0K)/]S]0'"
-                  "+?+'/.[r$fqBR^7iAjoPv4j6SWjeRsLGr%$3#p+buf&u_RC3i/mE3vS3*"
-                  "jp&B1qSJM431TmEg,YJ][ge;6-dJI69?-TB?!BI4?Uza63V3vMY3ake6a"
-                  "hj-%A-m_5lgab!OVR,!pR+;L]eLgilU")
-
+@pytest.mark.parametrize('log_event',
+                         sdvars.log_events_with_ossec_alerts)
+def test_ossec_expected_alerts_are_present(Command, Sudo, log_event):
     with Sudo():
         c = Command('echo "{}" | /var/ossec/bin/ossec-logtest'.format(
-                    test_alert))
-        assert "Alert to be generated" not in c.stderr
+                log_event["alert"]))
+        assert "Alert to be generated" in c.stderr
+        alert_level = alert_level_regex.findall(c.stderr)[0]
+        assert alert_level == log_event["level"]
+        rule_id = rule_id_regex.findall(c.stderr)[0]
+        assert rule_id == log_event["rule_id"]

testinfra/vars/mon-staging.yml

@@ -13,3 +13,75 @@ tor_services:
 
 # Disable Postfix in staging, since we don't have valid credentials.
 postfix_enabled: False
+
+# Log events for OSSEC alerts we suppress
+log_events_without_ossec_alerts:
+  # Check that using an overloaded guard does not produce an OSSEC alert
+  - name: test_overloaded_tor_guard_does_not_produce_alert
+    alert: >
+      Aug 16 21:54:44 app-staging Tor[26695]: [warn] Your Guard
+      <name> (<fingerprint>) is failing a very large amount of
+      circuits. Most likely this means the Tor network is
+      overloaded, but it could also mean an attack against you
+      or potentially the guard itself.
+
+  # Check that OSSEC keep alive messages sent to the OSSEC manager
+  # do not produce OSSEC alerts.
+  #
+  # For more information see:
+  # https://github.com/ossec/ossec-hids/issues/466
+  # http://ossec-docs.readthedocs.io/en/latest/faq/alerts.html
+  #
+  # Example alert is from:
+  # https://groups.google.com/forum/#!msg/ossec-list/dE3klm84JMU/kGZkRdSl3ZkJ
+  - name: test_ossec_keep_alive_mark_does_not_produce_alert
+    alert: >
+      Dec 02 09:48:40 app-staging ossec-keepalive: --MARK--:
+      &pQSW__BPa5S?%tyDTJ3-iCG2lz2dU))r(F%6tjp8wqpf=]IKFT%ND2k
+      P]ua/W)3-6'eHduX$;$Axqq7Vr.dVZ1SUDSaH)4xTXCIieaEKv47LD-b
+      U)SXMnXO/jPGKn3.!NGBR_5]jD2UoSV9)h%z8G%7.xhI;s)267.rV214
+      O@t2#w)Z(k'UQp9]MyDERrOrG[-,e?iS@B3Rg/kGiR[g6mc0K)/]S]0'
+      +?+'/.[r$fqBR^7iAjoPv4j6SWjeRsLGr%$3#p+buf&u_RC3i/mE3vS3*
+      jp&B1qSJM431TmEg,YJ][ge;6-dJI69?-TB?!BI4?Uza63V3vMY3ake6a
+      hj-%A-m_5lgab!OVR,!pR+;L]eLgilU
+
+
+# Log events we expect an OSSEC alert to occur for
+log_events_with_ossec_alerts:
+  # Check that a denied RWX mmaping would produce an OSSEC alert
+  - name: test_grsec_denied_rwx_mapping_produces_alert
+    alert: >
+      Feb 10 23:34:40 app kernel: [  124.188641] grsec: denied
+      RWX mmap of <anonymous mapping> by /usr/sbin/apache2
+      [apache2:1328] uid/euid:33/33 gid/egid:33/33, parent
+      /usr/sbin/apache2[apache2:1309] uid/euid:0/0 gid/egid:0/0
+    level: "7" # Level 7 alert should be triggered by rule 100101
+    rule_id: "100101"
+
+  # When Ansible playbooks are run, an informative alert should be triggered
+  - name: test_ansible_playbook_triggers_alert
+    alert: >
+      Jul 22 17:06:41 app ansible-apt_key: Invoked with file=None
+      keyserver=None url=None data=-----BEGIN PGP PUBLIC KEY BLOCK
+      -----#012Version: GnuPG
+      v1#012#012mQENBEqg7GsBCACsef8koRT8UyZxiv1Irke5nVpte54TDtTl1
+      za1tOKfthmHbs2I#0124DHWG3qrwGayw+6yb5mMFe0h9Ap9IbilA5a1IdRs
+      dDgViyQQ3kvdfoavFHRxvGON#012tknIyk5Goa36GMBl84gQceRs/4Zx3kx
+      qCV+JYXE9CmdkpkVrh2K3j5+ysDWfD/kO#012dTzwu3WHaAwL8d5MJAGQn2
+      i6bTw4UHytrYemS1DdG/0EThCCyAnPmmb8iBkZlSW8#0126MzVqTrN37yvY
+      WTXk6MwKH50twaX5hzZAlSh9eqRjZLq51DDomO7EumXP90rS5mT#012QrS+
+      wiYfGQttoZfbh3wl5ZjejgEjx+qrnOH7ABEBAAG0JmRlYi50b3Jwcm9qZWN
+      0#012Lm9yZyBhcmNoaXZlIHNpZ25pbmcga2V5iEYEEBECAAYFAkqqojIACg
+      kQ61qJaiiY#012i/WmOgCfTyf3NJ7wHTBckwAeE4MSt5ZtXVsAn0XDq8PWW
+      nk4nK6TlevqK/VoWItF#012iEYEEBECAAYFAkqsYDUACgkQO50JPzGwl0vo
+      JwCcCSokiJSNY+yIr3nBPN/LJldb#012xekAmwfU60GeaWFwz7hqwVFL23x
+      eTpyniEYEEBECAAYFAkt9ndgACgkQYhWWT1sX#012KrI5TACfcBPbsaPA1A
+      UVVXXPv0KeWFYgVaIAoMr3jwd1NYVD6Te3D+yJhGzzCD6P#012iEYEEBECA
+      AYFAkt+li8ACgkQTlMAGaGhvAU4FwCfX3H4Ggm/x0yIAvmt4CW8AP9F#012
+      5D8AoKapuwbjsGncT3UdNFiHminAaq1tiEYEEBECAAYFAky6mjsACgkQhfc
+      mMSeh#012yJpL+gCggxs4C5o+Oznk7WmFrPQ3lbnfDKIAni4p20aRuwx6QW
+      GH8holjzTSmm5F#012iEYEEBECAAYFAlMI0FEACgkQhEMxewZV94DLagCcD
+      G5SR00+00VHzBVE6fDg027e#012N2sAnjNLOYbRSBxBnELUDKC7Vjaz/sAM
+      iEwEExECAAwFAkqg7nQFgwll/3cACgkQ#0123nqvbpTAnH+GJA
+    level: "13"
+    rule_id: "400001"