OSSEC: Remove or modify daily "Ossec server/agent started" emails

[redshiftzero]

Description

The SecureDrop servers reboot every day. After reboot, the OSSEC server and agent start and one gets two emails:

Received From: (app) 10.20.2.2->ossec
Rule: 503 fired (level 3) -> "Ossec agent started."
Portion of the log(s):

ossec: Agent started: 'app->10.20.2.2'.

Received From: mon->ossec-monitord
Rule: 502 fired (level 3) -> "Ossec server started."
Portion of the log(s):

ossec: Ossec started.

Some administrators may not want these emails and would consider them spam. On the other hand, some may want these emails because it lets the administrator know that their monitor server is working and that the nightly reboot has occurred successfully. I have heard both these sentiments from administrators in the past on these particular alerts, so I'm opening this for discussion before taking action to see what others think.

Options

1. Remove both alerts. If people want to be reassured that their monitor server is working, they must SSH in or take some other action that triggers an alert.
2. Remove one alert. There are emails from both app and mon here. One is redundant: if the OSSEC server does not start, then the OSSEC agent started email will not be sent. I suggest we suppress the "Ossec server started" alert, as that is implicit if one is receiving an email alert from OSSEC (i.e. leave only the "Ossec agent started" alert). One might also edit the description in the alert to say "OSSEC restarted after nightly SecureDrop reboot" or something more explicit.
3. Leave as is (this option is unsatisfactory if the logic in Option 2 is sound).

User Stories

As a SecureDrop administrator, I want to decrypt only email alerts containing useful or actionable information.

[micahflee]

I vote for removing both alerts.

It has been 9 days since I've reinstalled SecureDrop and so far I have gotten 506 emails. I haven't finished going through them all, but I'm pretty sure that zero of them contain any information that's useful to me as an administrator.

[heartsucker]

@micahflee Could you post more of the unactionable alerts here so we can remove those too.

[conorsch]

@micahflee Well put. The flood of notifications due to reinstall boils down to:

• Ansible playbook runs generate tons of noise (OSSEC: Running Ansible playbooks triggers level 13 alerts #2156)
• Alert send threshold too low (we should bump it up so only "important" / actionable notifications are sent)
• More verbose notification (i.e. lower threshold alerts) can be sent to FPF for aggregation if admins opt-in (Automatically send a subset of system alerts to FPF #973)

Long story short, we need to reduce the amount of noise to make OSSEC useful again. I'm also strongly in favor of dropping encrypted email as the transport in favor of something more usable like Signal, but we'll need to reduce the noise first.

[conorsch]

@micahflee Could you post more of the unactionable alerts here so we can remove those too.

@heartsucker I'd prefer separate issues so we can knock them out and add discrete tests for "does send alert" and "does not send alert". Separate issues will streamline the review process and keep PRs small enough that we can merge them quickly.

[micahflee]

I opened a new issue with some specific OSSEC noise #2507

[micahflee]

Also, relevant for this issue, I think we should remove the Daily report: File Changes email as well. It gets sent every day, and since other file changes get sent in other emails, it's redundant.

[b-meson]

I think this is one of the few useful OSSEC alerts in that it shows OSSEC is actually monitoring. We discussed internally that the "monitoring server not actually monitoring" is part of the threat model that we want to worry about. I think the real issue is that OSSEC requires a very unwieldy workflow to deal with (i.e. GPG) and a move to something like signal-cli would reduce the burden of admins. (I definitely want admins to know if OSSEC starts daily).

@micahflee @emkll do you think we should suppress all emails for file changes except the Daily report: File Changes since I think that one is more user friendly.

[conorsch]

do you think we should suppress all emails for file changes except the Daily report: File Changes since I think that one is more user friendly.

Ported those comments to #1712, since that issue already exists for file integrity monitoring changes. This issue is tracking the "Ossec agent started" and "Ossec server started" messages.

I vote for removing both alerts.
It has been 9 days since I've reinstalled SecureDrop and so far I have gotten 506 emails.

+1 to @micahflee's comments. I also vote for removing both alerts.