feat: add systemd timer to run "manage.py remove-pending-sources" daily

[cfm]

Status

Ready for review

Description of Changes

Closes #6788.

NB. Per #6780 (comment), we are trusting that well-formed systemd units will Do the Right Thing without specific test cases.

Testing

1. Install securedrop-app-code built via make build-debs on this branch into a production or staging installation.
2. Generate more than 100 "pending" sources, e.g.: loaddata.py --source-count 110 --files-per-source 0 --messages-per-source 0 --replies-per-source 0
3. Wait for the nightly timers to fire.
4. Confirm that only 100 "pending" sources remain on the server.
5. Check that no OSSEC alert was logged when the securedrop-remove-pending-sources timer fired (i.e., its standard output was suppressed). (obsoleted by feat: add systemd timer to run "manage.py remove-pending-sources" daily #6826 (comment))
6. Bonus:
   1. Mangle manage.py remove-pending-sources so that it will raise an exception.
   2. Repeat steps (2) and (3).
   3. Confirm that an OSSEC alert was logged when the securedrop-remove-pending-sources timer fired (i.e., its standard error was not suppressed), because OSSEC triggered on a log message containing the string error. (clarified per feat: add systemd timer to run "manage.py remove-pending-sources" daily #6826 (comment))

Deployment

No deployment considerations, but it's worth documenting this behavior for administrators and flagging it in release notes, as noted below.

Checklist

If you made changes to the server application code:

• Linting (make lint) and tests (make test) pass in the development container

If you made changes to securedrop-admin:

• Linting and tests (make -C admin test) pass in the admin development container

If you made changes to the system configuration:

• Configuration tests pass

If you made non-trivial code changes:

• I have written a test plan and validated it for this PR

Choose one of the following:

• I have opened a PR in the docs repo for these changes, or will do so later
• I would appreciate help with the documentation
• These changes do not require documentation

[nathandyer]

Completed first portion of test plan this afternoon:

• Installed securedrop-app-code from this branch on a production SD install
• Generated more than 100 "pending" sources
• Confirmed that 110 pending sources were created on the server

Going to allow the nighty timers to fire overnight and will return to complete the remainder of the test plan tomorrow.

[cfm]

Thanks, @nathandyer! If you have time, you should be able to rebuild from 6f780c6 and shove it into your system without any other changes or extra testing steps, mainly for #6826 (comment). But it's such a trivial diff I wouldn't worry about it if it's not convenient for your testing. :-)

[nathandyer]

@cfm Not a problem at all to test with the latest changes from 6f780c6!

I am running into something a bit odd, and I think it's just from me not setting things up correctly somehow. From my Admin Workstation, I cloned the SecureDrop git repo and checked out the stg-6788-remove-pending-sources-nightly branch. From there, I did the standard procedure, including a ./securedrop-admin install.

The odd thing is, when I look on my app server itself, manage.py does not have the remove-pending-sources option, and the systemd service to enable the daily run isn't present either.

Is there some other method I should be using to get the newer securedrop-app-code onto my server?

[cfm]

Discussed <#6826 (comment)> out of band with @nathandyer and clarified the test plan to reflect both <#6826 (comment)> and the need to install a securedrop-app-code package built via "make build-debs" on this branch.

[nathandyer]

Thank you @cfm! Happy to report I successfully built the Debian packages and installed them on my server. Verified that manage.py is now the latest version with the remove-pending-sources option, and that the systemd timer is installed and active.

I should have definitive results first thing in the morning.

[nathandyer]

Happy to report that overnight the timer fired as expected and removed 10 of the pending sources, leaving only 100 on the server.

• Confirm that only 100 "pending" sources remain on the server.

Using systemctl I can see the timer is still active and running, and will be triggered again tonight.

This gets my approval, although since I'm not a maintainer we'll still need a final official "Approve" action.

[eaon]

Thanks Nathan! Merging