[wip] perf: speed up metadata sync time

[redshiftzero]

Description

Work in progress

This speeds up the metadata sync (15s -> 2s) and ensures that duplicate jobs don't get added since download actions from the previous metadata sync may not have completed by the time that the next one kicks off (if we don't add a bunch of duplicated jobs, then that's not a problem).

Closes #738
Closes #116
Related to #800 and #468

Test Plan

1. Slow down downloads server-side:

diff --git a/securedrop/journalist_app/api.py b/securedrop/journalist_app/api.py
index cf6d76585..d6ec1f91b 100644
--- a/securedrop/journalist_app/api.py
+++ b/securedrop/journalist_app/api.py
@@ -187,6 +187,8 @@ def make_blueprint(config):
                methods=['GET'])
     @token_required
     def download_submission(source_uuid, submission_uuid):
+        import time
+        time.sleep(20)
         get_or_404(Source, source_uuid, column=Source.uuid)
         submission = get_or_404(Submission, submission_uuid,
                                 column=Submission.uuid)

2. Run client with debug logs: LOGLEVEL=DEBUG ./run.sh
3. Tail logs and see that duplicate jobs aren't added:

2020-03-04 10:34:49,217 - securedrop_client.queue:80(add_job) DEBUG: Duplicate job <securedrop_client.api_jobs.downloads.MessageDownloadJob object at 0x113b15050>, skipping
2020-03-04 10:34:49,217 - securedrop_client.queue:80(add_job) DEBUG: Duplicate job <securedrop_client.api_jobs.downloads.MessageDownloadJob object at 0x113b15050>, skipping
2020-03-04 10:34:49,217 - securedrop_client.queue:80(add_job) DEBUG: Duplicate job <securedrop_client.api_jobs.downloads.MessageDownloadJob object at 0x113b15050>, skipping
2020-03-04 10:34:49,218 - securedrop_client.queue:80(add_job) DEBUG: Duplicate job <securedrop_client.api_jobs.downloads.MessageDownloadJob object at 0x113b15050>, skipping
2020-03-04 10:34:49,219 - securedrop_client.queue:80(add_job) DEBUG: Duplicate job <securedrop_client.api_jobs.downloads.ReplyDownloadJob object at 0x113b15050>, skipping
2020-03-04 10:34:49,219 - securedrop_client.queue:80(add_job) DEBUG: Duplicate job <securedrop_client.api_jobs.downloads.ReplyDownloadJob object at 0x113b15050>, skipping
2020-03-04 10:34:49,219 - securedrop_client.queue:80(add_job) DEBUG: Duplicate job <securedrop_client.api_jobs.downloads.ReplyDownloadJob object at 0x113b15050>, skipping
2020-03-04 10:34:49,219 - securedrop_client.queue:80(add_job) DEBUG: Duplicate job <securedrop_client.api_jobs.downloads.ReplyDownloadJob object at 0x113b15050>, skipping

Checklist

If these changes modify code paths involving cryptography, the opening of files in VMs or network (via the RPC service) traffic, Qubes testing in the staging environment is required. For fine tuning of the graphical user interface, testing in any environment in Qubes is required. Please check as applicable:

• I have tested these changes in the appropriate Qubes environment
• I do not have an appropriate Qubes OS workstation set up (the reviewer will need to test these changes)
• These changes should not need testing in Qubes
• I have not tested thoroughly in qubes but I will (currently reprovisioning on prod)

If these changes add or remove files other than client code, packaging logic (e.g., the AppArmor profile) may need to be updated. Please check as applicable:

• I have submitted a separate PR to the packaging repo
• No update to the packaging logic (e.g., AppArmor profile) is required for these changes
• I don't know and would appreciate guidance

[redshiftzero]

Picking #116 into another branch, closing this for now