Converts sd-proxy to SDW template base

[conorsch]

Status

Ready for review

Description of Changes

Refs #459, #456.

Changes proposed in this pull request:

• Converts sd-proxy to use the securedrop-workstation-buster template, rather than Whonix Gateway 15
• Updates config tests to match

We do this in order to speed up the time it takes for updates to run (#459), as well as nudge us toward full template consolidation (#471). Since sd-proxy still has its NetVM set to sd-whonix, it's able to resolve Onion URLs just fine.

The diff is quite simple here. The proof is in the test plan.

Testing

N.B. We tried to make this change a while back, and abandoned it due to client sync frequency. See related comment here: #206 (comment) (hat tip to @emkll for digging up that gem). Given the dramatic improvements to client sync functionality since then, we should be in a fine position to move forward now. Still, the test plan includes mention of idle state to make sure.

You'll need to apply the workaround #485 (comment) in order to review effectively.

Client connectivity

• make clone && make clean && make all && make test passes without error
• Run the client (OK to run securedrop-client directly in a terminal on sd-app, to sidestep the preflight updater)
• Log in as usual, confirm no errors
• Interact with test source and submissions, open a few in DVMs to confirm download working
• Leave the client idle for at least 10 minutes. This is to ensure that despite the networking change, the client is able to communicate with the server over time
• Return to the client, interacting with new sources, downloading new submissions, and confirm no networking errors

Update time

Next, let's confirm the assumption that apt updates are significantly faster.

• Open a terminal on sd-proxy-buster-template and run time { sudo apt-get update && sudo apt-get install -y firefox-esr; }, record the time
• Open a terminal on sd-proxy and run time { sudo apt-get update && sudo apt-get install -y firefox-esr; }, record the time

There are the times I observed when testing locally:

# sd-proxy-buster-template (clearnet)
real	0m24.228s
user	0m7.873s
sys	0m3.150s

# sd-proxy (tor)
real	2m18.338s
user	0m9.796s
sys	0m3.799s

Note that you'll need to edit dom0:/etc/qubes-rpc/policy/qubes.ClipboardPaste if you wish to copy/paste your results here.

Checklist

If you have made code changes

• Linter (make flake8) passes in the development environment (this box may
  be left unchecked, as flake8 also runs in CI)

If you have made changes to the provisioning logic

• All tests (make test) pass in dom0 of a Qubes install

• This PR adds/removes files, and includes required updates to the packaging
  logic in MANIFEST.in and rpm-build/SPECS/securedrop-workstation-dom0-config.spec

[eloquence]

Is it possible that the timeout behavior difference will impact sd-proxy while the client is not running at all, e.g., if the user shuts down the client and does other work, without shutting down sd-proxy? If so, that might be worth testing as well.

[emkll]

Thanks @conorsch this looks good to me. While I didn't observe similar gains to you, Visual diff and performed functional testing as follows:

Client connectivity

• make clone && make clean && make all && make test passes without error
• Run the client (OK to run securedrop-client directly in a terminal on sd-app, to sidestep the preflight updater)
• Log in as usual, confirm no errors
• Interact with test source and submissions, open a few in DVMs to confirm download working
• Leave the client idle for at least 10 minutes. This is to ensure that despite the networking change, the client is able to communicate with the server over time
• Return to the client, interacting with new sources, downloading new submissions, and confirm no networking errors

Update time

Next, let's confirm the assumption that apt updates are significantly faster.

• Open a terminal on sd-proxy-buster-template and run time { sudo apt-get update && sudo apt-get install -y firefox-esr; }, record the time :

real	0m35.856s
user	0m9.396s
sys	0m3.717s

• Open a terminal on sd-proxy and run time { sudo apt-get update && sudo apt-get install -y firefox-esr; }, record the time

real	1m18.341s
user	0m9.268s
sys	0m3.897s

Also tested reliability as follows:

• Put the workstation to sleep, client functionality works upon resuming from sleep (though not a scenario we should hit in production)
• Per @eloquence 's comment above, closes securedrop-client, waited 15 minutes and restarted client, functionality and connectivity work as expected
• Logs are correctly sent to sd-log

It might be good to merge #489 before cutting as new release as it will help with QA.

[rmol]

Client connectivity

• make clone && make clean && make all && make test passes without error
• Run the client (OK to run securedrop-client directly in a terminal on sd-app, to sidestep the preflight updater)
• Log in as usual, confirm no errors
• Interact with test source and submissions, open a few in DVMs to confirm download working
• Leave the client idle for at least 10 minutes. This is to ensure that despite the networking change, the client is able to communicate with the server over time
• Return to the client, interacting with new sources, downloading new submissions, and confirm no networking errors

Update time

Next, let's confirm the assumption that apt updates are significantly faster.

• Open a terminal on sd-proxy-buster-template and run time { sudo apt-get update && sudo apt-get install -y firefox-esr; }, record the time
• Open a terminal on sd-proxy and run time { sudo apt-get update && sudo apt-get install -y firefox-esr; }, record the time

There are the times I observed when testing locally:

# securedrop-proxy-buster-template
real	0m26.498s
user	0m9.017s
sys	0m3.454s

# sd-proxy
real	1m32.581s
user	0m18.107s
sys	0m5.774s