added --preserve-tor-config flag to securedrop-admin restore

[zenmonkeykstop]

Status

Ready for review

Description of Changes

Fixes #5107

Adds a --preserve-tor-config flag to the securedrop-admin restore command, that, if set, will restore a backup tarball without overwriting the server's existing Tor configurations. This will allow a dataset from an SD instance to be applied and available on another instance at a different onion address.

Sets flake8 max-line-length to 100, in line with standard for application code.

Testing

On prod instance (VMs or hardware):

• install latest release version, create JI accounts and add source submissions
• from the Admin Workstation, take a backup using ./securedrop-admin backup
• copy the initial install's THS, auth_private, and tor_v3_keys.jsonfiles from theinstall_files/ansible-base` directory to ~/Persistent/
• wipe the servers and reinstall from scratch, thereby generating new Tor service definitions
• check out this branch and restore from backup using the command:

  ./securedrop-admin restore --preserve-tor-config  <backupfilehere>
  

• confirm that the sources, submissions, and accounts from the initial install are present
• confirm that none of the Tor service addresses from the second install have changed
• next, delete all sources, then restore from backup using the command:

  ./securedrop-admin restore  <backupfilehere>
  

• The restore command will time out, as the SSH connection details will change. When it does, log into the application server from the console and issue the command sudo service tor reload to load the new service config.
• copy the initial install's THS, auth_private, and tor_v3_keys.jsonfiles from ~/Persistent/ to theinstall_files/ansible-base` directory, and run:

  ./securedrop-admin tailsconfig
  

  to restore the original local Tor config
• Confirm that the sources, submissions, and accounts from the initial install are present
• Confirm that the initial install's hidden services are available at their original addresses.

Deployment

(deployed when workstation copies of the repo are updated)

Checklist

If you made changes to securedrop-admin:

• Linting and tests (make -C admin test) pass in the admin development container

If you made non-trivial code changes:

• I have written a test plan and validated it for this PR

[emkll]

Thanks @zenmonkeykstop the diff looks good to me visually, but ran into some issues with (likely unrelated to this PR) with tailsconfig. Tested as follows, on prod VMs:

On prod instance (VMs or hardware):

• install latest release version, create JI accounts and add source submissions
• from the Admin Workstation, take a backup using ./securedrop-admin backup
• copy the initial install's THS, auth_private, and tor_v3_keys.jsonfiles from theinstall_files/ansible-base` directory to ~/Persistent/
• wipe the servers and reinstall from scratch, thereby generating new Tor service definitions
• check out this branch and restore from backup using the command:

  ./securedrop-admin restore --preserve-tor-config  <backupfilehere>
  

• confirm that the sources, submissions, and accounts from the initial install are present
• confirm that none of the Tor service addresses from the second install have changed
• next, delete all sources, then restore from backup using the command:

  ./securedrop-admin restore  <backupfilehere>
  

• The restore command will time out, as the SSH connection details will change. When it does, log into the application server from the console and issue the command sudo service tor reload to load the new service config.
• copy the initial install's THS, auth_private, and tor_v3_keys.jsonfiles from ~/Persistent/ to theinstall_files/ansible-base` directory, and run:

  ./securedrop-admin tailsconfig
  

  to restore the original local Tor config
• Confirm that the sources, submissions, and accounts from the initial install are present
• Confirm that the initial install's hidden services are available at their original addresses.

I seem to be running into an issue with tailsconfig after restoring the original backup without preserving tor. Running ./securedrop-admin tailsconfig will always provision shortcuts with the second clean install's tor service settings (even if I delete tor_v3_keys.json, app-journalist.auth_private and app-sourcev3-ths, and even if i disable v3 in site-specific). Placing the first install's tor settings has no effect, and the Tor config is not populated correctly. Anything I could be missing here?

For the step:

The restore command will time out, as the SSH connection details will change. When it does, log into the application server from the console and issue the command sudo service tor reload to load the new service config.

Should we automate this step with ansible? It sounds like something a user would easily forget.

[emkll]

Turns out the issues I described were because I copied the backed-up files to the wrong directory 🤦‍♂️ . The test plan was completed successfully 🎉

One last question before formally approving:
Should we update the docs? It seems we won't be able to automate the sudo tor service reload after restoring the backup, when using SSH over tor. It might be worth describing these steps in the docs (either dev docs or user-facing docs, if there's a compelling use-case)

[zenmonkeykstop]

This feature will support testing on the workstation side initially, and won't go out with the next point release. I'm inclined to leave it undocumented for now and fix the tor config handling for 1.3.0, because I don't want to make the existing backup documentation more complex and dissuade people from using it. But if you disagree, I'm not fundamentally opposed.

[emkll]

Since this operation is not destructive, and the help message seems sufficiently clear, I think that's fair. Thanks @zenmonkeykstop