Investigate Document -> Journalist Interface renaming side-effects

[conorsch]

In #1395 the "Document Interface" was renamed to "Journalist Interface". There are a few problems with the current implementation, chiefly that there's no migration strategy in place for currently running instances.

If we ship that change to the Ansible playbooks, then production instances will generate a new Authenticated Tor Hidden Services (ATHS) on next run, and lock out Journalists from the interface by updating the Apache vhost to listen to the new service, using a HidServAuth value that Journalists won't have on their workstations.

Currently the app-prod machine is broken in the develop branch, and fails during the provisioning run, due to the AppArmor profile changes dropping support for the old vhost file. That kind of makes sense: the prod machines install from the FPF apt repo, and changes in e.g. #1464 are intended to provide a more robust staging environment that would allow for testing the new changes. It makes running app-prod impossible right now, though, which has terrible implications for pre-release testing.

So the question becomes: do we want to commit to the rename and write migration tasks, or back out the changes to simplify the scope of the 0.4 release?

[heartsucker]

On the server side, I believe this is easily fixed via a preinst script addition on new packages that can move the hidden service dirs. Ensuring that admins have up to date playbooks that don't overwrite these changes might be a bit trickier (for example in install_files/ansible-base/roles/tor-hidden-services/tasks/configure_tor_hidden_services.yml).

How much leeway is there expected to be with admins running old versions of playbooks?

[conorsch]

@heartsucker Leveraging the Debian package scripts is really our only option here, short of backing out the changes (which I'm loath to do), so I agree. We'll need to straighten out the package build scripts in order to test this adequately, e.g. #1612 and #1464.

[conorsch]

#1464 and #1612 are merged, so drilling down on this now.

[conorsch]

In order to test the upgrade path here, we should:

1. Provision staging VMs based on the 0.3.12 release tag. Doing so will ensure we get the 0.3.12 packages from the apt repo, and the old Ansible config, referencing the "Document Interface," will be used to configure the tor and apache services.
2. Check out a feature branch based on current develop and patch the preinst script to handle the migration.
3. Build new deb packages with the patched preinst script and evaluate success.

The safest approach to testing 3 is to use a staging apt server; that way, only the cron-apt logic will trigger the update, which is an adequate test for how production instances will respond to the upgrade. We don't have apt server logic in this repo, so I'll spin that up separately and report back.

As far as what tasks the preinst script will need to handle, off the top of my head the following are necessary:

• Move ATHS directory /var/lib/tor/services/document/ -> journalist/.
• Update associated HiddenServiceDir line in /etc/tor/torrc.
• Move Apache vhost file /etc/apache2/sites-available/document.conf -> journalist.conf
• Update contents of old Apache vhost file to point to renamed log files.
• Clean up dangling references to old configs, e.g. symlinks for Apache vhosts.

That's a hefty list of changes for preinst, but let's see if it's feasible. Will report back with results from staging apt server upgrade.

[conorsch]

Currently testing a preinst-based migration strategy locally. If all goes well, will submit a PR requesting verification of the testing workflow I used.