Merge pull request #248 from freedomofpress/245-first-run-for-qubes-4…

….0.1

Updates docs and config for Qubes 4.0.1

README.md

@@ -36,11 +36,11 @@ While we are strongly committed to piloting the use of Qubes OS for SecureDrop,
 
 Installing this project is involved. It requires an up-to-date Qubes 4.0 installation running on a machine with at least 12GB of RAM. You'll need access to a SecureDrop staging server as well.
 
-#### Qubes 4.0
+#### Qubes 4.0.1
 
-Before trying to use this project, install [Qubes 4.0](https://www.qubes-os.org/downloads/) on your development machine. Accept the default VM configuration during the install process.
+Before trying to use this project, install [Qubes 4.0.1](https://www.qubes-os.org/downloads/) on your development machine. Accept the default VM configuration during the install process.
 
-After installing Qubes, you must update both dom0 and debian-9 template VM to include the latest version of the `qubes-kernel-vm-support` package.
+After installing Qubes, you must update both dom0 and the base templates to include the latest versions of apt packages.
 
 ##### `dom0`
 
@@ -50,20 +50,14 @@ Open a terminal in `dom0` by clicking on the Qubes menu top-right of the screen
 sudo qubes-dom0-update
 ```
 
-##### `debian-9`
-Open a terminal in the `debian-9` TemplateVM and run:
+Finally, update all existing TemplateVMs:
 
 ```
-sudo apt-get update
-sudo apt-get upgrade
-apt-cache policy qubes-kernel-vm-support
+qubes-update-gui
 ```
 
-After verifying that the latest version of `qubes-kernel-vm-support` is installed, shut down the template VM:
+Select all VMs marked as **updates available**, then click **Next**. Once all updates have been applied, you're ready to proceed.
 
-```
-sudo poweroff
-```
 
 #### Download, Configure, Copy to `dom0`
 
@@ -106,62 +100,6 @@ qfile-agent : Fatal error: File copy: Disk quota exceeded; Last file: <...> (err
 
 When the installation process completes, a number of new VMs will be available on your machine, all prefixed with `sd-`.
 
-Proceed to the following steps to clean up templates on workstation, which are necessary due to the inclusion of end-of-life templates in Qubes 4.0.
-
-##### Upgrading `sys-net`, `sys-usb` and `sys-firewall` to `fedora-28`
-
-Qubes 4.0 ships with end-of-life `fedora-26` templates which are used by default for `sys-net`, `sys-firewall` and `sys-usb`. You need to manually upgrade `sys-net`, `sys-firewall` and `sys-usb` VMs to `fedora-28` by running the following commands in `dom0`:
-
-```
-qvm-kill sys-net
-qvm-prefs sys-net template fedora-28
-qvm-start sys-net
-
-qvm-kill sys-firewall
-qvm-prefs sys-firewall template fedora-28
-qvm-start sys-firewall
-
-qvm-kill sys-usb
-qvm-prefs sys-usb template fedora-28
-qvm-start sys-usb
-
-```
-
-Any other `fedora-26` VMs you may have created or that are installed by default (`work`, `personal`, `untrusted`, `vault`) should also be updated in the same manner.
-
-You will also need to update the default disposable VM template to `fedora-28`:
-
-```
-qvm-create --template fedora-28 --label red fedora-28-dvm
-qvm-prefs fedora-28-dvm template_for_dispvms True
-qubes-prefs default_dispvm fedora-28-dvm
-```
-
-You can then delete the end-of-life `fedora-26` template in `dom0` by running:
-
-```
-sudo dnf remove qubes-template-fedora-26
-```
-
-If this command produces an error, open the Qube Manager and ensure that there are no remaining VMs using the `fedora-26` template.
-
-#### Upgrading `sys-whonix` and `whonix-ws` AppVMs to Whonix 14
-
-Qubes 4.0 also ships with end-of-life Whonix templates (`whonix-gw` and `whonix-ws`).`sys-whonix` is used by `sd-whonix` to fetch updates, and should be upgraded. You should destroy `whonix-gw` from the Qube Manager and re-provision a new `sys-whonix` AppVM based on `whonix-gw-14` with the option **provides network**. You will need to delete the `whonix-ws-dvm` and `anon-whonix` VMs. You can then remove the end-of-life templates by running the following commands in `dom0`:
-
-```
-sudo dnf remove qubes-template-whonix-gw
-sudo dnf remove qubes-template-whonix-ws
-```
-
-Upon release, Qubes 4.0.1 will no longer ship `fedora-26` or older Whonix templates, and the above steps will no longer be necessary.
-
-Finally, update all the templates and reboot the machine. Your workstation will then be ready for use. In `dom0`, run:
-
-```
-sudo securedrop-update
-```
-
 #### Using the *SecureDrop Client*
 
 **note:** the version of the client shipped here is currently unable to communicate with the server. Please use the [development environment](https://docs.securedrop.org/en/latest/development/setup_development.html?highlight=development#qubes) of [securedrop-client](https://github.com/freedomofpress/securedrop-client/) to test it instead, as more recent versions have that problem fixed.
@@ -250,8 +188,8 @@ qvm-copy-to-vm sd-export ~/.securedrop_client/data/name-of-file
 The development plan is to provide functionality in the *SecureDrop Client* that automates step 3, and assists the user in taking these steps via GUI prompts. Eventually we plan to provide other methods for export, such as [OnionShare](https://onionshare.org/) (this will require the attachment of a NetVM), using a dedicated export VM template with tools such as OnionShare and Veracrypt. The next section includes instructions to approximate the OnionShare sharing flow.
 
 ##### Transferring files via OnionShare
-1. Create an `sd-onionshare-template` VM based on `fedora-28`:
-   1. Click on the Qubes menu in the upper left, select "Template: Fedora 28", click on "fedora-28: Qube Settings", and click on **Clone Qube**
+1. Create an `sd-onionshare-template` VM based on `fedora-29`:
+   1. Click on the Qubes menu in the upper left, select "Template: Fedora 29", click on "fedora-29: Qube Settings", and click on **Clone Qube**
    2. Name the cloned qube `sd-onionshare-template`
    3. In the Qubes menu on the top-left, select "Template: sd-onionshare-template" and click on "sd-onionshare-template: Terminal"
    4. Install OnionShare: `sudo dnf install onionshare`
@@ -366,7 +304,7 @@ Be aware that running tests *will* power down running SecureDrop VMs, and may re
 
 ## Building the Templates
 
-1. Create a `fedora-28` AppVM for building
+1. Create a `fedora-29` AppVM for building
 2. Increase the disk size to at least 15GB (as the build uses over 10GB)
 3. Import the QubesOS master key and the GPG key used to sign tags (see https://www.qubes-os.org/security/verifying-signatures/)
 4. Run `make template` in the top-level of this repository.

dom0/fpf-apt-test-repo.sls

@@ -1,13 +1,28 @@
 # -*- coding: utf-8 -*-
 # vim: set syntax=yaml ts=2 sw=2 sts=2 et :
 
+# Handle misconfigured jessie-backports repo in default debian-9 TemplateVM.
+# The Jessie repos aren't maintained anymore, and their inclusion causes
+# even apt update to fail.
+remove-jessie-backports-repo:
+  file.line:
+    - name: /etc/apt/sources.list
+    # Unclear why "Delete" *must* be capitalized, but that's the case!
+    - mode: delete
+    - match: jessie-backports
+    # quiet param seems to be ignored, so using "onlyif" to test existence
+    - quiet: True
+    - onlyif:
+      - test -f /etc/apt/sources.list
 
 # That's right, we need to install a package in order to
 # configure a repo to install another package
 install-python-apt-for-repo-config:
   pkg.installed:
     - pkgs:
       - python-apt
+    - require:
+      - file: remove-jessie-backports-repo
 
 configure apt-test apt repo:
   pkgrepo.managed:

dom0/sd-dom0-files.sls

@@ -45,10 +45,10 @@ dom0-securedrop-icon:
       - file: dom0-securedrop-icons-directory
 
 # Install latest templates required for SDW VMs.
-dom0-install-fedora-28-template:
+dom0-install-fedora-29-template:
   pkg.installed:
     - pkgs:
-      - qubes-template-fedora-28
+      - qubes-template-fedora-29
 
 dom0-install-whonix-14-templates:
   pkg.installed:

dom0/securedrop-update

@@ -42,7 +42,7 @@ securedrop-update-feedback "Updating application..."
 # update only fedora template: dist_upgrade is required for debian package
 # upgrades and causes fedora template upgrades to fail.
 
-qubesctl --target fedora-28 pkg.upgrade refresh=true
+qubesctl --target fedora-29 pkg.upgrade refresh=true
 
 # upgrade all (other) templates
 qubesctl --skip-dom0 --templates \

tests/test_vms_platform.py

@@ -44,20 +44,47 @@ def _ensure_packages_up_to_date(self, vm, fedora=False):
         updates. Assumes VM is Debian-based, so uses apt, but supports
         `fedora=True` to use dnf instead.
         """
+        # Create custom error message, so failing test cases display
+        # which VM caused the looped check to fail.
+        fail_msg = "Unapplied updates for VM '{}'".format(vm)
         if not fedora:
             cmd = "apt list --upgradable"
             stdout, stderr = vm.run(cmd)
             results = stdout.rstrip().decode("utf-8")
             # `apt list` will always print "Listing..." to stdout,
             # so expect only that string.
-            self.assertEqual(results, "Listing...")
+            self.assertEqual(results, "Listing...", fail_msg)
         else:
             cmd = "sudo dnf check-update"
             # Will raise CalledProcessError if updates available
             stdout, stderr = vm.run(cmd)
             # 'stdout' will contain timestamped progress info; ignore it
             results = stderr.rstrip().decode("utf-8")
-            self.assertEqual(results, "")
+            self.assertEqual(results, "", fail_msg)
+
+    def _ensure_jessie_backports_disabled(self, vm):
+        """
+        Ensures that there are no Debian Jessie repos configured in
+        apt source lists. This is only relevant for Debian Stretch,
+        on which misconfigured apt sources containing Jessie references
+        will cause apt commands to fail.
+        """
+        # Use a fileglob to account for /etc/apt/sources.list.d/, as well.
+        # Add `|| true` to ensure dom0 receives a zero exit code.
+        cmd = "grep -i jessie /etc/apt/sources.list* || true"
+        # Will raise CalledProcessError if no hits found
+        stdout, stderr = vm.run(cmd)
+        results = stdout.rstrip().decode("utf-8")
+        # We expect zero hits, so confirm output is empty string.
+        self.assertEqual(results, "")
+
+    def test_all_jessie_backports_disabled(self):
+        """
+        Asserts that all VMs lack references to Jessie in apt config.
+        """
+        for vm_name in WANTED_VMS:
+            vm = self.app.domains[vm_name]
+            self._ensure_jessie_backports_disabled(vm)
 
     def test_all_sd_vms_uptodate(self):
         """
@@ -75,8 +102,8 @@ def test_all_fedora_vms_uptodate(self):
         """
         # Technically we want to know whether the sys-firewall, sys-net, and
         # sys-usb VMs have their updates installed. This test assumes those
-        # AppVMs are based on fedora-28.
-        vm_name = "fedora-28"
+        # AppVMs are based on fedora-29.
+        vm_name = "fedora-29"
         vm = self.app.domains[vm_name]
         self._ensure_packages_up_to_date(vm, fedora=True)
         vm.shutdown()
@@ -113,7 +140,7 @@ def test_dispvm_default_platform(self):
         """
         cmd = ["qubes-prefs", "default_dispvm"]
         result = subprocess.check_output(cmd).decode("utf-8").rstrip("\n")
-        self.assertEqual(result, "fedora-28-dvm")
+        self.assertEqual(result, "fedora-29-dvm")
 
 
 def load_tests(loader, tests, pattern):

tests/vars/qubes-rpc.yml

@@ -19,12 +19,12 @@
 
 - policy: GetDate
   starts_with: |-
-    $tag:anon-vm $anyvm deny
     ## Note that policy parsing stops at the first match,
     ## so adding anything below "$anyvm $anyvm action" line will have no effect
 
     ## Please use a single # to start your custom comments
 
+    $tag:anon-vm	$anyvm	deny
     $anyvm	$anyvm	allow,target=dom0
 
 - policy: GetRandomizedTime
@@ -131,13 +131,20 @@
 
 - policy: UpdatesProxy
   starts_with: |-
-    $tag:whonix-updatevm $default allow,target=sys-whonix
-    $tag:whonix-updatevm $anyvm deny
     ## Note that policy parsing stops at the first match,
     ## so adding anything below "$anyvm $anyvm action" line will have no effect
 
     ## Please use a single # to start your custom comments
 
+    # Upgrade all TemplateVMs through sys-whonix.
+    #$type:TemplateVM $default allow,target=sys-whonix
+
+    # Upgrade Whonix TemplateVMs through sys-whonix.
+    $tag:whonix-updatevm $default allow,target=sys-whonix
+
+    # Deny Whonix TemplateVMs using UpdatesProxy of any other VM.
+    $tag:whonix-updatevm $anyvm deny
+
     # Default rule for all TemplateVMs - direct the connection to sys-net
     $type:TemplateVM $default allow,target=sys-net