dom0 clock is behind reality

[legoktm]

Flagged by @rocodes:

INFO:[2024-06-10-23:18:38:891402]   ----------
INFO:[2024-06-10-23:18:38:891429]             ID: install-securedrop-keyring-package
INFO:[2024-06-10-23:18:38:891462]       Function: pkg.installed
INFO:[2024-06-10-23:18:38:891488]         Result: False
INFO:[2024-06-10-23:18:38:891521]        Comment: An error was encountered while installing package(s): E: Release file for https://apt-test.freedom.press/dists/bookworm/InRelease is not valid yet (invalid for another 41s). Updates for this repository will not be applied.
INFO:[2024-06-10-23:18:38:891549]        Started: 23:18:27.396498
INFO:[2024-06-10-23:18:38:891582]       Duration: 3442.29 ms
INFO:[2024-06-10-23:18:38:891608]        Changes:   
INFO:[2024-06-10-23:18:38:891641]   ----------

This is a race condition with apt-test being published to more frequently, but at that time the dom0 clock was 2024-06-10-23:18:38:891521, while apt-test had published at Date: Mon, 10 Jun 2024 23:19:59 UTC.

The clock is probably off, which is somewhat expected since we just booted the VM. We should force a sudo qvm-sync-clock before provisioning.

[legoktm]

Happened again in https://ws-ci-runner.securedrop.org/2024-06-26-191634739526-cbb6ad92028564b062e7412c49df722fc2f22332-Qubes_4.2_A-update_20240626043626.log.txt

E: Release file for tor+https://fasttrack.debian.net/debian/dists/bookworm-fasttrack/InRelease is not valid yet (invalid for another 7s). Updates for this repository will not be applied.

Not sure if whonix is special in this regard.

[legoktm]

https://ws-ci-runner.securedrop.org/2024-06-28-193030038253-02500bf950d67b6fd7433a0271dc8a76b3f3961c-Qubes_4.2_A-update_20240628045353.log.txt

INFO:[2024-06-28-19:59:31:338170]             ID: upgrade-all-packages
INFO:[2024-06-28-19:59:31:338206]       Function: pkg.uptodate
INFO:[2024-06-28-19:59:31:338234]         Result: False
INFO:[2024-06-28-19:59:31:338279]        Comment: E: Release file for tor+https://fasttrack.debian.net/debian/dists/bookworm-fasttrack/InRelease is not valid yet (invalid for another 41s). Updates for this repository will not be applied.
INFO:[2024-06-28-19:59:31:338311]        Started: 19:59:18.051638
INFO:[2024-06-28-19:59:31:338346]       Duration: 4095.405 ms
INFO:[2024-06-28-19:59:31:338375]        Changes:   
INFO:[2024-06-28-19:59:31:338410]   ----------
INFO:[2024-06-28-19:59:31:338438]             ID: install-securedrop-keyring-package
INFO:[2024-06-28-19:59:31:338474]       Function: pkg.installed
INFO:[2024-06-28-19:59:31:338503]         Result: False
INFO:[2024-06-28-19:59:31:338538]        Comment: An error was encountered while installing package(s): E: Release file for tor+https://fasttrack.debian.net/debian/dists/bookworm-fasttrack/InRelease is not valid yet (invalid for another 37s). Updates for this repository will not be applied.
INFO:[2024-06-28-19:59:31:338568]        Started: 19:59:22.152466
INFO:[2024-06-28-19:59:31:338603]       Duration: 2104.76 ms
INFO:[2024-06-28-19:59:31:338632]        Changes:   

[legoktm]

Not sure if whonix is special in this regard.

• https://www.kicksecure.com/wiki/Boot_Clock_Randomization
• https://www.whonix.org/wiki/Dev/TimeSync#Qubes_Time_Synchronization_Features

[rocodes]

According to the whonix docs, this feature can be disabled via sudo systemctl mask bootclockrandomization. A reboot is required afterwards.

The initial impetus for the feature is for anti-fingerprinting purposes, but there has been much discussion over its implementation, and also upstream discussion of breakage and whether to make the feature opt-out or opt-in.

Initially I was going to just propose disabling it on CI (easy/uncontroversial; we would apply the change, reboot the ci machine, then start from that change before doing our own sdw provisioning). But, per discussion in the forum and qubes repo, and knowing that Whonix updates are a frequent source of updater failures and therefore a big user pain point, it could be worth discussing disabling this on prod machines.

To make that decision I think we'd need more information on a couple things: 1) how viable an anti-fingerprinting method is it really (see forum post for discussion) and 2) if it's a viable anti-fingerprinting technique, is there a case for that for our journalist users (and is it worth the sometimes breakage). I haven't looked into this or even where the setting is applied, but if for example we were able to apply this setting to just sd-whonix, leaving the default whonix configuration untouched on prod machines, that would be a pretty uncontroversial change.

[rocodes]

I think we should prioritize freedomofpress/securedrop-workstation#1060, or at least the return to qvm.anon-whonix, and then assess. We may also need to examine the way fpf_apt_repo.sls works in sd-whonix - official whonix docs advise against updating using apt update, for example.

To expand a bit on my comment: We are downloading whonix templates, but we aren't running the recommended/provided qvm.anon-whonix Salt state afterwards. This does things that include adding tags (which, among other things, govern the qubes.GetDate RPC policy for whonix vms....), and which ensure that whonix vms are set up in the tested, upstream-expected manner. If after making this change, we still see regular failures of this kind, we can look again at clock randomization etc, but we may be causing a problem here by deviating from the expected whonix setup.