update of newer debian-based template fails because of dsa4371 check in qube-manager

[xaki23]

Qubes OS release

4.1

Brief summary

updating kali template from qubes manager fails because of dsa4371 check

Steps to reproduce

• install kali template on 4.1
• rightclick-update it in qubes-manager
• watch it fail with yet another opaque dsa4371 message

Expected behavior

working updates

Actual behavior

cryptic error message

root cause

the problem in this case turned out to be the "Message from the Kali developers" that newer kali versions show on each spawned shell. (which is bad in lots of other situations too, like rsync)

workaround

touch /root/.hushlogin

recommended solution

can we please get rid of the dsa4371 check in 4.1?
dsa4371 was 2019-01 and there never were any templates with this issue for 4.1.
and if you build your own lenny template, well, good luck with that on so many levels.
i dont think i have seen any qubes component fail in more different ways over the last years than the dsa4371 check...

[unman]

The 4371 check was removed last year, wasn't it? It doesn't seem to be present in my templates. Where did you get the Kali template you are using?

[xaki23]

The 4371 check was removed last year, wasn't it? It doesn't seem to be present in my templates.

the check lives in dom0 (qubes-manager pkg) and is pushed to the VMs via qubes.VMshell service as needed.
it is still very much present in my fully updated 4.1 dom0.

Where did you get the Kali template you are using?

https://ftp.qubes-os.org/repo/yum/r4.1/templates-community-testing/rpm/qubes-template-kali-4.0.6-202106171816.noarch.rpm

the root cause here is clearly kali doing something silly that breaks a lot of things, but "i cant update my template through rightclick in qubes-manager" is one of the more annoying impacts.

[xaki23]

not sure about what "needs diagnosis" here tbh.

what happens is ... the code that runs the dsa4371 check is this:

https://github.com/QubesOS/qubes-manager/blob/24d8c230fda1c28d71377822b3c4b44994ce94fa/qubesmanager/qube_manager.py#L610
lines 610-627

the way it runs the check is L 611-616.
replicating that on the commandline, what happens on kali is this:

[admin@dom0 ~]$ qvm-run -u root -p --service kali-20210617 qubes.VMShell < /usr/libexec/qubes-manager/dsa-4371-update 
______(_[1;31mMessage from Kali developers_[00m)
___
___ We have kept /usr/bin/python pointing to Python 2 for backwards
___ compatibility. Learn how to change this and avoid this message:
___ ___ https://www.kali.org/docs/general-use/python3-transition/
___
______(_[2;37mRun: ___touch ~/.hushlogin___ to hide this message_[0m)
changed=no
Ok: Unrecognized debian variant, but probably ok by now
[admin@dom0 ~]$ 

the last line of that is stderr, the rest is stdout.

qubes-manager then compares the stdout to the expected results in lines 617 and 622, which fails because of that silly kali banner. (it is a really, really bad plan to have any kind of banners spew on non-interactive shells)

because there is no match, it goes into the else branch in L 624, which raises an error that doesnt really point at the issue because it prints stderr (which is as expected), even though it failed because of stdout contents.
while debugging this, i changed that raised error text to stdout, which made qubes manager crash hard because the .decode(ascii) pops hard when it encounters the ANSI color code sequences in the banner.

the banner also points out how to disable it (once you get to see it), thats the workaround mentioned in the initial report.

i stand by the suggestion to just remove the dsa4371 workaround entirely on 4.1 at this point though.
even under most pessimistic pov, i really dont see it ever doing anything good on 4.1, and it is fragile in so many ways that it will just keep blowing up in different ways every other year.

i also just had a short talk with @fepitre about this on irc and suggested he just touches that .hushlogin file on the next kali build.
unless someone sees an actual reason to keep the dsa4371 check on 4.1, i will just roll a PR for removing it.

[xaki23]

and what i mean by the dsa4371 code in qubes-manager being fragile ...

if i replace the contents of /etc/qubes-rpc/qubes.VMShell inside a VM by ...

#!/bin/bash
echo -e "\xe2" 1>&2
echo boo

(this makes the qubes.VMShell service in that VM print a binary 0xe2 on stderr and boo on stdout)

if after that, i hit rightclick-update on that VM in qubes-manager ... qubes-manager just crashes.
afaict this is not exploitable, but "something inside a VM can make administrative tools in dom0/adminVM crash" is rarely a good thing.

and there is a confusing artifact to this:

• if i started qubes-manager from the Q-menu, it crashes.
• if i started it from a dom0 terminal (because i wanted to see the python exception of the crash to include it in this post), it doesnt.

(it was printing the crash-backtrace after starting from a terminal when i was debugging the kali issue earlier, and i cant really figure out whats the difference now. it doesnt seem to be ENV related.)

[unman]

I should apologise for the fact that I completely missed your clear statement that this was updating from Qube Manager. This is a clear error - although there is also #6635 which would resolve this by not having a completely separate mechanism from Manager.

[xaki23]

another case: the focal-20210924 template by @unman can not be rightclick-updated because the DSA-4371 check cant figure out the installed apt version (looks like renamed packages).

adjusted issue title.

i would prefer to keep the "old" update path currently. there recently was a case where the salt update path was broken, and running the "old" update once made the salt updates (via updatetool) work again.

unless someone sees a real reason to keep the DSA-4371 check, i will file a PR to remove it in the next days.

[TheDarkTrumpet]

I recently pulled the Kali community template, and it worked fine, but ran into the same issue being discussed here. So right now unable to update the qube due to this issue.

[unman]

On Wed, Nov 17, 2021 at 05:26:04PM -0800, David Thole wrote: I recently pulled the Kali community template, and it worked fine, but ran into the same issue being discussed here. So right now unable to update the qube due to this issue.

You can, of course, open a terminal in the template and update it there.

[TheDarkTrumpet]

On Wed, Nov 17, 2021 at 05:26:04PM -0800, David Thole wrote: I recently pulled the Kali community template, and it worked fine, but ran into the same issue being discussed here. So right now unable to update the qube due to this issue.
You can, of course, open a terminal in the template and update it there.

Yeah, I'm aware. We were talking about the template in the irc chat room, and when I mentioned the issue, the link was given with a request to update. I'm aware of how to get around it, but we were primarily testing it in relation to dom0's space. I plan on patching out the check and updating it that way so I'm not running "apt-get update/upgrade" directly within the template.