Using fish as user login shell breaks DisposableVM

[tyuio1]

Qubes OS version
R4.0

Affected component(s) or functionality
DisposableVMs based on a TemplateVM where fish is the login shell for user

Brief summary
DisposableVM starts up and halts after about a minute. Nothing is displayed.

To Reproduce
I'm going to use xterm in this example, but I also tried it with other applications, so this should not be xterm related.

Using a fresh fedora-30-minimal template
dom0:

$ sudo qubes-dom0-update qubes-template-fedora-30-minimal
$ qvm-create -t fedora-30-minimal test-dvm
$ qvm-prefs test-dvm template_for_dispvms True
$ qvm-features test-dvm appmenus-dispvm 1
$ qvm-run -u root fedora-30-minimal "dnf install -y qubes-core-agent-passwordless-root polkit fish"
$ qvm-shutdown fedora-30-minimal
$ qvm-run -q -a --service --dispvm=test-dvm -- qubes.StartApp+xterm

xterm opens as expected, inside this DisposableVM:

[user@disp8680 ~]$ rpm -q qubes-core-agent
qubes-core-agent-4.0.45-1.fc30.x86_64
[user@disp8680 ~]$ getent passwd user
user:x:1000:1000::/home/user:/bin/bash
[user@disp8680 ~]$ echo $PATH
/home/user/.local/bin:/home/user/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin

Then changing login shell to fish and starting another DisposableVM
dom0:

$ qvm-run -u root fedora-30-minimal "usermod --shell /usr/bin/fish user"
$ qvm-shutdown fedora-30-minimal
$ qvm-run -q -a --service --dispvm=test-dvm -- qubes.StartApp+xterm

I get the notification Domain disp806 has started and about a minute later Domain disp806 has halted. No xterm window appeared.

Starting xterm in the TemplateBasedVM via qvm-run -q -a --service -- test-dvm qubes.StartApp+xterm works as expected though

user@test-dvm ~> getent passwd user
user:x:1000:1000::/home/user:/usr/bin/fish
user@test-dvm ~> echo $PATH
/home/user/.local/bin /home/user/bin /usr/local/sbin /usr/local/bin /usr/sbin /usr/bin

Expected behavior
DisposableVM starts and works like the corresponding TemplateBasedVM

Actual behavior
DisposableVM starts but nothing happens. After a minute DisposableVM halts.

Solutions you've tried
Changing the login shell back to bash returns to expected behavior
dom0:

$ qvm-run -u root fedora-30-minimal "usermod --shell /bin/bash user"

Related, [non-duplicate] issues
#3434 is related to fish login shell, but it is set for root in that issue. Also, the PATH variable related problem does not exist in my issue, as can be seen above.

[dmoerner]

I've noticed this too. One way around this is to keep /bin/bash as the default shell in the template, install util-linux-user, and then add chsh -s /bin/fish user to /rw/config/rc.local for the templates where you want a fish shell. (Note that it seems you need to use chsh, not usermod for this to work.)

[marmarek]

@mkow do you see this problem?

[mkow]

I have the same problem with debian-9-dvm.

Additional info (not sure if useful): after a dvm has started (and failed to run the command), consecutive qvm-runs targeted at this dvm work, so I can e.g. open a terminal there (with fish running inside). After ~30s it dies with the whole dvm, but not always, for some reason sometimes everything survives.

Anyway, I can dump logs / do some debugging, but I'll need hints where to look exactly.

[marmarek]

If you're able to access such DispVM while it's still running, take a look at ~/.xsession-errors. Check also list of running processes, specially if there is anything like "qubes.WaitForSession".

Since DispVM is killed after a timeout (most likely related to qrexec_timeout VM's property), you may want to use sudo xl console VMNAME from dom0, so the output will remain accessible even after DispVM is killed. Note that terminal escape sequences are filtered out there (to protect your dom0 terminal emulator), so some output may be mangled.

[mkow]

Things from ~/.xsession-errors which look interesting:

dbus-update-activation-environment: setting SHELL=/usr/bin/fish
dbus-update-activation-environment: setting QT_ACCESSIBILITY=1
dbus-update-activation-environment: setting GPG_AGENT_INFO=/run/user/1000/gnupg/S.gpg-agent:0:1
dbus-update-activation-environment: setting PWD=<snip>
Failed to open /dev/tty: No such device or address
** Message: couldn't access control socket: /run/user/1000/keyring/control: No such file or directory
** Message: couldn't access control socket: /run/user/1000/keyring/control: No such file or directory
** Message: couldn't access control socket: /run/user/1000/keyring/control: No such file or directory
SSH_AUTH_SOCK=/run/user/1000/keyring/ssh
SSH_AUTH_SOCK=/run/user/1000/keyring/ssh
Warning:          Key <OUTP> not found in evdev+aliases(qwerty) keycodes
                  Symbols ignored
Warning:          Key <KITG> not found in evdev+aliases(qwerty) keycodes
                  Symbols ignored
Warning:          Key <KIDN> not found in evdev+aliases(qwerty) keycodes
                  Symbols ignored

and

executed QUBESRPC qubes.VMShell dom0 pid 844
executed QUBESRPC qubes.VMShell+WaitForSession dom0 pid 846
Activating service name='org.a11y.atspi.Registry'
Successfully activated service 'org.a11y.atspi.Registry'

(nm-applet:751): nm-applet-WARNING **: NetworkManager is not running
send exit code 0
pid 844 exited with 0

pstree shows such a process after the dvm was spawned:

├─qrexec-agent───qubes.WaitForSe───su───sleep

but after the timeout passed only qrexec-agent is left alive.

[dmoerner]

1. The bug is not reproducible without qubes-core-agent-passwordless-root installed, as the initial example implied.
2. There are a number of old reports of problems with sudo and tty_tickets and fish (not working with sudo's tty_tickets fish-shell/fish-shell#122), but they appear to be fixed by now and not related to this problem. Just to be sure, adding !tty_tickets to the Defaults in /etc/sudoers.d/qubes does not fix the problem for me.
3. My other proposed solution, using chsh in /rw/config/rc.local in a VM, seems to be non-deterministic (?!). Sometimes, the VM will start with fish, sometimes it will start with bash.
4. Like @mkow, I can open a terminal with qvm-run while the vm is running. And like him, sometimes this survives, sometimes it doesn't.
5. I now suspect that the problem is in some script that only runs if we have passwordless root. I tried to carefully examine the systemd log but found nothing suspicious.

[marmarek]

Found it:

cmd='echo $$ >> /tmp/qubes-session-waiter; [ ! -f /tmp/qubes-session-env ] && exec sleep inf'
if [ "$(id -un)" = "$USERNAME" ]; then
    sh -c "$cmd" 2>/dev/null
else
    su -c "$cmd" - "$USERNAME" 2> /dev/null
fi

su (unlike sh call just above) use login shell. And apparently && is not compatible with fish.

[tyuio1]

@marmarek I can confirm this fixes my problem
And fish finally supports && and || since v3.0.0

[qubesos-bot]

Automated announcement from builder-github

The package qubes-core-agent_4.1.3-1 has been pushed to the r4.1 testing repository for the Debian template.
To test this update, first enable the testing repository in /etc/apt/sources.list.d/qubes-*.list by uncommenting the line containing stretch-testing (or appropriate equivalent for your template version), then use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package core-agent-linux has been pushed to the r4.1 testing repository for the CentOS centos7 template.
To test this update, please install it with the following command:

sudo yum update --enablerepo=qubes-vm-r4.1-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package core-agent-linux has been pushed to the r4.0 testing repository for the CentOS centos7 template.
To test this update, please install it with the following command:

sudo yum update --enablerepo=qubes-vm-r4.0-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package qubes-core-agent_4.0.48-1 has been pushed to the r4.0 testing repository for the Debian template.
To test this update, first enable the testing repository in /etc/apt/sources.list.d/qubes-*.list by uncommenting the line containing stretch-testing (or appropriate equivalent for your template version), then use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package core-agent-linux has been pushed to the r4.0 stable repository for the CentOS centos7 template.
To install this update, please use the standard update command:

sudo yum update

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package qubes-core-agent_4.0.48-1+deb9u1 has been pushed to the r4.0 stable repository for the Debian template.
To install this update, please use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[ejose19]

@dmoerner Did you found any other solution for changing the default shell in a single appvm only? I can confirm the rc.local edit is not deterministic, so it can't be relied upon. My guess is that "user" session sometimes start before rc.local has finished.

@marmarek Do you know about any workaround for this besides changing shell at template level?

[dmoerner]

No, I just use fish as the shell in all my templates now.