Stretch template slow boot time

[unman]

Qubes OS version (e.g., R3.2):

R3.2

Affected TemplateVMs (e.g., fedora-23, if applicable):

Stretch

---

Expected behavior:

Template or Template-based-VM will boot normally

Actual behavior:

Very slow boot time - 90-120 secs.

Steps to reproduce the behavior:

Install Stretch template.
Fix X issues - see #2909
Start template or TemplateBasedVM

General notes:

delay seems to be on sys-subsytem-net-devices-multi-user.device

---

Related issues:

[tasket]

The delay is still there with the build from today.

Template upgraded from jessie to stretch doesn't have this issue.

[tasket]

Looking at the differences between upgraded debian-8-to-9 and the qubes-template-debian-9 VMs, I found this in the stalling (latter) one:

/etc/systemd/system/multi-user.target.wants/[email protected]

This was alongside wpa_supplicant.service (without 'at' symbol). So I deleted [email protected] and now the template and VMs based on it are booting fine, without the 90sec delay.

[tasket]

Wondering how this 'extra' link got into /etc, I noticed that qubes-builder-debian lists wpasupplicant in packages_wheezy.list and packages_stretch.list... but not packages_jessie.list.

[tasket]

BTW full path and command is:

sudo rm /etc/systemd/system/multi-user.target.wants/[email protected]

[unman]

wpasupplicant is installed in jessie as well as stretch, so the absence from packages is neither here nor there.
Deleting the link from multi-user.target.wants doesn't fix the issue for me.
@tasket Did you make any other changes?

[tasket]

No other changes I recall. Will try again with a fresh template.

Remember, wpa_supplicant.service is still there in debian-8 and debian-9. But debian-9 also has another link to service with at-sign; its this one I deleted.

[tasket]

@unman Tried a newly-installed debian-9 template. Bootup time = 130sec.

Then removed the @.service link then boot time = 12sec.

The template version I'm using is 9-4.0.0-201711030329

[tasket]

Re-adding the link causes template to return to 130+sec start time. The service in question has this status:

● [email protected] - WPA supplicant daemon (interface-specific version)
   Loaded: loaded (/lib/systemd/system/[email protected]; enabled; vendor preset: enabled)
   Active: inactive (dead)

Nov 14 13:37:40 debian-9 systemd[1]: Dependency failed for WPA supplicant daemon (interface-specific version).
Nov 14 13:37:40 debian-9 systemd[1]: [email protected]: Job [email protected]/start failed with result 'dependency'.

[tasket]

More complete from journalctl:

Nov 14 13:55:13 debian-9 systemd[1]: sys-subsystem-net-devices-multi-user.device: Job sys-subsystem-net-devices-multi-user.device/start timed out.
Nov 14 13:55:13 debian-9 systemd[1]: Timed out waiting for device sys-subsystem-net-devices-multi-user.device.
Nov 14 13:55:13 debian-9 systemd[1]: Dependency failed for WPA supplicant daemon (interface-specific version).
Nov 14 13:55:13 debian-9 systemd[1]: [email protected]: Job [email protected]/start failed with result 'dependency'.
Nov 14 13:55:13 debian-9 systemd[1]: sys-subsystem-net-devices-multi-user.device: Job sys-subsystem-net-devices-multi-user.device/start failed with result 'timeout'.

[unman]

Ok thanks for that. I'll have to see why it isnt working for me.

[tasket]

I can also report that adding the @.service link to my upgraded-8-to-9 template causes the delay to appear for the same 1:30 duration. (Prior comments said "130sec" but its really 1m 30sec; sorry about that.)

[unman]

So a newly built stretch template DOES show the interaction with @.service that @tasket describes, but my more elderly not updated template did not (afaik).
However, new builds DO show that behaviour.
We cant simply delete the wpasupplicant service because from a quick test it will break use of wifi in a stretch based sys-net. @tasket Can you check this?

[tasket]

I've been using for sys-net an upgraded-to-9 template with wpasupplicant for almost a week, no problems. Of course, it contains a link to regular .service but not for the @.service.

Question: Does a regular Debian 9 install setup this @.service link, or could this be some artifact from the template building process? I have to wonder since apt upgrade process doesn't put it there.

[unman]

No a standard install on laptop with wifi adapter doesn't include EITHER service in the "wants": neither the regular nor the @ service.
It isn't there in the "prepared" image and is in the qubeized image. The fact that the link is set up in qubeization explains why the @ link is not present in upgraded-to-stretch templates. Interesting.

[unman]

@tasket - would it be possible for you to test using a Debian-9 (not upgraded from 8) for sys-net, with the @.service link removed?
I have limited wifi adapters and they dont play well with NetworkManager anyway - everything works just fine with wicd.

[tasket]

@unman - OK. New template installed. Removed the @ link. Installed firmware-iwlwifi package.
Setup netvm. Works!

But doesn't work for me without adding firmware-iwlwifi.

[unman]

@tasket - Excellent - thanks for that.
Obviously the firmware requirement will be dependent on the NIC and isn't included in standard packages.

[tasket]

@unman - Did I mention I'm using R4.0-rc3? (Probably doesn't matter though.)

[unman]

After disabling wpa_supplicant (thanks @tasket), problem is resolved.