HA OS 14.1 crashes after boot on RPi 3b+

[mmalina]

Describe the issue you are experiencing

Today I updated from HA OS 13.x to 14.1 and now the OS fails to boot properly for me. Or rather, it seems to boot ok, but then prints some errors and reboots and gets stuck. Before this update today the installation worked for me without issues, only occasionally after reboot it would fail to boot from USB - see #1712 - and then a new boot would usually work.

Here's what happens in more detail:
At first the boot looks ok - all the lines with green OK. It even gets to the point where it prints versions and a ha > prompt. At this point the host responds to ping.
But then a few minute later it starts printing errors in the console and soon after it reboots itself.

[ 152.794026] humon hwmon1: Failed to get throttled (-110)
158.553999] ieee80211 phy: brcmf_escan_timeout: timer expired
161.114073] ieee80211 phy: brcmf_notify_escan_complete: Scan abort failed
163.674054]
bremfmac: brcmf_set_channel: set chanspec Ox1001 fail, reason
166.234053]
bremmac: brcmf_set_channel: set chanspec Ox1002 fail, reason
168.794090]
bremmac: bremf_set_channel: set chanspec Ox1003 fail, reason
170.854032] rcu: INFO: rcu preempt detected stalls on CPUs/tasks:
170.854149]
rcu: 00-...0: (1 GPs behind) idle=a18c/1/0x40000000 softirg=17526/17527 fqs=1028
170.854249] rcu: (detected by 1, t=2102 jiffies, g=16717, g=424 ncpus=4)

When it reboots, it will get stuck on "no ethernet found" error, exactly as reported in the past here: #1712

If I power off the RPi, wait 20 seconds and try again, it happens again. It happened to me 3 times in a row. But on fourth attempt it stayed up (at least for now).

Of course it could be a coincidence that my RPi is dying and maybe it has nothing to do with the update today. But I haven't experienced these errors until now, so it does seem to be related to the update.

What operating system image do you use?

rpi3 (Raspberry Pi 3 32-bit OS)

What version of Home Assistant Operating System is installed?

14.1

Did the problem occur after upgrading the Operating System?

Yes

Hardware details

I have a conbee II attached as well as an external SSD disk connected via an adapter (Sabrent SATA Cable Adapter). The SSD is what it boots from and it contains everything.

Steps to reproduce the issue

1. Update RPi 3b+ HA OS installation to 14.1 and wait for the boot

Anything in the Supervisor logs that might be useful for us?

Probably not. There is a bunch of warnings like these:

2025-01-11 10:33:16.992 WARNING (MainThread) [asyncio] Executing <Task pending name='Task-7' coro=<Core.setup() running at /usr/src/supervisor/supervisor/core.py:166> wait_for=<_GatheringFuture pending cb=[Task.task_wakeup()] created at /usr/local/lib/python3.12/asyncio/tasks.py:712> cb=[_run_until_complete_cb() at /usr/local/lib/python3.12/asyncio/base_events.py:181] created at /usr/local/lib/python3.12/asyncio/tasks.py:695> took 0.240 seconds
2025-01-11 10:33:17.281 WARNING (MainThread) [asyncio] Executing <Task pending name='Task-7' coro=<Core.setup() running at /usr/src/supervisor/supervisor/core.py:166> wait_for=<_GatheringFuture pending cb=[Task.task_wakeup()] created at /usr/local/lib/python3.12/asyncio/tasks.py:712> cb=[_run_until_complete_cb() at /usr/local/lib/python3.12/asyncio/base_events.py:181] created at /usr/local/lib/python3.12/asyncio/tasks.py:695> took 0.216 seconds
2025-01-11 10:33:18.320 WARNING (MainThread) [asyncio] Executing <Task finished name='Task-190' coro=<Addon.load() done, defined at /usr/src/supervisor/supervisor/addons/addon.py:216> result=None created at /usr/local/lib/python3.12/asyncio/tasks.py:695> took 0.285 seconds
2025-01-11 10:33:47.125 WARNING (MainThread) [asyncio] Executing <Task pending name='Task-7' coro=<Core.setup() running at /usr/src/supervisor/supervisor/core.py:166> wait_for=<Future pending cb=[Task.task_wakeup()] created at /usr/local/lib/python3.12/asyncio/base_events.py:448> cb=[_run_until_complete_cb() at /usr/local/lib/python3.12/asyncio/base_events.py:181] created at /usr/local/lib/python3.12/asyncio/tasks.py:695> took 0.110 seconds

### Anything in the Host logs that might be useful for us?

```txt
I don't think so. When this happens, HA won't likely even start.

System information

System Information

version	core-2025.1.2
installation_type	Home Assistant OS
dev	false
hassio	true
docker	true
user	root
virtualenv	false
python_version	3.13.1
os_name	Linux
os_version	6.6.62-haos-raspi
arch	armv7l
timezone	Europe/Prague
config_dir	/config

Home Assistant Cloud

logged_in	false
can_reach_cert_server	ok
can_reach_cloud_auth	ok
can_reach_cloud	ok

Home Assistant Supervisor

host_os	Home Assistant OS 14.1
update_channel	stable
supervisor_version	supervisor-2024.12.3
agent_version	1.6.0
docker_version	27.2.0
disk_total	109.3 GB
disk_used	16.5 GB
healthy	true
supported	true
host_connectivity	true
supervisor_connectivity	true
ntp_synchronized	true
virtualization
board	rpi3
supervisor_api	ok
version_api	ok
installed_addons	Samba share (12.3.2), File editor (5.8.0), Dropbox Sync (1.3.0), Mosquitto broker (6.4.1), Terminal & SSH (9.16.0), deCONZ (7.0.0), Home Assistant Google Drive Backup (0.112.1), zabiny-temp (0.10)

Dashboards

dashboards	2
resources	4
views	2
mode	storage

Recorder

oldest_recorder_run	December 13, 2024 at 12:27 PM
current_recorder_run	January 11, 2025 at 10:35 AM
estimated_db_size	99.68 MiB
database_engine	sqlite
database_version	3.47.1

Additional information

No response

[mmalina]

I tried again now and this time it did reboot a few minutes after the ha cli prompt was shown, but I didn't catch the errors. This time the reboot didn't get stuck, so that's kind of random and unrelated. But once again, the reboot got to the point of showing the ha cli prompt. Then a minute later it printed a bunch of errors and eventually rebooted itself again.

156.713970] humon hwmon1: Failed to get throttled (-110)
cce80211 phoo:
156.714198] brcmfmac: brcmf_dump_obss: dump_obss error (-110)
159.273954] ieee80211 phy: brcmf_set mpc: fail to set mpc
175.143926] rcu: INFO: rcu preempt detected stalls on CPUs/tasks:
175.144135] rcu: (detected by 2, t=2102 jiffies, g=17169, g=345 ncpus=4)
185.144500] rcu: RCU grace-period kthread stack dump:
185.144740] rcu: Stack dump where RCU GP thread last ran:

On this third attempt it worked, but after 6 minutes I got more errors and a reboot.

220.313773] brcmfmac: brcf_sdio_bus_rxctl: resumed on timeout
366.553859] INFO: task kworker/u9:1:41 blocked for more than 122 seconds.
366.554029]
Tainted: G
[
6.6.62-haos-raspi #1
366.554140] "echo 0 › /proc/sys/kernel/hung_task_timeout_secs" disables this message.
366.555014] INFO: task kswapd0:59 blocked for more than 122 seconds.
366.555137]
366.5552461 "echo
Tainted: G
6.6.62-haos-raspi #1
0>/proc/sys kernel/hung_task_timeout_secs" disables this message.
366.556117] INFO: task usb-storage:86 blocked for more than 122 seconds.
366.556248]
Tainted: G
6.6.62-haos-raspi #1
366.556356] "echo 0 › /proc/sys/kernel/hung_task_timeout_secs" disables this message.
366.557206] INFO: task dbus-broker:128 blocked for more than 122 seconds.
366.557331]
Tainted: G
6.6.62-haos-raspi #1
366.557448]
بابابابابالالالال
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
366.558392]
INFO:
task kworker/u13:3:233 blocked for more than 122 seconds.
366.5585171
Tainted: G
6.6.62-haos-raspi #1
366.558622]
"echo
o > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
366.5599981
INFO: task jbd2/sda7-8:284 blocked for more than 122 seconds.
366.560116]
Tainted: G
6.6.62-haos-raspi #1
366.560216]
"echo
> /proc/sys/kernel/hung_task_timeout_secs" disables this message.
366.560675]
INFO :
task upa_supplicant:452 blocked for more than 122 seconds.
6.6.62-haos-raspi #1
366.560794]
Tainted: G
366.5608991
"echo
0 › proc/sys/kernel/hung_task_timeout_secs" disables this message.
366.565311]
INFO:
task bluetoothd:463 blocked for more than 122 seconds.
6.6.62-haos-raspi #1
366.565452]
Tainted: G
366.565576]
"echo
0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
366.566490]
INFO:
task containerd:528 blocked for more than 122 seconds.
6.6.62-haos-raspi #1
366.566610]
Tainted: G
366.566715]
"echo 0 › /proc/sys/kernel/hung_task_timeout_secs" disables this message.
366.567259]
INFO: task systemd-timesyn:535 blocked for more than 122 seconds.
6.6.62-haos-raspi #1
366.567386]
Tainted: G
[
366.567493]
"echo 0 › /proc/sys/kernel/hung_task_timeout_secs" disables this message.

It seems similar to this perhaps? https://community.home-assistant.io/t/rpi3b-hassos-failed-to-get-throttled/754686/3

[mmalina]

OK, so this seems to be a duplicate of #3362

Related to Conbee II zigbee usb stick. Now sure why in my case it only started happening with HA OS 14 when it was originally reported for HA OS 12.

Specifically this comment contains instructions on how to update Conbee II firmware which seems to have fixed the issue for me (only a few minutes since the fix, so fingers crossed): #3362 (comment)