RPi 5 - unable to mount root fs on unknown-block(179,2)

[sairon]

          Hi, I also had the same problem on my Raspberry PI 5 with installation on SD memory (purchased less than a month ago).

Since I installed hassio on Pi 5, every time I updated HA OS I had this bootloop problem.
Initially I thought it was because the release was still a candidate.
In these weeks (since HA OS for PI5 was released) I will have restored my SD about 4 times (practically with every new release).
The latest one was yesterday (HA OS 11.4)... although in this case I didn't expect it to break, as it was updated from a stable release 11.3 to another stable release 11.4.
I can only provide a screenshot of the first boot loop that happened to me when I upgraded from 11.3.rc1 to 11.3.rc2
I hope it can help you understand the problem.
Thank you

Originally posted by @EnzoD86 in #3059 (comment)

[sairon]

@EnzoD86 I managed to reproduce this issue by intentionally breaking cmdline.txt in the boot partition. The kernel then wants to load ext4 rootfs from partition 2 which is the default. Can you take out the SD card and check contents of the cmdline.txt on another computer?

[PascalStieber]

I ran into this, after I configured the WIFI and restarted the system. May this helps to reproduce.

[timh0r]

$ cat /mnt/part1/cmdline.txt
zram.enabled=1 zram.num_devices=3 rootwait cgroup_enable=memory fsck.repair=yes console=tty1 root=PARTUUID=a3ec664e-32ce-4665-95ea-7ae90ce9aa20 rootfstype=squashfs ro rauc.slot=B systemd.machine_id=d07ea661a8484f7597b474b23fc64c74

Here is a cmdline.txt those was broken
I replaced part1 content with the one from 11.4 (replaced kernel too has they were different) and it worked again

[EnzoD86]

@EnzoD86 I managed to reproduce this issue by intentionally breaking cmdline.txt in the boot partition. The kernel then wants to load ext4 rootfs from partition 2 which is the default. Can you take out the SD card and check contents of the cmdline.txt on another computer?

hi, unfortunately as I wrote I restored the SD with the latest HA OS image. So I have no ability to recover that file.
If it happens again with the next update (even if I have currently deactivated the beta channel) I will do what you asked.

[Caboz]

@EnzoD86 I managed to reproduce this issue by intentionally breaking cmdline.txt in the boot partition. The kernel then wants to load ext4 rootfs from partition 2 which is the default. Can you take out the SD card and check contents of the cmdline.txt on another computer?

Unfortunately I flashed the SD Card after the problem. And the installation is on my other house far from me.

This was a Raspberry Pi 5 8Gb with a 32Gb SD Card.
Clean install of version 11.3 and immediate update to 11.4.
Two plugins installed: File Editor and Let's Encrypt.

If I do a restart from the HA - OK
If I shutdown from the HA and disconnect the power - NOK (the restart loop starts)
If I pull the plug from the HA - NOK (the restart loop starts)

[timh0r]

I also have a full image of the borked SD card, if needed

[joss044]

Hi,
There's also multiple reports of another startup failure mode in discussions/2844; the startup being halted with a "platform kgdboc: deferred probe pending" message. That is what I have experienced as well, also with OS 11.4.
Is this issue also addressed here or should we open another issue. I have a screen picture and cmdline.txt copies for troubleshooting, if needed.

Hardware:
Rpi-5 with 4GB, set to USB boot priority
Samsung 250GB 860 EVO SSD on USB - no SD-card

== Just found that this issue is addressed in # 3067 ==

[gentel27]

Version 11.3 seems to be ok, ( haos_rpi5-64-11.3.img.xz ) and RPi5 survive a "power failure" and will reboot. So for now do not update to 11.4

[davemanchap]

While I can't confirm the issue I do believe it's related to

I ran into this, after I configured the WIFI and restarted the system. May this helps to reproduce.

While I can't confirm the issue, I do believe it's something to do with WiFi settings. I've flashed my SD card a few times this week and have set everything up in different orders and rebooting at each step. Only when I have setup WiFi has it got stuck in the boot loop. I now have a fully operational 11.4 running (except Wifi) and have restarted a few times with no issues. Maybe a while for network needs to added to boot?

[lehno]

Same issue, but I never connected to any wifi. I was using cabled network

[EndermanAPM]

Same here, but running 11.3 started boot looping after a power outage. Also never used wifi.

[WhyAydan]

Just happened to me as well, if you need anything from my SD card let me know as im using another for now.

[BeZi80]

Same here, i bought 2 new SD cards, after i had this issue 4 times, Happens when i each time i switch off Rasp. I try to recreate from backup... (hope this will work)
if someone else know how to fix this, pleas write a description...
how can i open the cmdline.txt with windows? And what i have to change ?

--> BACKUP was working !!!! Puhhhh

[timh0r]

Same here, i bought 2 new SD cards, after i had this issue 4 times, Happens when i each time i switch off Rasp. I try to recreate from backup... (hope this will work) if someone else know how to fix this, pleas write a description... how can i open the cmdline.txt with windows? And what i have to change ?

--> BACKUP was working !!!! Puhhhh

Editing on windows require to mount the first partition. Windows might not let you mount it despite it being a fat32. Linux subsystem for windows will allow you to access the disk and partition if needed.

[EndermanAPM]

Is there a way to fix this without full reinstall?

[timh0r]

Is there a way to fix this without full reinstall?

Replace your cmdline.txt with the stock one of your release and retry ?

[EnzoD86]

Version 11.3 seems to be ok, ( haos_rpi5-64-11.3.img.xz ) and RPi5 survive a "power failure" and will reboot. So for now do not update to 11.4

Hi, I would like to specify that the problem occurs only when you update the operating system. Instead, starting from a clean version, i.e. directly flashing the SD card with version 11.4 and restoring the backup there are no problems... in fact I'm currently using the latest version (11.4) and even with complete reboots the system works perfectly .... so it's something that only breaks during the update phase from the old version to the new one.

[sdoe0]

Same here. Occurs every time I reboot the rpi5. Regardless if it is a fresh, "naked" image or the backup HA instance on a previously flashed SD

[sairon]

@timh0r

I also have a full image of the borked SD card, if needed

That would be very helpful, thanks. Theoretically only boot partition content is what is interesting here, so you can extract it with dd like this:

dd if=sdcard.img of=boot.img skip=2048 count=128k

If anything, you can hit me up on Discord (https://discordapp.com/users/544193690806255616).

[timh0r]

@timh0r

I also have a full image of the borked SD card, if needed

That would be very helpful, thanks. Theoretically only boot partition content is what is interesting here, so you can extract it with dd like this:

dd if=sdcard.img of=boot.img skip=2048 count=128k

If anything, you can hit me up on Discord (https://discordapp.com/users/544193690806255616).

boot.img.gz
attached is the image, sent over discord too to verify the format is good.

[phillipo6]

If this pr is completed, is it possible to publish a release? This issue is the only thing keeping me from using ha on the new pi 5

[timh0r]

thanks for the fix <3

[EndermanAPM]

For the broke installs, what should the the procedure?
Clean install a dev image? Or it's fine to clean install 11.4 and wait for 11.5 on stable?

[sairon]

@EndermanAPM Today's dev image (I will start build later today) will have it resolved, so force-installing it will be one way. Otherwise manual edit of the config.txt (changing cmdline=cmdline-tryboot.txt to cmdline=cmdline.txt) after a breaking update should fix the error and future beta/stable upgrades should be fine too.

[agners]

So this turned out to be a bug in the initial implementation of the update system. The initial boot after update worked, but not another reboot 🙈 . I am not exactly sure how I've missed that in my testing. @sairon thanks for the find and fixing this with #3085.

For everyone, the next dev/beta/stable update should work fine.

To fix your installation, insert the SD card into your system. On the first FAT partition of the SD card, you'll find a file named config.txt. Open it using a text editor. There should be a line starting with cmdline=. It probably is set to cmdline=/cmdline-tryboot.txt currently.Change it to cmdline=/cmdline.txt. Save and close the file. Unmount (cleanly eject) the SD card, and insert it back into your Raspberry Pi 5. This should make Home Assistant OS boot again.

[WillSoper]

Thanks @sairon for the fix and @agners for the detail!

Just wanted to check one point - will the next dev/beta/stable version update this file before rebooting (so it'll work right away), or will that update still break but every update AFTER that will work fine?

[sairon]

@WillSoper Yes, the file is modified before the reboot to the new OS version and then once the system successfully boots. If nothing happens during the upgrade, you should be fine from then on.

[yahwehPT]

does this fix #3067? I know it's a different problem but was introduced with the same update (11.4)

[Picard16]

I updated today my dev configuration from December with HAOS 11.5rc2 (and supervisor and core and frontend) and everything runs perfect on my RasPi5.
Thanks a lot @agners and @sairon for your fantastic work!! 🙏🏻💐

[EnzoD86]

I waited a few days before updating to version 11.5. I confirm that the problem has been solved!
Thanks @agners and @sairon