Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

How can i increase my swap file size? #968

Open
basdejong95 opened this issue Nov 7, 2020 · 59 comments
Open

How can i increase my swap file size? #968

basdejong95 opened this issue Nov 7, 2020 · 59 comments
Labels
enhancement

Comments

@basdejong95
Copy link

HassOS release with the issue:

HassOS 4.15

Supervisor logs:

Journal logs:

Kernel logs:

Description of problem:

Hi everybody,

I am using home assistant on a RPI 4 with 1GB of ram. My swap file is only 230mb and fills up pretty fast, in the last week or so i had home assistant crashing a lot and i noticed my swap file is almost pretty full. While reading about this there are pretty clear instructions on the internet on how to increase the swap file on a standard RPI, but i cannot find it anywhere for HASSOS. Does anybody know a command or a setting i can change to increase this? I have a 128GB SD card which is A2 class so i dont mind creating a swap file of 1-2GB.

Thank you!
@agners
Copy link
Member

agners commented Nov 9, 2020

Currently Home Assistant OS does not really support that use case. This is also taking a strain on your SD card as swap usually generates quite a bit of read and writes. Since we already have users which have problems with SD cards wearing out, in a lot of cases using swap on SD card does not make sense.

By default HAOS configures a part of the RAM as compressed swap storage (using zram). This helps quite a bit for boards with low memory.

That said, when you access HAOS directly (via SSH port 22222), you can create a swapfile and enable it as swap, but the swap will not be automatically re-enabled after a reboot:

# fallocate -l 4G /mnt/data/test.swap
# mkswap /mnt/data/test.swap
# chmod 0600 /mnt/data/test.swap
# swapon /mnt/data/test.swap

@bachy
Copy link

bachy commented Nov 11, 2020 •
edited
Loading

hi,
my hassos lives in a rpi3b+ with a 120Go ssd connected via usb
i moved the data partition on the ssd
my memory is at 80% and swap at 100%
i don't have so much devices and entities (357) (discovered HA 2 weeks ago)
using rfxtrx, z-wave, zigbee
top says that homeassistant is consuming 31%, mysqld 16%, systemd-journald 5%, node-red 5% and supervisor 2.5%
i succesfully added a 4G swap file as describe above, i'll see if it prevent my instance to freeze
is there any way to make it permanent ? i mean active on reboot
i saw that there is nothing on /etc/fstab
i know nothing about buildroot
thanks

@Jpsy
Copy link

Jpsy commented Nov 11, 2020

Do yourself a favor and switch over to an RPi4 with more than 1GB of RAM. You will only waste time with you trials on 1GB. The difference that more RAM makes is overwhelming.

@bachy
Copy link

bachy commented Nov 11, 2020

Yes i know, unfortunatly my z-stick gen5 is not compatible with rpi4 and i'm lazzy to rebuild all my zwave mesh (qubinos wire pilote on the wall behind heaters)

@frenck
Copy link
Member

frenck commented Nov 11, 2020

Do yourself a favor and switch over to an RPi4 with more than 1GB of RAM.

Better yet, grab an ODROID N2 or N2+

@basdejong95
Copy link
Author

Do yourself a favor and switch over to an RPi4 with more than 1GB of RAM. You will only waste time with you trials on 1GB. The difference that more RAM makes is overwhelming.

Home assistant says on the getting started website that 1GB ram should work... And it would work if my swap file could be bigger. However there is no way to permanently change this, only the commands from @agners are a manual solution for now?

@basdejong95
Copy link
Author

image

this is my swap file within 1 day of rebooting

@JZ-SmartThings
Copy link

@agners that standard solution doesn't seem to work... see the error which suggests that BusyBox and not the full version of swapon is what HassOS seems to be using. Error you get is invalid argument for swapon. FYI, swapon -a ran fine but did not use the newly created swap file either. With my SSD plugged into the RPI3B+, it'd be nice to be able to control the default swap behavior in HassOS.
image

@QbaF
Copy link

QbaF commented Dec 4, 2020

for me
image

@capstan1
Copy link
Contributor

capstan1 commented Dec 8, 2020

Currently Home Assistant OS does not really support that use case. This is also taking a strain on your SD card as swap usually generates quite a bit of read and writes. Since we already have users which have problems with SD cards wearing out, in a lot of cases using swap on SD card does not make sense.

By default HAOS configures a part of the RAM as compressed swap storage (using zram). This helps quite a bit for boards with low memory.

That said, when you access HAOS directly (via SSH port 22222), you can create a swapfile and enable it as swap, but the swap will not be automatically re-enabled after a reboot:

# fallocate -l 4G /mnt/data/test.swap
# mkswap /mnt/data/test.swap
# chmod 0600 /mnt/data/test.swap
# swapon /mnt/data/test.swap

This works like a treat! On my ssd that should be perfect to prevent filling up the ram completely.

Any idea on how to make this persistent?

@dpjrodrigues
Copy link

I do understand what @agners say and I do agree that if you use an SD card increasing the swap is not great but if you use an SSD then shouldn't be a problem.

Since few months ago my pi3b+ is getting very slow and HA ends up not responding. Restart fixes it and then after 1 week or so again the same thing.
I’ve added few checks and realized that it may be related with memory… once swap hits close to 100% and RAM 90% this happens.
I do not have that many addons and it was working ok before… there are few discussion on potential memory leaks but anyway it is not the purpose of this topic.

I’m running an SSD with my pi so raising the swap size could reduce or even solve some of these issues. The default setting is just 25% of the real device memory (in this case 256Mb) and I think that it is probably having in SDcards in mind (and not SSD)

I’ve managed to change it using the commands below and so far so good… the problem is that this is not persistent (and after restart goes back to original swap size). As far as I understood I would need to change the /usr/libexec/hassos-zram or /usr/lib/systemd/system/zram-swap.service but the filesystem is read-only. Is there any other way to do this?

swapoff /dev/zram0
zramctl --reset /dev/zram0
zramctl --find --size 1024M
mkswap /dev/zram0
swapon /dev/zram0

@dpjrodrigues
Copy link

Just a quick update to mention that increasing the swap size is working well so far. Now the RAM is stable 65% - 70% and the swap is between 30% - 35% so not that much from the forced 25% (the problem is that this configuration is lost when restarted)

@marcoevcom
Copy link

marcoevcom commented Dec 31, 2020 •
edited
Loading

I got the exact same problem. It would be great to edit somenthing on hassio config file prior to boot to set up as SSD.

@capstan1
Copy link
Contributor

capstan1 commented Dec 31, 2020 •
edited
Loading

I found a way to make this "persistant" using the SSH & Web Terminal Addon. When disabling protection mode for the addon you can simply place the swape file inside the Core folders (for example "/config" oder "/config/www") and issue the swapon command as initial command of the addon. The swapfile itself and the permissions survive the restarts.

##### Edit: Confirmed working solution:

  • install the addon via the repository
  • disable "protection mode" in the addons' settings (always be careful with your new powers!)
  • issue the first three commands inside terminal with changed folders:
fallocate -l 1G /backup/swapfile.swap
mkswap /backup/swapfile.swap
chmod 0600 /backup/swapfile.swap
  • set the third command as init_command in the addons' configuration:
    swapon /backup/swapfile.swap
Edit: Original post (not optimal, due to increased snapshot size (as described in the next post ))
fallocate -l 1G /config/www/swapfile.swap
mkswap /config/www/swapfile.swap
chmod 0600 /config/www/swapfile.swap
  • set the third command as init_command in the addons' configuration:
    swapon /config/www/swapfile.swap

@JZ-SmartThings
Copy link

@capstan1 it makes doing HA backups a pain... now there's a huge file rather than a few megs. Can the file reside in any other location outside of HA config folder?

@Bellavista
Copy link

Bellavista commented Jan 16, 2021 •
edited
Loading

I found a way to make this "persistant" using the SSH & Web Terminal Addon. When disabling protection mode for the addon you can simply place the swape file inside the Core folders (for example "/config" oder "/config/www") and issue the swapon command as initial command of the addon. The swapfile itself and the permissions survive the restarts.

fallocate -l 1G /config/www/swapfile.swap
mkswap /config/www/swapfile.swap
chmod 0600 /config/www/swapfile.swap
  • set the third command as init_command in the addons' configuration:
    swapon /mnt/data/test.swap

HI to all!
wonderful .. you are great!
i had... an rsp3 + with 1G ram and 200mb of swap file but now.... the swap file is 2Gbit ..
I read the post from JZ-SmartThings .... but I thought .... the folder / backup is not part of the backup and I tried to create the swap file in backup folder and i used SSH & Web Terminal Addon like @capstan1 GREAT!!!
after a snapshot whith Home Assistant Google Drive Backup and a restart
YESSS!!! IT WORKS !!! GREAT!
image
Cattura

@JZ-SmartThings
Copy link

@Bellavista notice that your init_command is not using the same file/location as you have on the swapon command.

We seem to be getting lucky with how swap files are considered/packed by TAR. Notice that 1GB is down to 1MB when compressed:

image

Seems like that 1MB of backup space is indeed a small price to pay for larger swap. However, after looking at init_commands, it made me wonder why not use that to run all of the necessary commands. Something worth trying... thanks for the more detailed steps, the INIT_COMMANDS piece was key for me.

@capstan1
Copy link
Contributor

capstan1 commented Jan 16, 2021 •
edited
Loading

Hi,
indeed, the snapshot size increases when putting the swapfile inside a folder that gets snapshoted. @JZ-SmartThings: This 1 MB is probably close after restarting. As I understand this, the 1GB is reserved space, hence a compressed file is small. But as swap fills up with time, the snapshots also increase in size. My snapshots grew from around 70MB so 160-200Mb during 1 week of uptime.

I changed the location to the /media folder and excluded this folder from my snapshots (very simple with the Google Drive Addon). Then the snapshots' size don't increase. As @Bellavista mentioned, the /backup folder should be a good choice, too!

However, after looking at init_commands, it made me wonder why not use that to run all of the necessary commands. Something worth trying...

Surely, you could put all commands inside the init_commands. But as the swapfile itself is persistent throughout restarts, it's just not necessary. Don't know if it adds a lot of workload to a restart, though. Only advantage I can see: it's also executed after restoring a snapshot on a fresh install. So you won't need to SSH into the host once after restoring.

@JZ-SmartThings
Copy link

JZ-SmartThings commented Jan 19, 2021 •
edited
Loading

@capstan1 and @Bellavista I'm so glad you both posted. It all led me to the final ONE-LINER resolution :) which is ultimately the best and most maintainable approach. Basically use the INIT_COMMANDS to run an if statement in the shell and you never really need to visit that shell yourself to make this work. It survives reboots and doesn't make my backups larger because I simply store it in the /backup folder anyway like suggested. It has a bit logic which will re-create the file if it's not there already, etc.

BTW @capstan1 that makes perfect sense... swap file will be all zeros until it's actually used. Now that the uptime is longer it is indeed impacting the backup size like I initially suspected.

Thanks again to you both! Here's the one-line solution which belongs on the Configuration page of the SSH & Web Terminal add-on in HA.

init_commands:
  - if [ ! -f /backup/_swap.swap ]; then fallocate -l 2G /backup/_swap.swap && mkswap /backup/_swap.swap && chmod 0600 /backup/_swap.swap && swapon /backup/_swap.swap ; else swapon /backup/_swap.swap; fi

@QbaF
Copy link

QbaF commented Jan 19, 2021

I am not sure why we have to act as an underground/resistance movement but You did it well;) Thank You.

@JZ-SmartThings
Copy link

@QbaF underground teamwork indeed! We also owe some gratitude to @agners who came with the right ammunition!

@Bellavista
Copy link

Great job from everyone. I am one of those users who knows little ... indeed nothing ... about programming and linux or yamal commands, I only have a scholastic knowledge ... (but I'm not young ... ;-) internet has been a turn fatal! Well!
The effect of the 2GB swap gave the system a lot of stability!
Maybe 2 gb is too much I based on the double of system memory rule ....
Still great result !!! Thank you all
ps. Thank you @agners

@JZ-SmartThings
Copy link

Please see the on-going thread here: https://community.home-assistant.io/t/how-to-increase-the-swap-file-size-on-home-assistant-os/272226

I modified the command in the 1st post to have a bit more logic where it checks if swap is already on with dmesg and if so NOT to repeat the command. Otherwise the SSH add-on fails a restart.

Pasting it one last time here and will continue with the community post:

init_commands:
  - if [ ! -f /backup/_swap.swap ]; then fallocate -l 2G /backup/_swap.swap && mkswap /backup/_swap.swap && chmod 0600 /backup/_swap.swap && swapon /backup/_swap.swap ; elif [[ ! $(dmesg|grep _swap.swap) = *swap\ on* ]]; then swapon /backup/_swap.swap; fi

@agners agners mentioned this issue Feb 10, 2021
Closed
@johnsmithmad
Copy link

Thank you guys, it helped me a lot. I've a raspberry pi 3b and HASSIO was unstable and unavailable almost one time per day ...

@stale
Copy link

stale bot commented Jun 11, 2021

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the wontfix label Jun 11, 2021
@agamemnen
Copy link

I can't understand it.
how to do it: disable "protection mode" in the addon?
Explain step by step.

PS
Home Assistant OS 6.2
supervisor-2021.09.0
core-2021.9.6

@stale stale bot removed the wontfix label Sep 16, 2021
@Vinz87
Copy link

Vinz87 commented Nov 2, 2021

I get the output swapon: /backup/_swap.swap: Operation not permitted when trying to execute the command swapon /backup/_swap.swap, anyone knows why?

@PandJoKell
Copy link

Hi fellow HA users,
Big, big thanks to the gurus who have contributed to this HA critical thread.
I'm running HAOS on a RPi3B+, 128GB SD card, no portable drive or SSD, running HA OS 7.0, supervisor-2021.12.2, docker 20.10.9.
I have the same HASSIO instability problems and I have almost given up on HA. I'm trying some of the ideas in this thread and are hitting some roadblocks that have prompted me to make this post. I'm a crusty old electrical engineer with fair bit of real-time SCADA computing experience. With limited RAM in old minis and mainframes swap files were common to keep the systems alive with some real time performance penalty to manage. Sizing the swap file was critical to the stability of the system. I find the modern computer jargon is quite challenging for me at times but I have some basic questions/observations:

  1. Why use the SSH & Web Terminal addon as the place to put the init_commands? Could any other addons that are initiated at restart be used?
  2. In the SSH & Web Terminal addon, thanks to agememnen, I added "protection mode: disable" in the 3rd line of the Configuration to put it into the required disabled "protection mode". On checking the addon log I couldn't find a specific note indicating protection mode disabled but no errors in the log so I assumed it worked. Also I couldn't find this as a setting option in the Documentation tab for the addon (version 9.2.1). Did I miss it somewhere?
  3. Before solving the protection mode disable problem, when I tried to add the init_command as per JZ-SmartThings to the Configuration options in the SSH & Web Terminal addon I got a permissions error and the Configuration reverted back to my last saved version. Perhaps others may have seen this error?
  4. I may have missed this in the above thread but I assume that once the addon is configured as above, I do need to restart the system to get it to create the swap file and allocate the ram?
  5. Should we be concerned that much of the accumulated knowledge in the thread may be lost if/when the thread goes stale? I looked at lots of disparate (and often not so helpful) posts on github (and elsewhere) before finding this thread which I consider to be the "jewel in the crown". I note also the Oct 2021 post by Biont with an alternative approach which I haven't got to yet, but it may be a better overall method. How can this knowledge base be preserved for others? Do we need to keep this thread fresh because it seems to me to be critical to the ongoing interests of all HA users and could be lost if it goes stale?
    Thanks again and keep up the good work.

@bkbartk
Copy link

bkbartk commented Jan 1, 2022

A lot of questions. Let me try to answer a few.

  1. I tried but couldn’t get it to work in any other way.
  2. protection mode actually grants root priveledges. This is why the security rating is 1. It only is available in the community add on. Not on the other ssh add on.
  3. You need to restart the add on. And make the add on boot on start.

No comment on 3 and 5.

@PandJoKell
Copy link

Thanks bkbartk,
I'll restart the system and check the swap file sensors which are currently:
sensor.swap_free 0.0
sensor.swap_use 227.4
sensor.swap_use 100.0
Cheers.

@PandJoKell
Copy link

Terminal & SSH Configuration Options were set as follows:
authorized_keys: []
apks: []
password: ...............redacted
server:
tcp_forwarding: false
init_commands:

    if [ ! -f /backup/_swap.swap ]; then fallocate -l 2G /backup/_swap.swap &&
    mkswap /backup/_swap.swap && chmod 0600 /backup/_swap.swap && swapon
    /backup/_swap.swap ; elif [[ ! $(dmesg|grep _swap.swap) = swap\ on ]];
    then swapon /backup/_swap.swap; fi
    protection mode: disable

After a server reboot checked for swap space - no material change.
sensor.swap_free 0.1
sensor.swap_use 227.3
sensor.swap_use 99.9

Checked backup directory and no file _swap.swap
[core-ssh ~]$ cd ../backup
[core-ssh backup]$ ls -al
total 3337604
drwxr-xr-x 2 root root 4096 Jan 2 04:07 .
drwxr-xr-x 1 root root 4096 Jan 2 09:44 ..
-rw-r--r-- 1 root root 243210240 Jan 1 20:07 337e5272.tar
-rw-r--r-- 1 root root 276951040 Dec 28 20:06 428863cb.tar
-rw-r--r-- 1 root root 256624640 Dec 27 23:20 51177d83.tar
-rw-r--r-- 1 root root 331489280 Dec 30 04:07 5879cd2f.tar
-rw-r--r-- 1 root root 257064960 Jan 2 04:08 5a814985.tar
-rw-r--r-- 1 root root 258293760 Dec 28 04:05 8908753d.tar
-rw-r--r-- 1 root root 377118720 Dec 30 20:12 93d34ed5.tar
-rw-r--r-- 1 root root 317460480 Dec 29 20:07 9b635441.tar
-rw-r--r-- 1 root root 390932480 Dec 31 04:12 db0d8ce8.tar
-rw-r--r-- 1 root root 202373120 Dec 31 20:09 e02f9483.tar
-rw-r--r-- 1 root root 290304000 Dec 29 04:06 e43966ea.tar
-rw-r--r-- 1 root root 215859200 Jan 1 04:07 f8fd67df.tar

Checked Supervisor log and found schema issue with init_commands and protection mode:
22-01-02 10:06:09 WARNING (MainThread) [supervisor.addons.options] Option 'init_commands' does not exist in the schema for Terminal & SSH (core_ssh)
22-01-02 10:06:09 WARNING (MainThread) [supervisor.addons.options] Option 'protection mode' does not exist in the schema for Terminal & SSH (core_ssh)
22-01-02 10:06:09 WARNING (MainThread) [supervisor.addons.options] Option 'init_commands' does not exist in the schema for Terminal & SSH (core_ssh)
22-01-02 10:06:09 WARNING (MainThread) [supervisor.addons.options] Option 'protection mode' does not exist in the schema for Terminal & SSH (core_ssh)
22-01-02 10:08:56 WARNING (MainThread) [supervisor.addons.options] Option 'init_commands' does not exist in the schema for Terminal & SSH (core_ssh)
22-01-02 10:08:56 WARNING (MainThread) [supervisor.addons.options] Option 'protection mode' does not exist in the schema for Terminal & SSH (core_ssh)
22-01-02 10:15:02 WARNING (MainThread) [supervisor.addons.options] Option 'init_commands' does not exist in the schema for Terminal & SSH (core_ssh)
22-01-02 10:15:02 WARNING (MainThread) [supervisor.addons.options] Option 'protection mode' does not exist in the schema for Terminal & SSH (core_ssh)
22-01-02 10:15:11 WARNING (MainThread) [supervisor.addons.options] Option 'init_commands' does not exist in the schema for Terminal & SSH (core_ssh)
22-01-02 10:15:11 WARNING (MainThread) [supervisor.addons.options] Option 'protection mode' does not exist in the schema for Terminal & SSH (core_ssh)

Any thoughts gratefully received.

@JZ-SmartThings
Copy link

I wish this thread/bug could be closed because the solution/links in previous posts keeps getting further and further buried. Please do NOT use protected_mode as an entry in the config of the SSH addon. It's a checkmark/slider on the Add-on and you should see it on the Info page of most add-ons. It's not allowed as a YAML entry AFAIK. Screenshot below along with a link that should be used as your source (I've updated the command since the last post here) of the command/script and instructions. I hope people stop continuing to comment and pose questions here and instead go to the HA community to post linked below.

https://community.home-assistant.io/t/how-to-increase-the-swap-file-size-on-home-assistant-os/272226

image

@bkbartk
Copy link

bkbartk commented Jan 2, 2022

Protection mode: disable shouldn’t be Parton the init command. It should only be set with the toggle.

`init_commands:

    if [ ! -f /backup/_swap.swap ]; then fallocate -l 2G /backup/_swap.swap &&
    mkswap /backup/_swap.swap && chmod 0600 /backup/_swap.swap && swapon
    /backup/_swap.swap ; elif [[ ! $(dmesg|grep _swap.swap) = swap\ on ]];
    then swapon /backup/_swap.swap; fi
    packages: []
    share_sessions: false
    ssh:
    allow_agent_forwarding: false
    allow_remote_port_forwarding: false
    allow_tcp_forwarding: false
    authorized_keys: [] …..`

@PandJoKell
Copy link

PandJoKell commented Jan 5, 2022 via email

@JZ-SmartThings
Copy link

JZ-SmartThings commented Jan 5, 2022 •
edited
Loading

@PandJoKell please go to the link below and follow the instructions to a tee... specifically the name of the add-on that you're installing --- bullet 3. You want the unofficial SSH & Web Terminal not what you're trying to use:

https://community.home-assistant.io/t/how-to-increase-the-swap-file-size-on-home-assistant-os/272226

@bkbartk
Copy link

bkbartk commented Jan 5, 2022

image
you need the one in the green square,
not the other one, You can use both together, but for increasing the swap file you need the community add-on.
image
there you will find the toggle in the red square.

@PandJoKell
Copy link

PandJoKell commented Jan 5, 2022 via email

@PandJoKell
Copy link

PandJoKell commented Jan 5, 2022 via email

@ce3a
Copy link

ce3a commented Oct 17, 2022

This is all very nice, but I would still prefer a native solution that does not require us to install an unofficial addon. Looking through the documentation, it turns out that this OS isn't completely read-only. We can define network configs, kernel modules and more importantly, udev rules! And all those are persistent!

@Biont thanks for sharing this. I did something similar.

For now, I am using a RPI 4 with 1GB of RAM (8GB versions are hard to get currently) and a 240 GB SSD.
I decided to spend 16 GB for a SWAP partition (maybe thats too much, but I have plenty of disk space).

Here is what I did:

  • Resized the hassos-data partition to make space for a SWAP partition. Created the SWAP partition as partition number 9 at the end of the disk.
    • Lessens learned: don't use parted for that (HA will not boot), use fdisk instead.
  • In order to get ssh access to port 22222(!), created directory CONFIG inside the hassos-boot partition and copied my public key to file authorized_keys in this directory.
  • Launched HA and logged in to port 22222.
  • Ran mkswap /dev/sda9.
  • Rebooted, I checked that it actually worked:
    grafik

@BlackRapsbeere
Copy link 

* Launched HA and logged in to port `22222`.

* Ran `mkswap /dev/sda9`.

* Rebooted, I checked that it actually worked:
  ![grafik](https://user-images.githubusercontent.com/5941722/196285696-5d55d212-fabd-47ca-8881-2e9bd66cb258.png)

So you are telling the mkswap command stays persistent if referencing a whole partition?
Or do you additionally use udev-rules to swapon this partition?

The swapsize, location and swappiness definitely need to be editable by the user via homeassistant-GUI and get persistent.

@PandJoKell
Copy link

PandJoKell commented Oct 30, 2022 via email

@krystof-k
Copy link

krystof-k commented Nov 4, 2022 •
edited
Loading

@ce3a thank you, I tried it your way and it worked all fine on Raspberry Pi 3B+ v1.3.

Here is my approach step by step. I also upgraded the SD card to a larger one in the process.

You will need either a larger SD card with a USB reader or a USB drive (sized at least as the SD card you're using).

⚠️ Warning! First, I did it yesterday and I'm not sure this guide is 100% correct. Always consult Google in case you are not sure about something. Second, if you make a mistake (or I did), you can lose all your data, always backup first. No guarantees!

  1. In case you are exchanging SD card, connect both old and new to a computer and clone it (I did it using dd on macOS)
  2. Setup SSH access
  3. Connect the old SD card / USB drive
  4. Stop all Docker containers (docker kill $(docker ps -q))
  5. Unmount no. 8 (hassio-data) partition (check lsblk for the name): umount /dev/sda8
  6. In case you are using the USB drive, clone the partition to the drive: dd if=/dev/sda8 of=/dev/yourexternalpartition (or just mount the drive and copy the files)
  7. Now delete the no. 8 partition using fdisk: fdisk /dev/sda, d, 8
  8. Next, create a new (hassio-data) partition while leaving space for SWAP partition: n, p, 8, use the default start sector and then set the size for the data, e.g. +50G, N
  9. Create a new swap partition: n, p, 9, keep default start and end sector (fills up the rest of the space)
  10. Change type to swap for the partition: t, 9, swap
  11. Check the result p, write partitions and quit fdisk w
  12. Format the data partition mkfs.ext4 /dev/sda8
  13. Restore the data from the old SD card / USB drive: dd if=/dev/yourexternalpartition of=/dev/sda9 (or again just mount & copy the files)
  14. Create swap mkswap /dev/sda9
  15. Reboot and check Home Assistant booted correctly & is using your new swap partition

Hope this helps!

@hlfbt
Copy link

hlfbt commented Nov 7, 2022

@Biont

KERNEL=="sd?", SUBSYSTEM=="block", DEVPATH=="*usb1/1-1*", NAME="sda", RUN+="/usr/sbin/swapon /dev/sda2"

PS: Perhaps it can be improved by introducing partition labels. Maybe the devs would be open to automatically swapon any partition labelled hassos-swap? Just as an idea.

Thanks for the idea of using udev for this!
Changing your udev rule a bit makes it match on any swap partition with the hassos-swap label:

ACTION=="add", SUBSYSTEM=="block", ENV{ID_FS_TYPE}=="swap", ENV{ID_FS_LABEL}=="hassos-swap", RUN+="/usr/sbin/swapon /dev/disk/by-label/hassos-swap"

Adding the label to the swap partition on my external drive worked perfectly in combination with this rule.

@Vinz87
Copy link

Vinz87 commented Nov 13, 2022

Hi, did anyone succeed in using the swap command with the latest version 12.1.2 of the SSH & Terminal add-on?

@koger23
Copy link

koger23 commented Nov 13, 2022

Hi, did anyone succeed in using the swap command with the latest version 12.1.2 of the SSH & Terminal add-on?

Hi, this is the first time I try increasing swap, and addon version is 12.1.2 and does not work.

@svh1985
Copy link

svh1985 commented Nov 27, 2022 •
edited
Loading

It doesn't work with the 12.1.3 version of the plugin.

Please make it a part of the Native Home Assistant OS settings. WTH? 😜
Definitely helps to run Home Assistant on RPi 4 with just 2GB memory and an SSD.

@Vinz87
Copy link

Vinz87 commented Nov 28, 2022

actually it worked for me after updating to v12.1.3

@PandJoKell
Copy link

PandJoKell commented Nov 28, 2022 via email

@jlunz
Copy link

jlunz commented Jan 21, 2023

Here is how I did it, and since I failed for a few times I want to share my learnings.

My setup is a Raspberry Pi Zero 2W with HassOS 8.5 (#2220).
Image was flashed to a 32GB microSD-Card and booted once to confirm it is working and also that the resize on first boot happened.

I wanted to add a 1G Swap Partition after resizing 'hassos-data'.

Partition layout before:

Partition layout after resize of 'hassos-data' + adding a 1G Swap Partition:

I did the resize and re-partitioning on my Desktop with a card reader.
I tried different tools (Gnome Disks, parted), but all modified images did no longer boot, bootcode.bin was no longer found / executed (Activity LED stayed off + Checked with enabled BOOT_UART)

Wanting to know why fdisk works and parted not, I searched for an explanation.

tl;dr
parted automatically convert(s) the hybrid MBR to a conventional protective MBR, this is the problem.
You can examine the content with fdisk / gdisk, look for the option protective MBR data / hybrid MBR.

Click me for details before and after resize

Before resize

gdisk Disk\ Image\ of\ sdb\ \(2023-01-19\ 2055\).img.orig
GPT fdisk (gdisk) version 1.0.9

Partition table scan:
  MBR: hybrid
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with hybrid MBR; using GPT.

Command (? for help): p
Disk Disk Image of sdb (2023-01-19 2055).img.orig: 62333952 sectors, 29.7 GiB
Sector size (logical): 512 bytes
Disk identifier (GUID): B21260BC-1889-4BAC-A825-A0F789E4F9DC
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 62333918
Partitions will be aligned on 2048-sector boundaries
Total free space is 2014 sectors (1007.0 KiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048           67583   32.0 MiB    0C01  hassos-boot
   2           67584          116735   24.0 MiB    8300  hassos-kernel0
   3          116736          641023   256.0 MiB   8300  hassos-system0
   4          641024          690175   24.0 MiB    8300  hassos-kernel1
   5          690176         1214463   256.0 MiB   8300  hassos-system1
   6         1214464         1230847   8.0 MiB     8300  hassos-bootstate
   7         1230848         1427455   96.0 MiB    8300  hassos-overlay
   8         1427456        62333918   29.0 GiB    8300  hassos-data

Command (? for help): r

Recovery/transformation command (? for help): o

Disk size is 62333952 sectors (29.7 GiB)
MBR disk identifier: 0x00000000
MBR partitions:

Number  Boot  Start Sector   End Sector   Status      Code
   1      *           2048        67583   primary     0x0C
   2                     1         2047   primary     0xEE

After resize

gdisk Disk\ Image\ of\ sdb\ \(2023-01-19\ 2055\).img
GPT fdisk (gdisk) version 1.0.9

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.

Command (? for help): p
Disk Disk Image of sdb (2023-01-19 2055).img: 62333952 sectors, 29.7 GiB
Sector size (logical): 512 bytes
Disk identifier (GUID): B21260BC-1889-4BAC-A825-A0F789E4F9DC
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 62333918
Partitions will be aligned on 2048-sector boundaries
Total free space is 10126269 sectors (4.8 GiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048           67583   32.0 MiB    0C01  hassos-boot
   2           67584          116735   24.0 MiB    8300  hassos-kernel0
   3          116736          641023   256.0 MiB   8300  hassos-system0
   4          641024          690175   24.0 MiB    8300  hassos-kernel1
   5          690176         1214463   256.0 MiB   8300  hassos-system1
   6         1214464         1230847   8.0 MiB     8300  hassos-bootstate
   7         1230848         1427455   96.0 MiB    8300  hassos-overlay
   8         1427456        50255871   23.3 GiB    8300  hassos-data
   9        50255872        52209663   954.0 MiB   8200  

Command (? for help): r

Recovery/transformation command (? for help): o

Disk size is 62333952 sectors (29.7 GiB)
MBR disk identifier: 0x00000000
MBR partitions:

Number  Boot  Start Sector   End Sector   Status      Code
   1      *              1     62333951   primary     0xEE
Click me for details how to fix it

Recreate hybrid MBR using gdisk

gdisk Disk\ Image\ of\ sdb\ \(2023-01-19\ 2055\).img
GPT fdisk (gdisk) version 1.0.9

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.

Command (? for help): p
Disk Disk Image of sdb (2023-01-19 2055).img: 62333952 sectors, 29.7 GiB
Sector size (logical): 512 bytes
Disk identifier (GUID): B21260BC-1889-4BAC-A825-A0F789E4F9DC
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 62333918
Partitions will be aligned on 2048-sector boundaries
Total free space is 10126269 sectors (4.8 GiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048           67583   32.0 MiB    0C01  hassos-boot
   2           67584          116735   24.0 MiB    8300  hassos-kernel0
   3          116736          641023   256.0 MiB   8300  hassos-system0
   4          641024          690175   24.0 MiB    8300  hassos-kernel1
   5          690176         1214463   256.0 MiB   8300  hassos-system1
   6         1214464         1230847   8.0 MiB     8300  hassos-bootstate
   7         1230848         1427455   96.0 MiB    8300  hassos-overlay
   8         1427456        50255871   23.3 GiB    8300  hassos-data
   9        50255872        52209663   954.0 MiB   8200  

Command (? for help): r
Recovery/transformation command (? for help): h

WARNING! Hybrid MBRs are flaky and dangerous! If you decide not to use one,
just hit the Enter key at the below prompt and your MBR partition table will
be untouched.

Type from one to three GPT partition numbers, separated by spaces, to be
added to the hybrid MBR, in sequence: 1
Place EFI GPT (0xEE) partition first in MBR (good for GRUB)? (Y/N): N

Creating entry for GPT partition #1 (MBR partition #1)
Enter an MBR hex code (default 0C): 
Set the bootable flag? (Y/N): Y

Unused partition space(s) found. Use one to protect more partitions? (Y/N): N

Recovery/transformation command (? for help): o

Disk size is 62333952 sectors (29.7 GiB)
MBR disk identifier: 0x00000000
MBR partitions:

Number  Boot  Start Sector   End Sector   Status      Code
   1      *           2048        67583   primary     0x0C
   2                     1         2047   primary     0xEE

Recovery/transformation command (? for help): w

Final checks complete. About to write GPT data. THIS WILL OVERWRITE EXISTING
PARTITIONS!!

Do you want to proceed? (Y/N): y
OK; writing new GUID partition table (GPT) to Disk Image of sdb (2023-01-19 2055).img.
Warning: The kernel is still using the old partition table.
The new table will be used at the next reboot or after you
run partprobe(8) or kpartx(8)
The operation has completed successfully.

With all that said, creating the swap partition was all I did, it was recognized automatically.

[    7.616672] Adding 976892k swap on /dev/mmcblk0p9.  Priority:-2 extents:1 across:976892k SSFS
...
[    8.797583] Adding 109116k swap on /dev/zram0.  Priority:-3 extents:1 across:109116k SSFS
...

Please also note that this is just an experiment, for a robust and long running system you might want to adjust the swap priority.

@JZ-SmartThings
Copy link

@jlunz that looks like so much work and who knows what will happen when you update the core or the OS --- meaning are you going to have to go through this gyration again? Seems like way too much work when compared to my little technique, which just works and puts this swap need behind you. Try this:

https://community.home-assistant.io/t/how-to-increase-the-swap-file-size-on-home-assistant-os/272226

@jlunz
Copy link

jlunz commented Jan 22, 2023

@JZ-SmartThings Absolutely, I wouldn't recommend it to someone who is interested in a stable and long-term working system.
As I wrote, this is / was an experiment and I felt that sharing the hybrid MBR story was worth my reply.
(I couldn't find a lot of references explaining what was going on)

@JZ-SmartThings
Copy link

@jlunz definitely and very interesting actually. I just wouldn't do it due to complexity and unknown future up-keep :) kinda sad that we don't have this at the HASS OS level. More importantly, the default swap allocation is way too small for real-world use.

@PandJoKell
Copy link

Hi HAOS Swapfile fans,

Have been tardy in not trying the more permanent fix Swap file issue solution by @Biont's posting on 14 Oct 2021, mainly because of unfamiliarity with Linux udev for managing partitions. My HA system (HAOS installed version 9.5) runs on a RPI 3B+ with a 32GB SD card for system and data storage. I haven't tried mounting a USB SSD per @Biont's posting which has the side benefit of creating a new data partition labelled "hassos-data" during the migration procedure. Pending a better understanding of udev

I've been trying to re-create the swap area on HAOS startup per JZSmartthings posts over the last couple of years. It uses the community SSH & Web Terminal addon with an init_commands script to allocate a 2GB swap file in the Backup folder on startup. I couldn't get the community SSH & Web Terminal addon to save the script (both UI mode and YAML mode) and got an error message:
"Failed to save add-on configuration, Invalid list for option 'init_commands' in SSH & Web Terminal (a0d7b954_ssh). Got {'init_commands': 'if [ ! -f /backup/_swap.swap ]; then fallocate -l 2G /backup/_swap.swap && mkswap /backup/_swap.swap && chmod 0600 /backup/_swap.swap && swapon /backup/_swap.swap && echo SWAP_NEW_FILE_CREATED_USED ; elif [[ ! $(dmesg|grep _swap.swap) = swap\ on ]] ; then swapon /backup/_swap.swap && echo SWAP_USING_OLD_FILE ; else swapoff /backup/_swap.swap && swapon /backup/swap.swap && echo SWAP_FINAL_ELSE_STATEMENT ; fi', 'packages': [], 'share_sessions': True, 'ssh': {'allow_agent..."

I then tried running the individual commands via ssh checking the proper file creation occurred. The following are the command lines and return message:
'fallocate -l 2G /backup/_swap.swap'
'mkswap /backup/_swap.swap'
Return message was:
Setting up swapspace version 1, size = 2147479552 bytes
UUID=7a48b755-4066-471f-9d01-43e7d643ba1b"

Then 'chmod /backup//_swap.swap'
'ls -al' confirmed the file permission:
total 2097328
drwxr-xr-x 2 root root 4096 Jan 28 06:22 .
drwxr-xr-x 1 root root 4096 Jan 29 09:36 ..
-rw-r--r-- 1 root root 143360 Jan 25 16:39 269e861c.tar
-rw-r--r-- 1 root root 10240 Jan 28 06:22 6a6485d4.tar
-rw------- 1 root root 2147483648 Jan 27 09:06 _swap.swap
-rw-r--r-- 1 root root 10240 Jan 27 07:58 b684cf86.tar

I then tried to add the init_command 'swapon /backup/_swap.swap' to the community addon but got the error:
Failed to save add-on configuration, Invalid list for option 'init_commands' in SSH & Web Terminal (a0d7b954_ssh). Got {'init_commands': 'swapon /backup/_swap.swap', 'packages': [], 'share_sessions': True, 'ssh': {'allow_agent_forwarding': False, 'allow_remote_port_forwarding': False, 'allow_tcp_forwarding': False, 'authorized_keys': [], 'compatibility_mode': False, 'password': 'John131150', 'sftp': False, 'username': 'pkelloway'}, 'zsh': True, 'log_level': 'debug'}

The init_commands script worked without hitch a year or so ago but not now. So this post is to enquire if I may have done something wrong or maybe later versions of the Home Assistant Community Add-on: SSH & Web Terminal (version is 13.0.2) don't allow the swap script to be inserted? Have other HA swapfile fans had issues with the init_commands in the Community addon SSH & Web Terminal lately and if so has anybody found a way to make it?

@JZ-SmartThings
Copy link

@PandJoKell if you feel like troubleshooting it, please post it on the community thread on HA rather than GitHub. Post your screenshot of the YAML config. I'll also update mine and see if it breaks... so strange, but every time folks come with problems, turns out it's typically user error :) not insinuating, just informing.

@PandJoKell
Copy link

PandJoKell commented Jan 31, 2023 via email

@omelettedufromagee
Copy link

@jlunz You saved me.
Due to many circomstances, I lost the SWAP created from the init_commands and my instance was not able to load fully before it crashes due to memory issues.

I decided to create a Swap partition directly on the microSD card as @jlunz mentionned.

Below are the steps I did using resize2fs, fdisk and mkswap. It might help others in need.
I wish we could have different flavors of the HassOS img with SWAP or some boot config file to add more SWAP, at least enough to match the available RAM.

`omelette@omelette:~/Downloads$ sudo e2fsck -f /dev/sdc8
e2fsck 1.46.5 (30-Dec-2021)
Pass 1: Checking inodes, blocks, and sizes
Inode 135489 extent tree (at level 1) could be shorter. Optimize? yes
Pass 1E: Optimizing extent trees
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information

hassos-data: ***** FILE SYSTEM WAS MODIFIED *****
hassos-data: 144352/3874816 files (0.0% non-contiguous), 878750/15468283 blocks
omelette@omelette:~/Downloads$ sudo resize2fs /dev/sdc8 48G
resize2fs 1.46.5 (30-Dec-2021)
Resizing the filesystem on /dev/sdc8 to 12582912 (4k) blocks.
The filesystem on /dev/sdc8 is now 12582912 (4k) blocks long.

omelette@omelette:~/Downloads$ sudo fdisk /dev/sdc

Welcome to fdisk (util-linux 2.37.2).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

A hybrid GPT was detected. You have to sync the hybrid MBR manually (expert command 'M').

Command (m for help): p

Disk /dev/sdc: 59.69 GiB, 64088965120 bytes, 125173760 sectors
Disk model: SD Transcend
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 6433A305-2E32-4085-975B-138CE6B4F6B2

Device Start End Sectors Size Type
/dev/sdc1 2048 67583 65536 32M Microsoft reserved
/dev/sdc2 67584 116735 49152 24M Linux filesystem
/dev/sdc3 116736 641023 524288 256M Linux filesystem
/dev/sdc4 641024 690175 49152 24M Linux filesystem
/dev/sdc5 690176 1214463 524288 256M Linux filesystem
/dev/sdc6 1214464 1230847 16384 8M Linux filesystem
/dev/sdc7 1230848 1427455 196608 96M Linux filesystem
/dev/sdc8 1427456 125173726 123746271 59G Linux filesystem

Command (m for help): d
Partition number (1-8, default 8): 8

Partition 8 has been deleted.

Command (m for help): n
Partition number (8-128, default 8):
First sector (1427456-125173726, default 1427456):
Last sector, +/-sectors or +/-size{K,M,G,T,P} (1427456-125173726, default 125173726): +48G

Created a new partition 8 of type 'Linux filesystem' and of size 48 GiB.
Partition #8 contains a ext4 signature.

Do you want to remove the signature? [Y]es/[N]o: N

Command (m for help): n
Partition number (9-128, default 9): 9
First sector (102090752-125173726, default 102090752):
Last sector, +/-sectors or +/-size{K,M,G,T,P} (102090752-125173726, default 125173726): +4G

Created a new partition 9 of type 'Linux filesystem' and of size 4 GiB.

Command (m for help): t
Partition number (1-9, default 9):
Partition type or alias (type L to list all): swap

Changed type of partition 'Linux filesystem' to 'Linux swap'.

Command (m for help): w
The device contains hybrid MBR -- writing GPT only.

The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.

omelette@omelette:~/Downloads$ sudo mkswap /dev/sdc9
Setting up swapspace version 1, size = 4 GiB (4294963200 bytes)
no label, UUID=2b162ac5-3c33-457b-a7ee-bd30258d038a`

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests