prelockd

prelockd is a daemon that locks memory mapped executables and shared libraries in memory to improve system responsiveness under low-memory conditions.

What is the problem?

Would it be possible to reserve a fixed (configurable) amount of RAM for caches, and trigger OOM killer earlier, before most UI code is evicted from memory? In my use case, I am happy sacrificing e.g. 0.5GB and kill runaway tasks before the system freezes. Potentially OOM killer would also work better in such conditions. I almost never work at close to full memory capacity, it's always a single task that goes wrong and brings the system down.

— lkml

Why not fix the problem in the kernel?

Like not swap executable code when the system is near-OOM?

— www.phoronix.com

No caches means all executable pages, ro pages (e.g. fonts) are evicted from memory and have to be constantly reloaded on every user action.

It is indeed a difficult problem - some cached pages (streaming IO) will likely not be needed again and should be discarded asap, other (like mmapped executable/ro pages of UI utilities) will cause thrashing when evicted under high memory pressure.

— lkml.org

I would like to try disabling/limiting eviction of some/all file pages (for example exec pages) akin to disabling swapping, but there is no such mechanism. Yes, there would likely be problems with large RO mmapped files that would need to be addressed, but in many applications users would be interested in having such options.

— lkml.org

Once swap runs out, the kernel stops having a choice. It can only make room by reclaiming important caches. And this will turn bad quickly, as it will eventually include the executables/libraries that must be loaded as they are doing work!

So, we don't want to get the kernel into the situation where it must remove executables/libraries from main memory. If that happens, you can end up hitting the disk for every function call.

— lists.fedoraproject.org

Effects

• OOM killer comes faster (especially with noswap).
• Fast system reclaiming after OOM.
• Improved system responsiveness under low-memory conditions.

Demo

https://www.youtube.com/watch?v=vykUrP1UvcI

On this video: running fast memory hogs in a loop on Debian 10 GNOME, 4 GiB MemTotal without swap space.

• prelockd enabled: about 500 MiB mlocked. Starting while true; do tail /dev/zero; done: no freezes. The OOM killer comes quickly, the system recovers quickly.
• prelockd disabled: system hangs with while true; do tail /dev/zero; done.

See also https://youtu.be/fPnbnNX9CPE, https://youtu.be/O8QNnfb_Vm0.

Install

For Arch Linux there's an AUR package

Use your favorite AUR helper. For example,

$ yay -S prelockd
$ sudo systemctl enable --now prelockd.service

prelockd-git is also available.

To install on Debian and Ubuntu-based systems:

It's easy to build a deb package with the latest git snapshot. Install build dependencies:

$ sudo apt install make fakeroot

Clone the latest git snapshot and run the build script to build the package:

$ git clone https://github.com/hakavlad/prelockd.git && cd prelockd
$ deb/build.sh

Install the package:

$ sudo apt install --reinstall ./deb/package.deb

Start and enable prelockd.service after installing the package:

$ sudo systemctl enable --now prelockd.service

On other distros:

Install:

$ git clone https://github.com/hakavlad/prelockd.git && cd prelockd
$ sudo make install
$ sudo systemctl enable --now prelockd.service

Uninstall:

$ sudo make uninstall

How to configure

Edit the config (/etc/prelockd.conf or /usr/local/etc/prelockd.conf) and restart the service.

TODO

• Improve the documentation.

Report bugs and ask any questions

https://github.com/hakavlad/prelockd/issues