Reduce memory usage of Fedora qubes

[marmarek]

How to file a helpful issue

The problem you're addressing (if any)

Recent Fedora versions (33, 34) require noticeably more RAM even when no application is running inside. For several qubes I have it at 700MB+, which reduce the number of qubes that can be started.

The solution you'd like

Review list of processes running inside and consider disabling by default some that are not needed - especially those irrelevant when running in a VM (various hardware support services for example).

The value to a user, and who that user might be

Smaller hardware requirements.

[DemiMarie]

I am running a Fedora qube that provides IPsec VPN services at 268MB.

[jevank]

I have such additional fedora excludes:
+--exclude=gnome-user-share
+--exclude=gnome-online-accounts
+--exclude=gnome-photos
+--exclude=gnome-control-center
+--exclude=gnome-contacts
+--exclude=gnome-calendar
+--exclude=gnome-maps
+--exclude=gvfs-goa
+--exclude=totem
+--exclude=annobin
+--exclude=xdg-desktop-portal*
+--exclude=gnome-remote-desktop

[marmarek]

I'd rather not uninstall additional stuff, but this may be a good hint what to look for to disable in autostart.

[brendanhoar]

User process nm-applet appears to be running in all Linux VMs whether the VM is configuring hardware or not.

Uses 80MB physical memory when not under paging pressure.

Outside of a sys-net, does it serve other purposes that lower level processes aren't already handling?

B

[DemiMarie]

@brendanhoar It does not. Also, to be fair, my firewall qube may be under significant paging pressure.

[brendanhoar]

The nm-applet use of significant RAM goes at least as far back to (non-minimal) fedora-29, where it's using 65MB. Fedora-29 is the earliest template I have installed on R4.0.

RAM used seems to increase as fedora version increases. :)

B

[skull-squadron]

Slimming templates is good.

Also, there is/was RAM page deduplication (transparent page sharing) (if carefully considering side-channel implications) and page compression from Oracle included up to Xen 4.7-testing. tmem_compress and tmem_dedup I think it was a mistake to remove tmen without considering constant-time alternative technical solutions and the benefits of reduced resource consumption of many nearly-identical VMs. Everything was so simple back under PVM. :o)

[marmarek]

tmem was way too complex to be security supported technology. It never went out of "experimental" stage, and there was only a single developer who understood its internals and quirks (who later stopped working on Xen, thus the feature was removed).

And generally, various memory deduplication technologies are very risky in terms of side channels.

[skull-squadron]

Yuck. Thanks for the info. I saw the patches removed from the Linux kernel and the discussions there, but I didn't realize the whole implementation was like playing with mud. I don't understand throwing terrible code over the wall in order to publish a relatively trivial paper.

Thoughtful, attack surfaces-mitigated dedupe and compression would be challenging, complicated, and interesting.

1. A Memory-Deduplication Side-Channel Attack to Detect Applications in Co-Resident Virtual Machines (2018) pdf

2. CATalyst: Defeating Last-Level Cache Side Channel Attacks in Cloud Computing (2016) pdf

3. Software Side Channel Attack on Memory Deduplication (2011) pdf

[DemiMarie]

A safer approach would be to share only data that is provided by dom0 and is strictly read-only, such as dom0-provided kernel and initramfs pages.

[skull-squadron]

? That doesn't make sense or provide any benefit.

…

On Thu, Nov 11, 2021 at 3:46 AM Demi Marie Obenour ***@***.***> wrote: A safer approach would be to share only data that is provided by dom0 and is strictly read-only, such as dom0-provided kernel and initramfs pages. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#7028 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABWYMFUOEWRMVGLN6PXBKDULOGHDANCNFSM5HCGKNUA> .

[DemiMarie]

? That doesn't make sense or provide any benefit.

It does if the kernel and initramfs are provided by dom0: they can be mapped read-only into every qube that uses them, without any need for deduplication.

[marmarek]

Deduplicating initramfs this way is not very helpful, it's freed early in the boot process anyway.

[skull-squadron]

I think this issue is best addressed in the Xen project. I'm sorry I brought it up. :)

…

On Thu, Nov 11, 2021 at 5:26 PM Marek Marczykowski-Górecki < ***@***.***> wrote: Deduplicating initramfs this way is not very helpful, it's freed early in the boot process anyway. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#7028 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABWYMHTXF3KZ35PXVIKXFDULRGLTANCNFSM5HCGKNUA> .

[DemiMarie]

I think this issue is best addressed in the Xen project. I'm sorry I brought it up. :)

Thanks for asking! This should be addressed in some sort of FAQ; it is by no means obvious.

[tlaurion]

Cross-linking some forum discussions directly linked to the need of increasing sys-net and sys-usb assigned memory for Fedora based service qubes, otherwise consistent weird issues happening:

• sys-usb webcam "disappearing": https://forum.qubes-os.org/t/librem-14-camera-sporadically-disconnecting-from-vms/12582
• sys-net driver issues and loss of networking capability : https://forum.qubes-os.org/t/large-consistent-volume-of-intranet-traffic-causes-networking-to-fail/13460/2

This is an example of why deploying Qubes ISO as is requires end users to search for the same bugs over and over, and why OEM (or certified resellers) can't simply deploy Qubes OS with the defaults. Or requires OEM/Organizations to maintain an OEM disk image and clone on their fleet to reduce redundant support requests (and then cryptsetup-reencrypt, which is not perfect since some UUIDs and seeds won't be unique), and why... those issues should be fixed upstream instead.

[DemiMarie]

I suggest masking the following user services:

• tracker-minier-fs.service
• tracker-miner-rss.service
• tracker-extract.service
• tracker-store.service
• tracker-writeback.service

Also, PackageKit should default off outside of templates, and CUPS and systemd-resolved should be opt-in.

[kalkin]

@DemiMarie There are people who are using tracker directly as a search or indirectly via e.g a Music Player.

[brendanhoar]

In the past folks have suggested distributing and installing minimal templates in the ISO but IIRC that was rejected due to DVD space constraints.

An alternative might be to, as part of install, clone the standard fedora-xx template, then script (salt?) some modifications (disable services, remove packages, other memory reduction changes, etc.) and name the clone something like fedora-xx-sys. Then use that for the sys-* components.

B

[DemiMarie]

@kalkin Tracker could be made controllable via qvm-service. Tracker is also a significant attack surface so it is best to not run it by default. It can be turned on explicitly in VMs that need it. sys-net and sys-usb shouldn’t.

[jamke]

I think that tracker packages and services should be removed at least from minimal templates.

I created the ticket here with reasons for this action:
#8403 - Remove GNOME Tracker from the Minimal templates of Fedora (e.g. fedora-38-minimal)

If user installs something manually that requires tracker (e.g. Nautilus) - it will be installed as dependency.
But currently it has no obvious reason to be in the minimal template in the first place.