Linux install without Lima/qemu

[mgingell]

It might be useful to have a rancher-desktop that could install resources and tools locally, rather than using qemu VMs through Lima. This would allow rancher-desktop to operate on virtualized platforms such as VBox and vSphere/ESXi. The current version has no options for Lima to change qemu options to run on virtualized hardware and there are no obvious ways to install tools (nerdctl, etc) locally. Installing k3s and other associated tools is possible, but rancher-desktop could be a one-stop shop for this use case.

• Mike

[jandubois]

If I understand you correctly, you expect Rancher Desktop to manage VMs running on other hosts besides the current desktop/laptop. That would be outside the current scope of the project.

As for running nerdctl locally against a remote cluster, that isn't even possible, as nerdctl needs direct access to the file system managed by containerd. Even with Rancher Desktop, the "local" nerdctl is just a shim that executes the command inside the VM and relies on the filesystem mapping between the host and the guest machine to work.

[mgingell]

Thanks for replying, I guess I really didn't explain the use case very well.
Rancher-Desktop installs and uses Lima to launch and manage the lightweight VMs that hold the nerdctl executable among others. What I would like to be able to do is to configure Rancher-Desktop usage of Lima to launch the lightweight VMs (with qemu) with a different configuration that would allow qemu to run in a virtual machine (VM in VM) which it is capable of doing. In order to do that, I need to tell Lima to launch qemu with different cpu and machine/accelerator options. I can't seem to find any way to change those qemu command line options. There exists qemu options that are settable but not -cpu and -machine. Maybe this is not a rancher desktop issue but rather a lima issue. Thoughts?

[jandubois]

I need to tell Lima to launch qemu with different cpu and machine/accelerator options. I can't seem to find any way to change those qemu command line options.

There is a debugging feature in lima that would allow you to do that, e.g. you can override the -cpu choice like this:

$ QEMU_SYSTEM_X86_64="qemu-system-x86_64 -cpu Haswell-v4" limactl start

There is extra code in lima to make sure some options will not be added by lima if they were already passed in by the environment variable: https://github.com/lima-vm/lima/blob/master/pkg/qemu/qemu.go#L121-L142

I think I might still not fully understand what you are trying to do, but it does feels more like a lima issue/question to me right now.

[mgingell]

I am able to run qemu using the QEMU_SYSTEM_X86_64 env var to set specific cpu and machine options. Thanks for that information. How can I get rancher-desktop to use this for when it launches lima? I have tried /etc/environment and my .bashrc but it doesn’t seem to inherit or use the QEMU_SYSTEM_X86_64 var. From: Jan Dubois ***@***.***> Sent: December 7, 2021 1:08 AM To: rancher-sandbox/rancher-desktop ***@***.***> Cc: Mike Gingell ***@***.***>; Author ***@***.***> Subject: Re: [rancher-sandbox/rancher-desktop] Linux install without Lima/qemu (Discussion #1069) I need to tell Lima to launch qemu with different cpu and machine/accelerator options. I can't seem to find any way to change those qemu command line options. There is a debugging feature in lima that would allow you to do that, e.g. you can override the -cpu choice like this: $ QEMU_SYSTEM_X86_64="qemu-system-x86_64 -cpu Haswell-v4" limactl start There is extra code in lima to make sure some options will not be added by lima if they were already passed in by the environment variable: https://github.com/lima-vm/lima/blob/master/pkg/qemu/qemu.go#L121-L142 I think I might still not fully understand what you are trying to do, but it does feels more like a lima issue/question to me right now. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1069 (reply in thread)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABUSX34LAL23J5ZDBDAGHNLUPWJE3ANCNFSM5JLZBWNQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub> . <https://github.com/notifications/beacon/ABUSX353CMLSDHOKNWVVKY3UPWJE3A5CNFSM5JLZBWN2YY3PNVWWK3TUL52HS4DFWFCGS43DOVZXG2LPNZBW63LNMVXHJKTDN5WW2ZLOORPWSZGOAANOKSI.gif>

[jandubois]

How can I get rancher-desktop to use this for when it launches lima? I have tried /etc/environment and my .bashrc but it doesn’t seem to inherit or use the QEMU_SYSTEM_X86_64 var.

These changes will not work when you launch the app via Finder, which starts it via launchd...

All of this is far beyond the supported use case, but with enough determination you can make it work like this:

Set the environment variable inside the Info.plist file of the app:

$ cat ~/tmp/info-env.sh
#!/bin/sh

set -eux

PLISTBUDDY="/usr/libexec/PlistBuddy"
INFO_PLIST="/Applications/Rancher Desktop.app/Contents/Info.plist"

NAME="QEMU_SYSTEM_X86_64"
VALUE="qemu-system-x86_64 -cpu Haswell-v4"

"${PLISTBUDDY}" -c "Add :LSEnvironment dict" "${INFO_PLIST}"
"${PLISTBUDDY}" -c "Add :LSEnvironment:${NAME} string" "${INFO_PLIST}"
"${PLISTBUDDY}" -c "Set :LSEnvironment:${NAME} \"$VALUE\"" "${INFO_PLIST}"
"${PLISTBUDDY}" -c "Print :LSEnvironment" "${INFO_PLIST}"
$ ~/tmp/info-env.sh
+ PLISTBUDDY=/usr/libexec/PlistBuddy
+ INFO_PLIST='/Applications/Rancher Desktop.app/Contents/Info.plist'
+ NAME=QEMU_SYSTEM_X86_64
+ VALUE='qemu-system-x86_64 -cpu Haswell-v4'
+ /usr/libexec/PlistBuddy -c 'Add :LSEnvironment dict' '/Applications/Rancher Desktop.app/Contents/Info.plist'
+ /usr/libexec/PlistBuddy -c 'Add :LSEnvironment:QEMU_SYSTEM_X86_64 string' '/Applications/Rancher Desktop.app/Contents/Info.plist'
+ /usr/libexec/PlistBuddy -c 'Set :LSEnvironment:QEMU_SYSTEM_X86_64 "qemu-system-x86_64 -cpu Haswell-v4"' '/Applications/Rancher Desktop.app/Contents/Info.plist'
+ /usr/libexec/PlistBuddy -c 'Print :LSEnvironment' '/Applications/Rancher Desktop.app/Contents/Info.plist'
Dict {
    QEMU_SYSTEM_X86_64 = qemu-system-x86_64 -cpu Haswell-v4
}

This will break the code signatures, so if you are doing this with a signed release version, and not one you just built yourself, you will have to resign it. Using ad-hoc signature is ok, but you need XCode to get codesign, I think:

$ codesign --sign - --force --deep --preserve-metadata=entitlements "/Applications/Rancher Desktop.app"
/Applications/Rancher Desktop.app: replacing existing signature

The launch services database has cached this information, so refresh the database (this may take a little bit). Afterwards you need to reboot, or at least restart Finder to use the new settings:

$ /System/Library/Frameworks/CoreServices.framework/Frameworks/LaunchServices.framework/Support/lsregister -kill -r -domain local -domain system -domain user
$ killall Finder

If you now start Rancher Desktop again, it will use the settings:

$ grep qemu-system.*cpu "$HOME/Library/Application Support/rancher-desktop/lima/0/ha.stderr.log" | cut -c -160
{"level":"debug","msg":"qCmd.Args: [/Applications/Rancher Desktop.app/Contents/Resources/resources/darwin/lima/bin/qemu-system-x86_64 -cpu Haswell-v4 -machine q