Pick a working network interface for the bridged rd0 vmnet interface

[jandubois]

The host interface is specified in ~/Library/Application\ Support/rancher-desktop/lima/_config/networks.yaml:

networks:
  bridged:
    mode: bridged
    interface: en0

This must be set before the sudoers file is created:

$ grep en0 /etc/sudoers.d/rancher-desktop-lima
    /opt/rancher-desktop/bin/vde_vmnet --pidfile=/private/var/run/rancher-desktop-lima/bridged_vmnet.pid --vde-group=staff --vmnet-mode=bridged --vmnet-interface=en0 /private/var/run/rancher-desktop-lima/bridged.ctl, \

So the default configuration only works if en0 is connected to a network that provides a DHCP server.

Some suggestions to find the network automatically:

Using networksetup

This will work on older macOS versions like Mojave, but requires parsing plain text output:

$ networksetup -listnetworkserviceorder
An asterisk (*) denotes that a network service is disabled.
(1) Ethernet
(Hardware Port: Ethernet, Device: en0)

(2) Wi-Fi
(Hardware Port: Wi-Fi, Device: en1)

(3) Bluetooth PAN
(Hardware Port: Bluetooth PAN, Device: en4)
[...]

This provides all network services in priority order, and the device name.

Then query each device in turn:

$ networksetup -getinfo "Ethernet"
DHCP Configuration
Client ID:
IPv6: Automatic
IPv6 IP address: none
IPv6 Router: none
Ethernet Address: 38:c9:86:21:6d:c1
$ networksetup -getinfo "Wi-Fi"
DHCP Configuration
IP address: 192.168.17.21
Subnet mask: 255.255.0.0
Router: 192.168.17.1
Client ID:
IPv6: Automatic
IPv6 IP address: none
IPv6 Router: none
Wi-Fi ID: 28:f0:76:12:69:14

This shows that en0 has no IP address, and that en1 should be configured for the bridged network.

Using system_profiler SPNetworkDataType -json

The -json option is only supported on macOS Catalina and later. It is possible to parse the regular output, but feels rather brittle.

I don't see this documented, but it looks like the network devices are also returned in priority order:

$ system_profiler SPNetworkDataType -json | jq '.SPNetworkDataType[] | [._name, .interface, .ip_address]'
[
  "Ethernet",
  "en0",
  null
]
[
  "Wi-Fi",
  "en1",
  [
    "192.168.17.21"
  ]
]
[
  "Bluetooth PAN",
  "en4",
  null
]
...

This means we could select the device to use like this:

$ system_profiler SPNetworkDataType -json | jq -r 'first(.SPNetworkDataType[] | select(.ip_address) | .interface)'
en1

Roaming

This is out-of-scope for this issue, but I wanted to mention that we'll eventually have to deal with roaming too: A laptop can travel to different networks, so the bridged network address may change. Ethernet may be plugged in and Wi-Fi disconnected. Rancher Desktop should detect these situations and prompt the user to restart the VM to adapt.

[jandubois]

My recommendation would be to use systemprofiler ... -json on Catalina and later, and leave it to the user to manually update ~/Library/Application\ Support/rancher-desktop/lima/_config/networks.yaml with the correct interface on older versions. We do not promise to support any older versions anyways, so just providing a manual mechanism to keep them working is fine imo.

Lima already has the same restriction that proxy settings will not be auto-detected on older macOS version for exactly the same reason (it also uses system_profiler).

[jandubois]

Note that k3s will not work properly if rd0 is not working because we pass --flannel-iface rd0 to the server.

So maybe we also need to support the case when the host is offline and cannot create a bridged interface. In that case we should not specify the flannel interface, so k3s can use eth0 instead.

Alternatively we could switch to a "shared" network, but this is more tricky because we would want to switch back to "bridged" when a network is available again, but not if the switch to "shared" was done by the user and not RD. So maybe let's not go there yet.

[jandubois]

To avoid getting a sudo prompt every time the active network device changes, we should define a bridge for each one. Given

$ system_profiler SPNetworkDataType -json | jq -r '.SPNetworkDataType[] | select(.interface) | .interface'
en0
en1
en4
bridge0

we should add these networks in ~/Library/Application\ Support/rancher-desktop/lima/_config/networks.yaml:

networks:
  bridged_en0:
    mode: bridged
    interface: en0
  bridged_en1:
    mode: bridged
    interface: en1
  bridged_en4:
    mode: bridged
    interface: en4
  bridged_bridge0:
    mode: bridged
    interface: bridge0

Then when we detect that en1 is the active interface, we set this in lima.yaml (this part is already dynamically generated, because it is not used on Linux):

networks:
- lima: bridged_en1
  interface: rd0

This way the sudoers file only needs to be updated when the list of network devices changes.

If we don't find any active device, we omit the networks section from lima.yaml completely and also don't set the --flannel-iface rd0 arguments to k3s server.

[jandubois]

I don't see this documented, but it looks like the network devices are also returned in priority order:

I've changed the service order to move Wi-Fi ahead of Ethernet, and the system_profiler output reflects the updated order:

$ system_profiler SPNetworkDataType -json | jq -r '.SPNetworkDataType[] | select(.interface) | @text "\(._name):\(.interface)"'
Wi-Fi:en1
Ethernet:en0
Bluetooth PAN:en4
Thunderbolt Bridge:bridge0

[manno]

On my machine I need to use en8 (the one from route get default). However system_profiler reports:

LG Monitor Controls:usbmodem103NTPCK76082
USB 10/100/1000 LAN:en7
USB 10/100/1000 LAN 2:en8
Wi-Fi:en0
Bluetooth PAN:en6
Thunderbolt Bridge:bridge0

It works when I manually edit the lima.yaml and restart rancher-desktop.

[jandubois]

However system_profiler reports:

@manno Yes, but do the earlier entries have an IP address. I suggested this check to determine the interface to use:

system_profiler SPNetworkDataType -json | jq -r 'first(.SPNetworkDataType[] | select(.ip_address) | .interface)'

However, using route get default may be a better option indeed.

[manno]

system_profiler SPNetworkDataType -json | jq -r 'first(.SPNetworkDataType[] | select(.ip_address) | .interface)'

That works and returns only en8, which is the only one with an IP. (Wifi is off).

[mook-as]

Oddly, for me both en8 (wired ethernet) and en0 (Wifi) are DHCP clients with addresses, but I can't get DHCP acks over Wifi. I assume something odd is going on in my local network. At least en8 is also the preferred interface, so things will work given the direction we're currently going.

[jandubois]

Just for the record, one user on Slack could not get a DHCP address on en0, even though it was the active (and only) network connection they had. This happened on their office network, but not on their home network. We could not figure out what was causing this, and switched to a "shared" network to make RD useable for them.

I wonder if e.g. their Wifi setup does not allow multiple MAC addresses on a single radio (not really sure how this works). Or if the MAC addresses need to be on some kind of allow-list.