macvtap CNI

This plugin allows users to define Kubernetes networks on top of existing host interfaces. By using the macvtap plugin, the user is able to directly connect the pod to a host interface and consume it through a tap device.

The main use cases are virtualization workloads inside the pod driven by Kubevirt but it can also be used directly with QEMU/libvirt and it might be suitable combined with other virtualization backends.

macvtap CNI includes a device plugin to properly expose the macvtap interfaces to the pods. A metaplugin such as Multus gets the name of the interface allocated by the device plugin and is responsible to invoke the cni plugin with that name as deviceID.

Deployment

The device plugin is configured through environment variable DP_MACVTAP_CONF. The value is a json array and each element of the array is a separate resource to be made available:

• name (string, required) the name of the resource
• lowerDevice (string, required) the name of the macvtap lower link
• mode (string, optional, default=bridge) the macvtap operating mode
• capacity (uint, optional, default=100) the capacity of the resource

In the default deployment, this configuration shall be provided through a config map, for example:

kind: ConfigMap
apiVersion: v1
metadata:
  name: macvtap-deviceplugin-config
data:
  DP_MACVTAP_CONF: |
    [ {
        "name" : "dataplane",
        "lowerDevice" : "eth0",
        "mode": "bridge",
        "capacity" : 50
    } ]

$ kubectl apply -f https://raw.githubusercontent.com/kubevirt/macvtap-cni/main/examples/macvtap-deviceplugin-config.yaml
configmap "macvtap-deviceplugin-config" created

This configuration will result in up to 50 macvtap interfaces being offered for consumption, using eth0 as the lower device, in bridge mode, and under resource name macvtap.network.kubevirt.io/dataplane.

A configuration consisting of an empty json array, as proposed in the default example, causes the device plugin to expose one resource for every physical link or bond on each node. For example, if a node has a physical link called eth0, a resourced named macvtap.network.kubevirt.io/eth0 would be made available to use macvtap interfaces with eth0 as the lower device

The macvtap CNI can be deployed using the proposed daemon set:

$ kubectl apply -f https://raw.githubusercontent.com/kubevirt/macvtap-cni/main/manifests/macvtap.yaml
daemonset "macvtap-cni" created

$ kubectl get pods
NAME                                 READY     STATUS    RESTARTS   AGE
macvtap-cni-745x4                      1/1    Running           0    5m

This will result in the CNI being installed and device plugin running on all nodes.

There is also a template available to parameterize the deployment with different configuration options.

Usage

macvtap CNI is best used with Multus by defining a NetworkAttachmentDefinition:

kind: NetworkAttachmentDefinition
apiVersion: k8s.cni.cncf.io/v1
metadata:
  name: dataplane
  annotations:
    k8s.v1.cni.cncf.io/resourceName: macvtap.network.kubevirt.io/dataplane
spec:
  config: '{
      "cniVersion": "0.3.1",
      "name": "dataplane",
      "type": "macvtap",
      "mtu": 1500
    }'

The CNI config json allows the following parameters:

• name (string, required): the name of the network. Optional when used within a NetworkAttachmentDefinition, as Multus provides the name in that case.
• type (string, required): "macvtap".
• mac (string, optional): mac address to assign to the macvtap interface.
• mtu (integer, optional): mtu to set in the macvtap interface.
• deviceID (string, required): name of an existing macvtap host interface, which will be moved to the correct net namespace and configured. Optional when used within a NetworkAttachmentDefinition, as Multus provides the deviceID in that case.
• promiscMode (bool, optional): enable promiscous mode on the pod side of the veth. Defaults to false.

A pod can be attached to that network which would result in the pod having the corresponding macvtap interface:

apiVersion: v1
kind: Pod
metadata:
  name: pod
  annotations:
    k8s.v1.cni.cncf.io/networks: dataplane
spec:
  containers:
  - name: busybox
    image: busybox
    command: ["/bin/sleep", "1800"]
    resources:
      limits:
        macvtap.network.kubevirt.io/dataplane: 1 

A mac can also be assigned to the interface through the network annotation:

apiVersion: v1
kind: Pod
metadata:
  name: pod-with-mac
  annotations:
    k8s.v1.cni.cncf.io/networks: |
      [
        {
          "name":"dataplane",
          "mac": "02:23:45:67:89:01"
        }
      ]
spec:
  containers:
  - name: busybox
    image: busybox
    command: ["/bin/sleep", "1800"]
    resources:
      limits:
        macvtap.network.kubevirt.io/dataplane: 1 

Note: The resource limit can be ommited from the pod definition if network-resources-injector is deployed in the cluster.

The device plugin can potentially be used by itself in case you only need the tap device in the pod and not the interface:

apiVersion: v1
kind: Pod
metadata:
  name: macvtap-consumer
spec:
  containers:
  - name: busybox
    image: busybox
    command: ["/bin/sleep", "123"]
    resources:
      limits:
        macvtap.network.kubevirt.io/dataplane: 1